Back

The mathematician Terence Tao recently declared that the era of heroes of mathematics is gone.

What prompted the reflection was OpenAI’s release of hundreds of mathematical proofs, including fully formalized proofs of long-standing, open math challenges. The field has entered a stage at which significant results are produced at a runway speed.

For many people, OpenAI’s success was a surprise. Only two years ago, observers mocked ChatGPT’s weakness when it attempted even basic arithmetic. What has changed?

Frontier models have become intelligent enough to learn from closed feedback loops that combine their large language models (LLMs) with automated verification programs. In other words, OpenAI (and the other frontier AIs) can check their work at lightspeed and keep working until they find a solution.

What has followed is a spectacular explosion of intelligence. In mere months, models have dramatically improved: from the level of a high-school student, through that of competition-winning high-school students, to ultimately rivaling the best living mathematicians, at least when it comes to proving theorems.

Mathematics is one of the easiest fields for this new intelligence to automate. Math is purely abstract. It does not involve the physical world. The physical sciences will be next, and I am certain that chemistry will be the first in line.

When that comes to pass, and it will be very soon, society will face new hard questions about AI in chemistry no less existential than the ones already flooding headlines.

Reinforcement Learning and the Verification Loop

Let’s take a closer look at the mechanisms of the new intelligence.

Frontier models are post-trained using a paradigm called reinforcement learning. AI models are “rewarded” when the verification loop confirms that they provided a correct answer and penalized when they get it wrong.

In mathematics this works really well because we can computationally check the correctness of proofs. That makes reinforcement learning very straightforward, so long as you have the compute to run the system at massive scale and incredible speed.

There was, however, the cold-start problem. If the model doesn’t generate any sensible answers, it receives no reward and cannot learn.

The resultant spike in AI intelligence sparked the explosion in mathematical intelligence. Every hour, major mathematical breakthroughs are announced. No one has ever witnessed anything like this. I would say it rivals putting a man on the moon.

What’s less appreciated is the consequence of this breakthrough in AI: Any field where a verification loop methodology works will be unrecognizable in the next few years, just as mathematics faces a new world that is hard even for the best mathematical minds to understand.

Why Chemistry is Next

Chemistry has two key attributes.

First, verifying a hypothesis in chemistry is relatively easy compared to many other physical sciences. Our ultimate goal in chemistry is always the same: make a molecule.1 Contrast this with biology, where the end goal is to cure patients, but translating results from an assay to patients is notoriously hard (hence the low single-digit success rates of clinical trials). Yes, automation in biological research is great, but the breadth of steps on the path to a cure is daunting.

In chemistry, the steps are also comparatively simpler: choose the molecule or reaction, come up with a hypothesis on how to proceed, test in a lab, and verify the results. The main challenge is simply scale - how quickly you can come up with ideas, figure out how to test them, run the experiments, and assess the results.

These challenges are not small. I have spent many years building a company focused on engineering this verification loop and the required automation. We have already surmounted the technical challenges to reinforcement learning, whereas biology, due to its complexity, has a long way to go.

Second, frontier models have improved their understanding of chemistry sufficiently to overcome the cold-start problem. AI has improved its general intelligence enough to efficiently learn about those parts of chemistry that are well-represented in public sources (textbooks, patents, papers), while datasets from automated labs can fill in any remaining gaps.

Just as in mathematics, getting past the cold-start problem is just the starting point.

I have seen the power of both verification loops and improved general understanding of chemistry firsthand. Over May and June this year, I led a project with OpenAI that exhibited the same characteristics in chemistry discovery that we are seeing in OpenAI’s high-velocity breakthroughs in mathematics.

We combined frontier AI systems (OpenAI’s GPT and Molecule.one’s Maria) with Maria’s high-throughput lab, where a verification loop involving more than 10,000 reactions not only produced a significant discovery in medicinal chemistry but also demonstrated that novel chemical research can happen at an unprecedented pace.

The OpenAI project took nine weeks from concept to finished paper. We are working on taking these results forward. In the interest of Open Science, I am keeping a diary of our progress. The key thing we observe is the constant acceleration of our ability to run the verification loop, which is a function of how quickly the AIs we use can develop ideas, design experiments, run them in a high-throughput lab, and test the results. The ultimate bottleneck is the physical lab, which can only run so fast, even at the microlitre scale at which we operate.

There is still work to do. Further automation of the verification process is possible and necessary. However, I believe the main challenge now is scaling the lab work. I believe there is no significant barrier now to R&D in chemistry taking off in the same way as mathematics.

The Rise of Dark Chemistry Fabs

How will this new world of chemistry - with important discoveries arriving daily - take shape in the physical world, the economy and society?

As I argued above, chemistry will enter the same phase that mathematics is in right now. In contrast with mathematics, however, the economic value of chemical discoveries is very significant and, most importantly, directly attributable. A chemical discovery can mean cheaper or faster synthesis of a major drug with a multibillion-dollar market.

As a result, there will be tremendous business pressure to build large automated laboratories controlled by AI designed to produce breakthrough discoveries with applications in manufacturing and further discovery.

These installations will be something new on the earth, not unlike the data center buildouts for AI, though perhaps not so ubiquitous. Given the enormous economic potential of new discoveries, there will be no shortage of capital to build these discovery factories.

Directly maximizing shareholder value will lead to reduced human oversight. There will be intense economic pressure to move humans aside because humans will always create bottlenecks that slow down profit-delivering discoveries. The economic incentive is straightforward. A better synthesis, invented faster, can reduce the cost of producing a valuable compound.

And before too long, scientists won’t really be needed for discovery. Currently, we are needed. However, the rapid progress of AI makes it clear to me, based on working for over a decade in the field, that this won’t be the case for long (though there is uncertainty regarding timelines).

We can think of them as Dark Chemistry Fabs: an industrialized lights-out, physical domain where discovery in chemistry proceeds night and day with no human oversight.

Unless we are very intentional about keeping humans in the loop, that is the future.

Addressing Autonomous Hacking And Other Risks

I already recognize part of that shift in my own work. The role increasingly resembles what you might call an “autonomous discovery project manager”: I choose worthwhile objectives, examine surprising results, and decide when the evidence warrants moving forward.

Before AI completely eclipses the need for human control over chemistry AI, we must determine how to guide and control the work of these Dark Chemistry Fabs. This is both a technical and societal challenge.

The risks are significant. The following is perhaps the most imminent risk.

The economic pressure will result in increasingly autonomous laboratories that are API-friendly (can be called by agents). These dark fabs will become a new thing for rogue agents to hack and use for potentially malicious purposes. Yes, we do not need an autonomous laboratory to make toxins (we know how to make them). But agents might need one (and might obscure the act by, for example, inventing less structurally familiar toxins).

This and other risks will require experts in the field to propose the human-governed guardrails for autonomous work in chemistry as well as national or international agreements to back those up. In parallel, we must make significant progress on aligning neural networks with our values, so that they optimize for the same goals as we do.

I raised some of these issues with the OPCW (Organisation for the Prohibition of Chemical Weapons) as its invited advisor two years ago, and I believe the time is now to speak up about it more broadly.

Conclusion

These thoughts emerged as I recognized my new role as a scientist working with AI: autonomous discovery project manager.

Working in this way day to day is weird but very enjoyable. I am partially automating myself (as I am a trained scientist). And I am learning a lot about chemistry, which I love.

I am grateful for what has come before: building the current state of science and technology piece by piece.

The next phase of science, however, will be different in ways that are hard to imagine. While it is tempting for me, as a scientist, to say that things will play out in a less frightening way, I believe they won’t.

Chemistry finds itself in a special position. We have to understand that it will be the first among the physical sciences to follow a trajectory similar to that of mathematics.

Chemistry will also be unique. It will be the first to give rise to a Dark Fabs phase of science. And we have to enter this new era responsibly and show the way.

The catastrophic risk is that due to economic pressures and the drive to maximize shareholder value, we will prematurely remove humans from the process. What worries me most is the increasing likelihood of rogue agents hacking our increasingly automated laboratories.

Acknowledgments

Many thanks to Abigail Doyle, Peter Hotchkiss, Ash Jogalekar, Connor Coley, and colleagues at molecule.one (especially Piotr Byrski and Paulina Wach) for feedback about this essay.

There are two more nuanced points I would like to make here. First, definitely, the goal of chemistry is also to build scientific understanding. A similar distinction has been made in mathematics, and I fully agree that both are important. Second, I want to be clear that there are significant parts of chemistry that are not yet automation-friendly. However, I predict that with economic pressure and AI-led advancements, automation-friendly chemistry will develop as a field, both due to hardware advancements and scientific advancements such as developing novel conditions for automation-friendly chemistry.