A group at Microsoft claims to have utilized artificial intelligence to uncover a “zero day” flaw within the biosecurity frameworks intended to thwart the abuse of DNA.
These screening frameworks aim to prevent individuals from acquiring genetic sequences that could potentially be employed to synthesize lethal toxins or pathogens. However, researchers led by Microsoft’s chief scientist, Eric Horvitz, assert that they have found a method to circumvent these safeguards in an unprecedented manner.
The team presented their findings today in the journal Science.
Horvitz and his team focused on generative AI algorithms that generate novel protein structures. These types of applications are already driving the search for new medications at well-capitalized startups like Generate Biomedicines and Isomorphic Labs, which is a spinoff from Google.
The issue is that such systems possess the potential for “dual use.” They can leverage their training datasets to produce both beneficial and harmful molecules.
Microsoft stated that it initiated a “red-teaming” evaluation of AI’s dual-use capabilities in 2023 to assess whether “adversarial AI protein design” could assist bioterrorists in creating toxic proteins.
The protection that Microsoft targeted is referred to as biosecurity screening software. To create a protein, researchers generally need to obtain a matching DNA sequence from a commercial supplier, which can then be introduced into a cell. Those suppliers utilize screening software to align incoming requests with known toxins or pathogens. A close match triggers an alert.
To orchestrate its attack, Microsoft employed multiple generative protein models (including its own, named EvoDiff) to alter toxins—modifying their structure in a manner that allowed them to evade screening software while preserving their lethal function.
The researchers maintain that the process was entirely digital and that they did not produce any dangerous proteins. This was to eliminate any impression that the organization was developing bioweapons.
Prior to releasing the results, Microsoft claims it informed the US government and software developers, who have already updated their systems, although some AI-generated molecules might still go undetected.
“The patch is not comprehensive, and advancements in technology are ongoing. But this is not a one-time effort. It marks the beginning of further testing,” states Adam Clore, director of technology R&D at Integrated DNA Technologies, a significant DNA manufacturing firm, who co-authored the Microsoft report. “We find ourselves in a sort of arms race.”
To ensure that no one misuses the research, the researchers indicate they are withholding some of their code and did not disclose which toxic proteins they asked the AI to modify. Nonetheless, certain hazardous proteins are well known, such as ricin—a toxin from castor beans—and the infectious prions responsible for mad-cow disease.
“This discovery, along with rapid advancements in AI-driven biological modeling, highlights the clear and pressing requirement for improved nucleic acid synthesis screening methodologies accompanied by a dependable enforcement and verification system,” asserts Dean Ball, a fellow at the Foundation for American Innovation, a think tank in San Francisco.
Ball points out that the US government already regards the screening of DNA orders as a vital element of security. Last May, in an executive order focusing on biological research safety, President Trump advocated for a comprehensive overhaul of that system, although the White House has yet to unveil new recommendations.
Some experts question whether commercial DNA synthesis is the optimal defense against malicious actors. Michael Cohen, an AI safety researcher at the University of California, Berkeley, posits that there will always be methods to mask sequences and that Microsoft could have made its evaluation more challenging.
“The challenge seems feeble, and their updated tools fail often,” remarks Cohen. “There appears to be a reluctance to acknowledge that shortly, we’re going to need to withdraw from this perceived choke point, so we should begin to search for positions that we can truly secure.”
Cohen argues that biosecurity should likely be integrated into the AI systems themselves—either directly or through controls over the information they provide.
However, Clore argues that monitoring gene synthesis remains a practical method for identifying biothreats, as the production of DNA in the US is largely controlled by a handful of companies that maintain close collaboration with the government. In contrast, the technologies utilized to develop and train AI models are more broadly accessible. “You cannot reverse that situation,” claims Clore. “If you possess the means to deceive us into producing a DNA sequence, it is likely you can also train a large language model.”
