This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.
For those of you enjoying your summer unaware of Anthropic’s latest feud with the US government, here’s a recap: In April the company said it had built an AI model called Mythos that was so good at working with code it could pose a global cybersecurity risk. Anthropic gave access to a small group of cybersecurity specialists so they could understand what they were up against. Then it released a modified version called Epic which it said was safer to the public on Tuesday, June 9. That Friday, the government told the company it was a threat to national security and placed export controls on the new release. Anthropic revoked access to both models hours later.
People worried about catastrophic outcomes of AI — broadly labeled “doomers” — have said for years that the technology poses a risk to humanity and published proposals for how the government should intervene in its development. The doomers just got their government intervention — not over a bioweapon or rogue AI, but in response to an AI model that is basically just really good at coding. And the outcome so far looks less like a security plan than a superficial response.
There’s a lot to dissect about what happened in those few days that led to such drastic action from the government, and it’s worth noting that Amazon CEO Andy Jassy was the one who told government officials that Epic could be dangerous (Amazon is both invested in Anthropic and building its own competing AI models). It’s also possible this could be a short-lived ban from the government that doesn’t survive legal scrutiny (it’s unclear that Anthropic offering access to Epic actually counts as “exporting” it, for example).
But there are ripple effects happening already.
For one, this is making a lot of people not want to rely on American AI companies. French politician Bruno Retailleau described it as a “wake-up call” that should motivate Europe to build more AI. But any vision of turning Paris into Silicon Valley — touted by many European leaders following the shutdown of Anthropic’s models — is complicated by one key factor: China.
Open-source models from China are very capable and incredibly cheap, and they can be downloaded to run on anyone’s servers with no rules or guardrails. (This makes them ideal for companies that don’t want access turned off on the basis of a decision from the White House — but equally ideal for cybercriminals, the kind that Anthropic hoped to fend off by building security guardrails into its models.)
It’s possible that companies, including those in the US and Europe, will decide that working with Chinese models is just easier, as the skyrocketing stock of Chinese startup Zhipu suggests. Playing this forward, is it possible the government’s next drastic decision will be to claim that US companies using models from China pose a threat to national security? I wouldn’t rule it out.
Second, it’s possible that shutting off access to Anthropic’s models will leave the country more vulnerable to cybersecurity attacks, not less. Leading cybersecurity experts have said as much in an open letter to the government, writing that access to Anthropic’s models was helping researchers prepare defenses, and that the company’s models are no more dangerous than a host of leading models which are widely available. Such is the risk of applying the concept of nonproliferation to software — trying to contain and restrict dangerous AI models in the manner of the uranium standard for nuclear weapons.
The third thing worth watching is how US lawmakers will react. Consider that following Anthropic’s last feud with the government over how the Pentagon could or couldn’t use its models, a slate of new bills was introduced that would define the boundaries of military AI.
Right now, the key players shaping how AI gets regulated are the companies and the White House. There’s been a lot of focus on more federal AI legislation, and polling suggests most Americans want it. Lawmakers are still figuring out whether to create rules on how children use chatbots and are far from a clear answer on the extent to which the government should vet the safety of AI models. But with every drastic action from the White House, the pressure for regulation rises.
To state the obvious, predictions are tricky when the administration’s attitudes toward AI change with the wind. When President Trump took office, he threw out the restrictive rulebook for how to make AI safe and promised to get out of the way of tech companies. The White House has now called the most expensive AI startup a threat to national security once in the spring, and again in summer. What will fall bring?
