Policy on the AI Exponential
Another (short) essay by Dario Amodei
Dario Amodei published a relatively short1 essay last week called “Policy on the AI Exponential.”2 It’s good and you can read it in about 10-15 minutes. This essay was published two days before the US government pulled the plug on Fable, but I only just read it myself today (4 days after Fable’s takedown).
As with Dario’s previous (longer) essays this one’s a good read. Given what happened two days after this essay was published, I had to pause and read this section twice.
It is time to go beyond transparency to more serious and binding regulation of AI. I believe the best analogy, at least at the current stage of the exponential, is to cars, airplanes, or drugs—powerful technologies essential to the modern economy, but capable of killing large numbers of people if designed or operated poorly. I therefore believe we should model AI regulation on agencies like the Federal Aviation Administration (FAA). Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety. I am grateful to see the Trump administration’s Executive Order move incrementally towards a greater role for government in AI, though Anthropic’s proposal recommends even further action. Our proposal includes the following elements:
Models above a threshold of compute should undergo mandatory testing by a qualified third party for their level of risk in four specific areas: cybersecurity, biological weapons, loss of control of AI systems, and automated R&D that could accelerate these other risks.
The government should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks. This power must be scoped to the above four specific risks and there must be protective measures against political favoritism or arbitrary decisions.
Relatively short in the sense that it’s a fraction of the length of The Adolescence of Technology or Machines of Loving Grace (both worth reading in full).