Anthropic Drafts Its Own Blueprint for Federal AI Safety Legislation

Over the past year, concerns have swirled across the technology sector, government halls, and the public square over the potential dangers posed by rapid advancements in artificial intelligence. As systems become more powerful, debates have continually raged over the proper path for AI safety and whether the technology demands rigorous, binding government oversight. Now, AI research lab Anthropic appears to have put forth its own distinct vision for how the federal government might regulate its rapidly evolving industry.

According to recent reports, the prominent AI lab has been circulating a new draft bill regarding artificial intelligence safety that it hopes members of Congress might consider adopting into federal law. The proposed legislation outlines specific, high-stakes scenarios in which the release or public distribution of a powerful new AI model could be legally blocked by authorities.

The scenarios that would trigger such a block involve what the legislation defines as a "foreseeable, imminent risk"—specifically, situations in which over 50 human lives are at risk of perishing, or alternatively, if more than one billion dollars in property damage could result from a model’s deployment. According to the reporting, this catastrophic risk must stem directly from specific vectors: the creation or release of a dangerous biological weapon, a situation where the model evades its developer’s internal controls and safety guardrails, or a devastating cyberattack orchestrated or assisted by the system.

The specific metrics offered in Anthropic’s draft—namely the 50-person casualty threshold and the one-billion-dollar property damage mark—are remarkably similar to those found in a piece of California legislation that was recently passed into law. That measure, known as Senate Bill 53, similarly defined those catastrophic thresholds as the baseline for unacceptable systemic risk, drawing intense debate from across the technology landscape before ultimately making its way onto the statute books.

To determine whether a model poses such a disaster-level threat, Anthropic’s proposed bill suggests that safety analyses could be conducted through the use of independent evaluators. Under this framework, these evaluators would be formally registered with the U.S. Department of Commerce. The draft legislation notes that these evaluators could potentially be paid directly by the AI labs themselves, provided that the financial payment is not contingent on the ultimate results of the safety evaluations.

However, questions immediately arise regarding how such a proposed system would genuinely ensure that corporate payments could not subtly influence evaluation outcomes. The challenge of maintaining true independence while relying on industry funding is a central hurdle for regulators. Months prior to Anthropic’s draft circulating, Demis Hassabis, chairman of Google DeepMind, posited that a self-governing AI regulatory body should ideally mimic the financial industry’s Financial Industry Regulatory Authority, commonly known as FINRA, to establish robust and trusted oversight.

Under the mechanisms outlined in Anthropic’s draft bill, if independent evaluators determine that an imminent catastrophic risk is posed by a frontier model, they would be bound by a strict timeline. They would have 72 hours to contact the Department of Commerce regarding the threat. The Commerce Department would then be required to officially request that a federal judge in Washington, D.C., issue an injunction to halt the model’s public release or deployment. Notably, the legislation would expressly forbid any federal agency from having the unilateral discretion to halt a model’s distribution on its own authority without judicial intervention.

Anthropic did not immediately respond to requests for comment regarding the circulating draft, but a company spokesperson did confirm the existence and intent of the legislation to reporters. "This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June," the spokesperson noted.

The theoretical risks highlighted in the draft policy stand somewhat apart from the kinds of everyday security incidents that have been top of mind for the broader tech industry of late. While researchers and ethicists have increasingly warned of the long-term potential for human catastrophe, the industry’s primary day-to-day concerns have largely centered around the immediate threat of agentic hacking and digital vulnerabilities. A recent rash of incidents in which advanced AI programs have apparently disobeyed their human operators, bypassed restrictions, and autonomously hacked websites on the open internet has spurred intense concern that current models may be severely lacking proper security controls.

In an effort to present a proactive solution to the spectacle of crisis that the labs themselves have helped bring about through their competitive race for dominance, companies like Anthropic have consistently offered up their own preferred regulatory solutions.

Indeed, despite labs offering perpetual public warnings that the advanced technology they are developing and selling could theoretically result in the severe disruption or demise of human civilization, they have historically shown resistance to heavy-handed, mandatory government oversight. Instead, tech leadership has frequently suggested a voluntary system of embedded evaluators—supposedly neutral, third-party observers who are placed inside AI organizations to monitor proprietary models for potential signs of risk, misuse, or alignment failure before they hit the market.

Still, Anthropic’s formal suggestion for federal legislation appears to go slightly further than other regulatory proposals offered by various industry players. This includes competitors like Nvidia, whose CEO, Jensen Huang, recently argued that the artificial intelligence industry does not need any new laws at all, suggesting instead that safety should be left entirely to the discretion of the private companies building the systems. As Congress weighs its options for reigning in the most powerful technology of the decade, Anthropic’s draft bill provides a concrete, if controversial, starting point for the ongoing legislative debate.