TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Anthropic has circulated a draft AI safety bill that could lead to a court-ordered halt of a model’s release if an independent evaluator identifies a foreseeable, imminent catastrophic risk. The described thresholds include a risk of more than 50 deaths or over $1 billion in property damage, tied to biological weapons, loss of developer control, or cyberattacks. The proposal is a draft, not enacted law, and key implementation details remain unsettled.
Anthropic is circulating a draft AI safety bill that could allow a court to halt an AI model’s release if independent evaluators find a foreseeable, imminent risk of more than 50 deaths or over $1 billion in property damage. The proposal, reported by Politico and covered by TechCrunch on October 20, is not law; it would give the U.S. Commerce Department a role in seeking a court order rather than allowing federal agencies to block a release on their own.
According to the reporting, the draft applies to catastrophic risks stemming from creating or releasing a biological weapon, a model evading its developer’s controls, or a cyberattack. The specified thresholds are more than 50 people at risk of dying or potential property damage exceeding $1 billion. The report says these figures resemble the thresholds used to define catastrophic risk in California’s recently enacted Senate Bill 53, although the two measures are separate policies.
The draft would rely on independent evaluators registered with the Commerce Department to assess whether a model presents an imminent catastrophic risk. The evaluators could be paid by AI developers, provided payment was not contingent on the results of their assessments, Politico reported. If evaluators identify an imminent risk, they would have 72 hours to notify Commerce. The department would then have to ask a judge in Washington, D.C., to halt distribution.
The proposal would expressly bar federal agencies from independently deciding to stop a model’s distribution, according to Politico’s account. Anthropic did not immediately respond to TechCrunch’s request for comment, but a spokesperson confirmed the draft to Politico and said it reflects proposals in the company’s Advanced AI Framework, which Anthropic has been advocating with federal and state lawmakers since June.
A Court Gate for High-Risk Releases
The draft would create a legal route to pause the release of a model before the specified harms occur, while reserving the final halt decision for a federal judge. That is a more formal intervention than voluntary company testing alone, but it is also narrower than a system in which a government regulator could directly block distribution. For developers, the proposal could make independent safety assessments and their findings consequential to release schedules.
Its significance depends on how the bill defines imminent risk and how the evaluation process operates. The reported thresholds focus on potential deaths and property damage linked to a limited set of pathways. The TechCrunch report notes that recent industry concern has also centered on agentic hacking and models acting against operator instructions. The draft’s references to control evasion and cyberattacks may encompass some of those worries, but the available account does not establish how particular incidents or model capabilities would be judged under the proposed standard.
As an affiliate, we earn on qualifying purchases.
Anthropic’s Framework and State Law
Anthropic’s spokesperson told Politico that the draft represents proposals in its Advanced AI Framework and that the company has advocated for them with lawmakers since June. The move comes amid continuing disputes over whether AI safety should be governed through new mandatory rules, voluntary company commitments, or some combination. TechCrunch reported that AI companies have often favored systems involving third-party evaluators embedded in or working with organizations, rather than broad mandatory oversight.
The draft’s thresholds have a parallel in California Senate Bill 53, which TechCrunch describes as recently passed into law and using similar figures in defining catastrophic risk. That comparison does not make Anthropic’s draft an extension of California law: the proposal is a separate federal legislative effort, and the report does not say that Congress has introduced or advanced it. The article also contrasts Anthropic’s approach with Nvidia CEO Jensen Huang’s recent argument that the industry does not need new laws, illustrating that technology companies do not share one regulatory position.
“The legislation posits certain scenarios in which the release of an AI model could be blocked from distribution.”
— TechCrunch’s account of the draft, citing Politico
Oversight Rules Remain Unsettled
The proposal is a draft being circulated, and the report does not establish that it has been formally introduced in Congress, received bipartisan support, or entered a legislative process. The specific tests for a “foreseeable, imminent risk,” the evidence evaluators would need, and how disputes over their findings would be handled are not detailed in the material provided.
It is also unclear how the system would protect evaluator independence if AI labs could pay them. The reported condition that payments not depend on evaluation outcomes addresses one potential conflict, but TechCrunch noted that the draft does not explain how that rule would prevent payment arrangements from influencing assessments. The scope of “evading” developer controls, the relationship between the proposal and existing federal authority, and the practical timing of court proceedings are also not specified in the account.
Anthropic did not immediately comment to TechCrunch. The report gives no response from other AI companies, lawmakers, the Commerce Department, or potential evaluators, so their positions on the draft remain unknown.
Congressional Consideration Is Unclear
The next step described is continued discussion with federal and state lawmakers; Anthropic’s spokesperson said the company has been advocating for the framework since June. Whether lawmakers will introduce, revise, or take up the draft is not established. Any bill would need to pass through the legislative process before its proposed evaluation and court-petition system could take effect.
Further details from Anthropic or lawmakers could clarify how evaluators would be registered, funded, and held accountable, as well as how Commerce would decide whether to seek an injunction. Until the draft’s text and its legislative status are clearer, the reported thresholds should be understood as part of a company-backed proposal, not an operative federal rule.
Key Questions
Would Anthropic’s proposal automatically ban an AI model?
No. As described in the reporting, an independent evaluator could identify an imminent catastrophic risk, after which the Commerce Department would ask a Washington, D.C., judge to halt distribution. The draft reportedly would not let a federal agency stop a release on its own.
What risks and thresholds does the draft describe?
The reported thresholds are a risk of more than 50 deaths or over $1 billion in property damage. The risks would have to stem from biological weapons, a model evading developer controls, or a cyberattack.
Who would evaluate the models?
The draft proposes independent evaluators registered with the Commerce Department. They could be paid by AI companies if payment did not depend on their findings, though the report says the proposal does not explain fully how evaluator independence would be protected.
Is this proposal already law?
No. The reported measure is a draft bill Anthropic is circulating. The source material does not say it has been formally introduced in Congress or enacted.
Source: rss
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
