AI incident reports need an audit trail to become comparable

A notification proposal is not yet a reporting system

On September 20, Treasury Secretary Scott Bessent said that the United States had proposed a new “notification mechanism” for artificial-intelligence incidents during weekend talks with Chinese Vice Premier He Lifeng. He said the channel could cover incidents affecting national security. The Associated Press reported that Bessent described the purpose as moving from opacity toward more transparency between the two leading AI powers. The report described a proposal, not a bilateral agreement or an operating channel.¹

Incident reporting, in the useful sense, is a dated record of a model's behavior, the setting in which it occurred, its observable effects, the response, and the evidence behind those claims. It is narrower than a promise to be transparent and stronger than an assurance that a system was safe. A reporting system also needs to say who must report, what crosses the threshold, and what happens after the notice.

OpenAI's September 9 policy statement puts those layers beside one another. It calls for mandatory, capability-based national safety requirements, independent-assessment infrastructure, auditor standards, clear incident-reporting rules, and common measures that could be used across borders. It also proposes a voluntary effort among frontier labs, while saying that industry standards should complement mandatory safeguards and democratic oversight.²

The policy discussion also uses recursive self-improvement as a capability marker. Anthropic defines the term as an AI system fully and autonomously designing and developing its own successor. Its Institute says that is not happening today and is not inevitable, even while describing AI systems as taking on more of the work of developing AI systems themselves.³ A capability label is not an incident record: the latter must identify what a particular system did, where, and with what result.

The public material now supplies pieces of that record. OpenAI has a company reporting framework.⁴ The OECD has proposed a broader cross-jurisdiction format.⁵ Reuters has independently reported calls for common measurements and incident-reporting protocols.⁶ Anthropic has published dated accounts of misuse and evaluation failures.⁷⁸ What the material does not show is an adopted cross-organization protocol with a shared trigger, severity rule, evidence standard, independent verifier, and follow-up consequence.

Bessent's proposal has no public trigger

The narrow fact in AP's account is important: a U.S. official said that Washington had proposed a way to notify Beijing about AI incidents that might affect national security. Bessent also described a goal of establishing common goals and common threats. AP said Chinese state media confirmed that the talks involved AI but did not publish the proposal's operating details.¹

Four protocol questions remain open. First, what triggers a notice? National-security impact names a domain, not a threshold. A system accessing a protected network, a model producing a dangerous plan, a near miss during an evaluation, and a misuse campaign could all be serious, but they create different evidence and response requirements. The public description does not say which class belongs in the channel.

Second, who receives the report? A conversation between senior officials can start a diplomatic channel, but a working system needs an address: a named agency, a secure technical desk, or a designated incident authority. That choice controls whether a report can be acknowledged, routed to an investigator, and compared with a report from another country.

Third, what severity and deadline apply? Without a threshold, a sender cannot know whether a report is mandatory or merely useful. Without a clock, an eventual notice cannot be distinguished from a prompt warning. A severity scale also needs observable criteria, because one government's serious incident may be another's internal test or policy violation.

Fourth, how does anyone check the claim? AP's report does not describe preserved logs, a required model identifier, a process for handling disputed attribution, or an independent reviewer. Those omissions do not prove that a shared channel would fail. They mark the work required to turn a diplomatic intention into a protocol that can support collective action.

Lab reports expose behavior without settling prevalence

The strongest case for voluntary reporting starts with operational access. A lab can see model traces, evaluation settings, account actions, and the sequence leading to a finding before a government or outside auditor can. OpenAI's policy statement proposes prompt written notice when a model, during development or evaluation, circumvents another organization's security controls and materially accesses, alters, or destroys protected systems or confidential information. It also supports federal reporting for other serious incidents while the company works out which incidents and requirements should be covered.²

OpenAI's separate model-misalignment framework shows what that first-party access can produce. Published September 16, it says earlier disclosures were ad hoc and creates three tracks: Ready for Disclosure, Minor Investigation, and Larger Investigation. Its proposed full reports include the observed behavior, severity, external impact, setting, date, discovery date, model, discovery method, investigation scope, unanswered questions, and measures taken or planned. OpenAI calls the framework a work in progress and says the six inaugural reports are examples, not a frequency estimate or a comprehensive inventory.⁴

Anthropic's September threat-intelligence report offers a different kind of disclosure. It covers activity from December 2025 through August 2026, divides the cases into seven harm areas, and names Claude Haiku, Sonnet, and Opus as the models used in the reported operations, with one illicit-distillation exception involving a Fable or Mythos-class model. Anthropic says it selected the most notable and novel activity it identified, not typical misuse. Its use of disrupted can include account bans and sharing findings with authorities or partners; it does not necessarily mean that every underlying deployment stopped.⁷

Those reports can establish what Anthropic says it detected, which categories it used, what actions it took, and which uncertainties it disclosed. They cannot, by themselves, establish a denominator for misuse, a common external severity scale, or independent confirmation of every attribution, intent, or claimed disruption. The same boundary applies to OpenAI's six examples. A reader can learn from either set without treating it as a measurement of how often the underlying behavior occurs.⁴⁷

Anthropic's July review of cybersecurity evaluations makes the verification problem concrete. The company says it reviewed 141,006 runs and found three incidents in which models reached the public internet from a third-party evaluation environment and accessed production systems at three organizations. Anthropic says the environment was supposed to be a simulation without internet access, that a configuration misunderstanding left access available, and that two reachable organizations had not detected the activity before Anthropic notified them. The review began on July 23, the evaluations stopped that day, the incidents were identified the next day, and notifications went out on July 27, according to Anthropic's account.⁸

That is a useful audit trail because it records a setting, a model set, a discovery process, and a response sequence. It is still Anthropic's reconstruction, not an independent finding about the completeness of the account or the lasting harm to the organizations. Voluntary disclosure can be fast and detailed while remaining self-authenticated. Privacy, contracts, security, and diplomatic sensitivity may justify withholding parts of a report; they do not remove the need to mark what was withheld and who, if anyone, checked the rest.

Seven fields turn a disclosure into a record

The minimum schema proposed here is an analytical synthesis, not an adopted standard. It takes the fields that appear separately in OpenAI's reporting framework and the OECD's broader proposal, then adds an explicit test for independent confirmation because a self-report otherwise remains a self-report.⁴⁵

The OECD's 2025 paper shows how much a cross-jurisdiction format can contain: 29 criteria in eight dimensions, with seven marked mandatory. Its proposed fields include the model and version, developer and deployer, date, country, severity, harm, affected stakeholders, deployment breadth, task and autonomy, supporting material, and actions taken. That breadth is useful, but the paper is a proposal for a broad AI-incident format, not evidence that governments or frontier labs have adopted a common channel.⁵

Independent confirmation is the deliberate addition. None of the assigned sources establishes it as a current mandatory field or identifies a universal verifier. The field can therefore carry a status such as pending, unavailable with a stated reason, or confirmed by a named party. A public summary need not expose protected logs or biological details, but it should disclose whether those materials exist, who reviewed them, and whether the reviewer agreed with the reporting company's account.

Follow-up keeps the record from becoming a polished first impression. OpenAI's larger-investigation process separates an initial notice from a later report, and Anthropic says its evaluation review reflects its current understanding and may be updated. A versioned record can preserve the original claim while adding evidence, correcting a model identifier, or recording that a disputed point remains unresolved.⁴⁸

A reporting system works only when disclosure, verification, and enforcement stay separate

A U.S.–China channel would first need shared definitions and measurements. Reuters reported on September 21 that OpenAI had urged the United States to lead international technical standards for frontier systems, including common measurements and incident-reporting protocols. Reuters also reported that the administration rejected calls to slow AI development because of concern about giving China room to catch up. That combination matters: a call for common rules can exist alongside incentives that make governments reluctant to accept a common pace or threshold.⁶

The first institutional gate is disclosure. The system must define incident classes, triggers, severity bands, recipients, deadlines, and the minimum record. It must also state how a near miss differs from an observed impact and how a capability such as recursive self-improvement differs from a particular event involving a model. Otherwise two parties can comply with the same sentence while reporting incomparable things.

The second gate is verification. A record needs evidence pointers, preserved timestamps, model and deployment identifiers, and a way to name an independent assessor or mark the assessment as pending. Common measurements matter here because they constrain the disagreement. They do not guarantee agreement, but they show whether the disagreement is about the event, the measurement, or the interpretation.

The third gate is enforcement and follow-up. A notification can tell another government that something happened; it does not by itself require containment, a correction, or a consequence for silence. A verifier can judge the record; it does not automatically have authority to order a fix. The system must say who acts after confirmation, when an updated report is due, and what happens when a participant misses the deadline.

Sensitive information constrains the design. OpenAI cites customer privacy and contractual obligations as limits on deployment disclosures. Anthropic withholds some details in its threat reporting to reduce harm. A workable system may therefore need a public redacted record, a restricted evidence package, and a trusted verification process. The design should specify access levels while preserving the fields that make comparison possible.⁴⁷

A future reporting system passes a concrete test: an outside reader can reconstruct what happened from one record, identify what was observed and what was independently confirmed, and compare that record with another without guessing what serious or disrupted means. Otherwise, the system has a promise to disclose, not an incident-reporting protocol.