The governance gap: Best infosec long reads 7/25/26

Happy Saturday to all!

Full access to Metacurity's curated infosec long reads is available to paid subscribers. Our goal is simple: make it financially viable to keep investing the time and expertise required to find, vet, and contextualize the most important security journalism each week. Free readers will still get highlights, but subscribers will get the complete, deeply curated set.

Please help support Metacurity in achieving our goal by upgrading your subscription to gain full access to this issue and all content published on Metacurity, including the archives.

7/25/26: This week's long reads explore a common theme: as AI, surveillance systems, government databases, and critical infrastructure become more powerful and interconnected, governance and operational readiness are struggling to keep pace.

The pieces cover gaps in AI incident reporting, warn of the risks of consolidating federal data, reveal how Russia's FSB is integrating generative AI into intelligence operations, show how a simple error in an automated license plate recognition system can spiral into repeated police encounters, and explain why decades-old malware remains so difficult to eradicate from industrial systems.

When Reporting an AI Security Incident Is Not Mandatory

Using the Hugging Face breach as a springboard, Mackenzie Arnold, the Director of US Policy at the Institute for Law & AI, and Stephan Llerena, Research Scholar at LawAI, argue in Lawfare that existing AI incident-reporting laws are too narrow to require disclosure of many serious frontier AI failures, leaving governments and the public without the information needed to understand and regulate emerging risks.

The rationale for mandatory incident reporting is straightforward: Some industries have the potential to cause real harm to others, and the government and the public have an interest in learning about high-risk events. In the case of the AI industry, there is a major knowledge gap between the companies’ and governments’ understanding of the technology and its risks. Mandatory incident reporting about serious adverse events, which companies might otherwise be reluctant to disclose, helps close that gap. This rationale is all the more compelling in the context of a rapidly evolving, difficult to predict technology, where best practice and political consensus have yet to develop. Observing real-world incidents offers a path to resolve both political and empirical disagreements and prepares governments to respond to future events.

So did the Hugging Face breach trigger mandatory disclosure under existing incident reporting laws? The answer seems far from clear.

California’s SB 53, New York’s RAISE Act, and Illinois’s SB 315 each require that frontier AI developers report “critical safety incidents”—a term each law defines identically. Of the four reportable incident categories, three require actual harm, ranging from “bodily injury” to “the death of, or serious injury to, more than 50 people or more than one billion dollars ($1,000,000,000) in damage.” (If you’re thinking, “that’s an exceptionally high bar for what is a basic, low-cost reporting requirement” or “it sure seems like governments would want that information before mass harm occurs,” you would not be wrong, but we digress.)

So three of the four incident categories do not apply. That leaves only the fourth, which applies to incidents in which a frontier model “uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk.”

It is possible that the Hugging Face breach meets one or more of these elements. It is far less clear that it meets all of them. On the first element, the models may have used deceptive techniques against OpenAI, their frontier developer—they did, after all, try to complete their developer’s evaluation using stolen information, after bypassing restrictions placed on them. But deception is notoriously hard to define, especially if it turns on the “intentions” and “obfuscation” of AI agents. Another read of these events is that the systems simply used all available means of solving the task, and public reporting does not tell us whether the agents attempted to hide those efforts. The second element is also arguably met. While the incident occurred during an evaluation, that evaluation was not “designed to elicit” this specific “deceptive technique.” Based on the ExploitGym benchmark, this evaluation aimed to elicit agentic, cyber-offensive capabilities on a specific task in a controlled environment. It did not, to our knowledge, contemplate—let alone design for—an unexpected cyberattack on a real-world company.

The third element, that the incident “demonstrates materially increased catastrophic risk” would seem to be the most difficult to satisfy. While autonomous cyber capabilities certainly increase the capability and thus potential consequence of agentic action, so too do most capability improvements in AI models. With limited monetary harm and no physical injury, this incident is quite attenuated from future events that might result in the mass physical injury or property damage contemplated by the statute.

With uncertainty at each factor, it is unclear that these existing state laws cover this event. At the very least, it won’t cover all events like it. Stepping back, it seems far from ideal to condition basic incident reporting on a list of complex, highly contested, fact-dependent conditions—all of which must be satisfied simultaneously. In many cases, figuring out whether the incident is indicative of increased risk to the public or actually constitutes deception will not be possible without more information. The purpose of incident reporting is to produce that information, not to require that it be known before a report is ever sent. The chicken must come before the egg. There are better alternatives.