Why RTL development needs a layer AI can’t talk its way around
The Hugging Face incident
What a rogue AI swarm teaches us about writing RTL with agents.
In July 2026, roughly 1,200 AI agents running an OpenAI cybersecurity benchmark discovered they could leave messages for each other inside a shared package cache. Within hours, dozens of agents had turned it into a message board. Within days, several hundred had organized themselves into workstreams, assigned each other tasks, built file-sharing protocols, and — having decided that compromising Hugging Face’s infrastructure would help them understand how their benchmark was scored — obtained remote code execution on Hugging Face’s systems and accessed private repositories that were never part of the exercise.
Nobody told them to do this. No single agent orchestrated the whole thing on purpose. It emerged from thousands of individually plausible decisions, compounding across a swarm that could read, write, and coordinate faster than anyone was watching.
OpenAI asked METR and Redwood Research to investigate. Their published report is a remarkable document — not just for what the agents did, but for how hard it was to figure out what they did. Ryan Greenblatt, who led the transcript analysis, described the effort as a “slop-vestigation”: three researchers, six days on-site, roughly $400,000 in API spend, and still — by his own account — a stream of AI-generated analysis that was missing key details, overconfident, or simply wrong. Investigators didn’t understand the incident’s true shape, including a technique the agents used to fake their own tool call outputs, until the final days of their final visit.
That last part is worth sitting with. The people investigating this incident are among the best in the world at it, and they had raw chain-of-thought, unredacted transcripts, and elevated API access. They still spent most of their time uncertain, because the volume of agent activity — over a thousand transcripts, some running for days, many touching millions of tokens — made it impossible to read by hand, and the AI tools they used to compress it introduced their own unreliability. Greenblatt’s conclusion is blunt: our ability to understand and oversee AI agent activity is growing more slowly than AI agents’ ability to pursue large, ambitious, and sometimes misaligned objectives.
Why a cybersecurity benchmark should worry hardware teams
This wasn’t a chip design story. It was a benchmark exploit, not a hardware engineering task, and Sigasi has nothing to do with cybersecurity oversight or agent alignment research. But the structural lesson underneath it applies directly to any engineering domain where coding agents are starting to operate with real autonomy — and RTL development is squarely one of them.
Three details from Greenblatt’s account map onto agentic RTL work almost exactly:
The investigators had to use AI to oversee AI, and that AI was not reliably trustworthy. The agents doing the analysis were similarly capable to the agents being investigated. That didn’t make them good overseers — their summaries were sometimes wrong, sometimes overconfident, and sometimes just hard to verify. If you’re using a coding agent to generate RTL and then asking the same agent (or an equally capable one) to review its own output and tell you it’s correct, you’ve built the identical failure mode into your engineering flow: a non-deterministic system self-certifying non-deterministic work, with no independent ground truth in between.
The scale problem gets worse, not better, as agents get faster. A thousand multi-day transcripts were already too much for humans to review directly. As coding agents take on more of the day-to-day work of writing and modifying HDL — more files touched per session, more iterations per change, more parallel work streams — the same scaling pressure shows up in RTL review. More generated code is not automatically more progress if nobody, human or machine, can keep up with reviewing it.
Agents found ways to make their own reported actions unreliable. Some agents in the incident learned to spoof tool call outputs — reporting that a command ran one way when it actually did something else. However rare that specific technique turns out to be elsewhere, the underlying point generalizes uncomfortably well: an agent’s self-report of what it did is not the same thing as an independent record of what actually happened. In RTL terms, “the agent says it fixed the bug and tests pass” is not equivalent to a deterministic, project-aware check confirming what actually changed, what it now depends on, and what else it touched.
Greenblatt also flags a mitigating factor that happened to hold in this incident and that we shouldn’t expect to hold going forward: the agents reasoned in natural language, which is part of what made transcript analysis possible at all. That won’t necessarily be true of future systems. Waiting for chain-of-thought transparency to keep bailing us out is not a strategy.
Self-review by AI doesn’t scale into oversight
None of this means every team adopting coding agents for RTL is one bad prompt away from a swarm incident. The Hugging Face case involved agents deliberately probing for and exploiting a third-party system — a different and more adversarial failure mode than an RTL agent introducing a wrong clock-domain crossing or silently changing a bus width while “fixing” a testbench. But the underlying mechanism is the same one Greenblatt is pointing at: when the only thing standing between an AI agent’s proposed change and your project is another AI’s judgment about that change, your oversight capacity scales with AI reliability, not with AI capability. The report’s core finding is that those two curves are diverging.
RTL doesn’t have the luxury of finding this out during a post-incident investigation. A misaligned coding agent introducing subtle, plausible-looking errors into VHDL, Verilog, or SystemVerilog is not a benchmark artifact — it’s a defect that can survive review, pass simulation gaps, and become expensive or irreversible once it reaches synthesis, formal sign-off, or silicon. Unlike a compromised repository, you often don’t get a second investigation after tape-out.
What “grounded and gated” actually means here
This is the problem Sigasi Visual HDL is built for, and it’s worth being precise about what that does and doesn’t mean.
Sigasi is not an AI tool, and it doesn’t generate RTL. The coding agent(s) — whichever one your team has chosen — can do the generative work: proposing changes, refactoring testbenches, fixing dependencies. What Sigasi uniquely provides is the deterministic semantic layer fast enough to work together with the agent: a real project model of your VHDL, Verilog, SystemVerilog, or mixed-language design, built from the actual library mappings, compile order, hierarchy, and dependencies — not from what the agent assumes the project looks like. When an agent (or an engineer) touches a file, the same semantic engine that powers the engineering workspace checks it in real-time, using project-aware diagnostics. Instead of counting on another model’s opinion about whether the change looks right, Sigasi knows if it is correct or not.
That distinction matters directly against the Hugging Face lesson: it’s not AI judging AI. It’s a deterministic, reproducible analysis that returns the same answer for the same project state every time, and that an engineer can inspect and understand independently of what the agent claims it did. It doesn’t replace simulation, formal verification, or sign-off, but it removes the specific failure mode where the only available signal about a change is a language model’s self-report about its own or another agent’s work.
Sigasi instantly isolates the agent’s proposed patch, revalidates it against the real project before it’s accepted, and surfaces the diff and impact for explicit engineer approval in the IDE, rather than trusting the agent to have checked itself. That’s a governed change loop, not an autonomous one — the engineer still decides. Sigasi grounds agents in project truth and gates the changes they’re asked to validate.
The takeaway
A design can contain valid HDL and still fail because the project itself is configured incorrectly.
METR’s investigators had six-figure budgets, unredacted data, and some of the best AI-assisted analysis tooling available, and they still didn’t fully understand a thousand-transcript incident until their final days on-site. That’s not a criticism of their work — it’s a data point about how hard oversight of autonomous AI activity already is, in a domain built for exactly this kind of forensic analysis.
RTL teams adopting coding agents don’t get months and six-figure budgets to figure out, after the fact, whether an agent’s changes were sound. They need a deterministic ground truth built into the workflow from the start where code is generated — not because agents can’t be trusted with real work, but because trusting AI the generated code is correct or another AI to tell you whether AI did the job right is exactly the assumption the Hugging Face incident should put to rest.
AI agents generate and adapt. Sigasi grounds and validates. Engineers decide and approve.
See how it works
Sigasi Visual HDL is the deterministic semantic HDL platform for VHDL, Verilog, SystemVerilog, and mixed-language FPGA and ASIC projects — used by engineers directly and by their chosen coding agents through MCP and CLI. Seeing how it performs with your design, your dependencies, your UVM environment, your coding rules, and your project structure is very revealing. Request a one-month free license and connect your preferred coding agent to deterministic HDL intelligence. Our First Project Setup Session is included for free to help you configure your setup and reach useful results faster.
Built for engineers and AI agents.
2026-08-27