# Verification is the most verifiable job AI can't do

> Chip verification exists to check whether things are right, and its tools are machines built to decide. It should be the easiest job to train for. It is where models score worst.

The ChipTrain team · November 17, 2026 · https://chiptrain.ai/verification-is-the-most-verifiable-job

Reinforcement learning works best where answers can be checked. That is why it transformed maths and code first: a proof checks, a test suite runs. By that standard, chip verification should be the easiest job in engineering to train for. Its whole purpose is checking, and its tools (simulators, reference models, formal provers) are machines built to decide whether a design is right.

Yet verification is the part of chip design where AI models score worst. We think that is a data problem, not a ceiling, and that verification will be one of the next domains where RL produces a jump like the one coding saw. We expect the gap between models' verification and design-generation scores on public hardware benchmarks to close substantially within eighteen months, and we expect it to close first for whoever trains on graded verification work rather than on more RTL.

## The gap

Two independent groups have now measured it. On [NVIDIA's CVDP benchmark](https://arxiv.org/abs/2506.14074), the best-scoring model passes verification tasks (writing stimulus, writing checkers, writing assertions) 6–25% of the time, against 34–48% for generating RTL. Writing a testbench that catches bugs is harder for a model than writing the design it tests.

[Phoenix-bench](https://arxiv.org/abs/2605.15226), built from real issues in real hardware repositories, lands in the same place from a different angle. The same agents that do well on SWE-bench lose 37 to 58 points when the repository is hardware, and the subcategories where the bug lives in the testbench fail 74–79% of the time. The best product agent resolves fewer than two in five issues.

This is not a niche weakness. Verification takes the largest share of engineering effort on a chip project, and several frontier labs now design their own accelerators. The capability their models lack most is the one their own silicon teams spend most of their time on.

## Why it should be trainable

Most RL environments outside maths and code rely on one of two graders: tests, which can be satisfied without solving the problem, or a model-written rubric, which can be argued with. Reward hacking is the first thing a lab asks an environment vendor about, because both graders leak.

Hardware verification brings graders that are stronger in kind, not just degree:

- **Formal equivalence** proves a repaired module is the same circuit as the reference, or returns the exact input sequence on which it is not.
- **Mutation scoring** grades a testbench by how many hidden injected bugs it catches: a continuous score against a target the model never sees.
- **Co-simulation** checks every retired instruction against a reference model, not just the final state.
- **The physical flow** returns frequency, power and area: numbers, not opinions, with no ceiling to saturate.

These graders are deterministic, and they are cheap. A simulation or a bounded proof on a single module runs in seconds to minutes of CPU, next to the dollars of inference a long agent transcript costs. The reward signal is the one part of hardware RL that is already better than software's.

## Why it hasn't happened

If the graders are this good, why hasn't anyone scaled it? Because the material is missing, and what exists is graded the software way.

What would teach a model verification is not final RTL. It is the record of how designs were made correct: bug reports and the fixes that closed them, regressions that failed and the debugging that followed, coverage plans and the tests that reached them. Nearly all of it sits inside chip companies, behind customer contracts, foundry agreements, third-party IP licences and export rules. Public hardware benchmarks are small by comparison (CVDP kept 783 problems, Phoenix-bench 511) and each becomes contaminated the day it is published.

And most of them grade by running tests. In hardware that is not enough. We have found patches that pass a public hardware benchmark's own tests and are provably not equivalent to the accepted fix: a formal tool produces the input on which they diverge. For a benchmark, that is a measurement error. For training, it is a reward that teaches a model to satisfy the tests instead of the design.

## What it takes

Three things, none of them exotic. Graders that use hardware's own ground truth instead of borrowing software's. Tasks whose difficulty is measured on frontier models and tuned into the band where they teach, not the band where they impress. And a supply that does not depend on how many verification engineers can be hired to write tasks by hand, because there are not enough of them and they have better things to do.

Verification was built to answer the question reinforcement learning keeps asking: is this right? The field has the machinery to ask it at scale. It has not yet had the data.

---
ChipTrain builds graded chip-design environments. Sample tasks: hello@chiptrain.ai
