In hardware, cheap tasks don't have to waste compute
Hand-written chip-design tasks cost more than the compute they feed. The way out is the thing the cheap-tasks argument warns against, and hardware is the one domain where it works.
The ChipTrain team · November 17, 2026 · read as .md
Last year Mechanize argued that cheap RL tasks will waste compute. A frontier RL run samples dozens of long transcripts per task and reuses each task across runs, so the compute a task consumes dwarfs what labs typically paid to make it. Spending $100 on a task that will burn $2,400 of compute is, in their phrase, putting cheap tires on a Ferrari.
We think the argument is right, and that it lands harder in chip design than anywhere else. But hardware also breaks it in an instructive way. Run the numbers for a chip-design task and a hand-written task costs more than the compute it feeds. In software, the remedy is to pay experts more. In hardware that doesn't clear, and the only way out is the thing the argument warns against: generating tasks by machine. What makes that safe in hardware, and not in software, is the grader.
We predict that within about a year, most chip-design RL tasks that labs buy will be machine-generated and graded by formal proof, and hand-written tasks will become a premium tier rather than the inventory.
What a hardware task consumes
Hardware transcripts are long. Phoenix-bench, a 2026 benchmark built from real issues in Verilog and SystemVerilog repositories, measured one agent consuming about 673 million tokens across 511 instances: roughly 1.3 million tokens per task. Compile logs, simulation logs and waveform queries are verbose, and agents read them again and again.
Pricing those tokens is the uncertain step. Much of a long transcript is re-read context that serves cheaply from cache, so we use a blended range of $3 to $15 per million tokens, the top being the figure Mechanize used. Keeping their other assumptions, a group of 64 rollouts per task and five reuses over its life:
1.3M tokens × $3–15 per 1M × 64 rollouts
≈ $250–1,250 per task per run
× 5 reuses
≈ $1,250–6,250 lifetime compute per task
That brackets Mechanize's $2,400 software estimate, and hardware transcripts, like software ones, are getting longer.
What a hand-written hardware task costs
Now the other side. Data platforms have listed experienced RTL and verification engineers at $100 to $190 an hour. Mechanize has written that a well-built task takes about one engineer-week, most of it spent not writing the task but making its grader fair and deterministic. And QA throws work away: NVIDIA's CVDP, the largest authored hardware benchmark, had about 35 experienced engineers write 1,313 problems and discarded 40% of them in review, keeping 783.
40 hours × $100–190
≈ $4,000–7,600 per task written
÷ 0.6 surviving review
≈ $6,700–12,800 per usable task
A usable hand-written hardware task costs somewhere between about the same as the compute it will consume and ten times more. The complementarity argument says the data should be worth the compute. Here it already costs more than the compute, before anyone has asked whether it is any good.
Why paying more doesn't fix it
In software, the prescription is clear: raise the task budget toward the compute budget, hire better people, make fewer and better tasks. In hardware the task budget is already there, and the constraint is supply. CVDP took 35 engineers to produce 783 problems. At one task per engineer-week, a bench of twenty hardware engineers produces something like a thousand tasks a year. That is a benchmark, not a training corpus, and the people who could write more are the same people every chip company is trying to hire.
Hardware RL data will not scale by hand. It scales by machine or not at all.
Why machine-made hardware tasks are different
Mechanize's warning about cheap tasks is, underneath, a warning about cheap graders. A generated software task usually comes with a generated check: a test suite that can be satisfied without solving the problem, or a model-written rubric that can be argued with. Spend $2,400 of compute against that and you buy a model that is better at satisfying checks.
Hardware has a grader software does not: formal equivalence. Take a working module, inject a defect, and ask the model to restore it. The grade is not whether some tests pass. It is whether the result is provably the same circuit as the original, and when it is not, the tool returns the exact input sequence on which the two differ. There is no rubric to talk round, because the reference is the design itself.
Verification tasks get the same treatment from the other direction. A model-written testbench is scored by how many hidden injected bugs it catches, and any injected bug that turns out to be indistinguishable from the original is removed by proof, not by judgement. The score is continuous, the target is hidden, and the grader runs outside the model's sandbox.
Two things follow.
- Trust comes from the grader, not the author. A generated task graded by proof is as trustworthy as a hand-written one graded by proof, and far cheaper.
- Difficulty becomes a dial. Subtler defects, deeper in the design, further from the symptom. This matters more than it sounds. A task set frontier models score zero on teaches nothing; Epoch AI reports that labs want a minimum pass rate of around 2–3%, or at least one success in 64 or 128 rollouts. Generated tasks can be tuned into that band. Historical bug fixes arrive at whatever difficulty they happen to have.
Where the experts go
Not out of the loop. Experts decide what to generate: which capability gaps matter, which designs, which defect classes are realistic rather than merely obscure. And they write the tier no generator reaches: long flows from specification to measured silicon, ambiguous specs, change requests that are subtly wrong and should be pushed back on. Expert hours move from writing tasks to designing task factories, which is a far better use of the scarcest people in the field.
The lesson of the cheap-tasks argument was that the data has to be worth the compute. In hardware, the cheapest way to make it worth the compute is a better grader, not a more expensive author.
We build graded chip-design environments at ChipTrain. If you train models, ask for a sample task and its grader. If you hold chip-design history, talk to us about licensing it.