Prior work
PCBBench stands on a lot of good work. We surveyed every benchmark we could find for AI on boards, schematics and circuits as of October 2026. Each one tests a slice of the job well. None grades the whole job, from datasheet to Gerbers, with cost tracked and enough tasks to separate models.
Knowledge is saturated, doing the work is not
Frontier models clear 90% on PCB multiple choice. On executable, DRC- or simulation-graded tasks at realistic scale, the same models fall to single digits.
Format decides the score
The same model read designs at 43% accuracy from PDF and much higher from structured data. Raw KiCad S-expressions fail where a tool API succeeds. A benchmark has to fix and report the representation.
Small benchmarks cannot rank
With 13 to 25 tasks, standard errors of ±6 to ±12 points put most of a leaderboard inside each other's error bars.
| Benchmark | What it tests | Graded by | Size | Layout | Cost | What we borrowed | Gap |
|---|---|---|---|---|---|---|---|
| EEBench atopile · 2026 | Agentic board-level circuit design in atopile | SPICE at worst-case corners + BOM cost vs reference | 13 tasks × 3 attempts | – | Yes | Corner simulation with MLCC derating; cost inside the score; never-published tasks | 13 tasks put ranks 2–11 inside each other's error bars; scaffold differs per model; no layout |
| PCBWorld LG AI Research · 2026 | Routing placed KiCad boards through 58 engine APIs | KiCad DRC + connectivity (Clean Pass), wirelength, vias | 679 real + synthetic boards | Yes | – | Clean Pass as the hard gate; engine-grounded feedback; real-board scale ladder | Routing only; no placement, schematic or physics checks |
| PCB-Bench DiGAI Lab (ICLR 2026) · 2026 | Knowledge of placement, routing, SI, EMC, DFM | Multiple choice, semantic similarity, P/R/F1 | ~3.7k text, ~500 image, 174 projects | – | – | Topic coverage map for the reasoning suite | Models never place or route; frontier MCQ scores are above 90% |
| PCBSchemaGen UT Austin / ASU · 2026 | Schematics from text with real ICs, via SKiDL | Five-layer deterministic verifier: ERC, pin roles, VF2 topology, power invariants | 62 + 165 tasks | – | – | Layered verifier with pin-level localization; VF2 sub-circuit signatures | Template matching can reject valid alternative topologies; no layout |
| HWE-Bench GDUFE / Shandong U. · 2026 | Board-level schematics from real projects with datasheets | Static rule checks, then LTspice transient | 300 tasks, 2,914 datasheets | – | – | Static-then-dynamic grading, both stages reported | Datasheets pre-converted to JSON; single-shot; JSON netlists, not EDA files |
| Cua-Bench KiCad Snorkel AI + Cua · 2026 | Computer-use agents driving the KiCad GUI | Netlist vs golden final project | 25 tasks | – | – | Edit tasks graded by netlist diff | Mostly measures GUI navigation: most failures were perception and clicking, not electronics |
| SchGen Microsoft Research / UCSD · 2026 | Text to KiCad schematic through a Python API | Valid-circuit rate, netlist Jaccard, expert check | 8,420 train + 988 unseen | – | – | Evidence that representation matters: raw S-expressions fail where an API succeeds | Training-oriented; similarity to one reference, not electrical intent |
| PCB-QA UNSW / Calgary · 2026 | Questions about existing KiCad designs | Yes/No accuracy | 480 questions, 8 projects | – | – | Report results by input representation (PDF vs native vs JSON) | Comprehension only; small |
| ICGeo8K SJTU / PSU / MSR · 2025 | Footprint geometry from datasheet drawings | Pad IoU, pin count, center error | 8,608 samples | – | – | Pad IoU as the footprint metric | Geometry only, no symbol or pin map |
| Long-Horizon Analog Design ETH / Tsinghua + 13 · 2026 | Transistor-level analog design over hours | ngspice, 11–2,083 checks per task | 50 tasks | – | Yes | Cost per attempt with cached tokens separated; long budgets | IC-level, not boards |
| CIRCUIT MIT · 2025 | Analog circuit reasoning | Numeric accuracy, grouped into variant unit tests | 510 questions | – | – | A question only counts if every numeric variant passes | Q&A only |
| CircuChain Independent · 2026 | Circuit analysis with convention traps | SPICE ground truth | 100 tasks | – | – | Trap variants that break common defaults | Analysis only |
| NetlistBench MLCAD 2026 · 2026 | SPICE netlist recognition and editing | Structure-aware deterministic oracle | 2,342 cases | – | – | Canonicalize before comparing | Netlist level only |
| CVDP NVIDIA · 2025 | RTL design and verification (digital) | Simulation testbenches | 783 problems | – | – | Agentic and single-shot variants of the same task | Digital RTL, not boards |
| tscircuit benchmarks tscircuit · ongoing | Prompt benchmarks and autorouter regression sets | Compile + checks; DRC, completion and vias for routing | SRJ18: 16 real boards | Yes | – | Reusable real-board routing set | Evaluates prompts and algorithms, not a model leaderboard |
From AI leaderboards in general
- Artificial Analysis: cost per task from measured tokens instead of a blended price; the score-versus-cost chart with a frontier and a “most attractive” quadrant; every reasoning setting as its own entry.
- ARC Prize: a human cost-per-task reference on the same chart; labeling the harness on every result.
- SWE-bench: a minimal, neutral harness for the model ranking; published traces; maintainers rerun results instead of trusting submissions. Its Verified saga is also the warning: audit tasks nobody solves for broken graders.
- LMArena and Scale SEAL: ranks with uncertainty, so overlapping models share a rank; a public changelog.
- τ-bench: pass^k, so reliability is measured and not just best case.
- METR: hierarchical bootstrap over task families; reading high-scoring runs to catch reward hacking.
- LiveBench: date-versioned releases and held-back fresh tasks against contamination.
- GDPval and SWE-Lancer: grade real deliverables, and price them in dollars of real work.
A note on the name
Several research artifacts use similar names: PCBench (a 2024 routing dataset), PCB-Bench (an ICLR 2026 knowledge benchmark) and a 62-task PCBBench suite inside the PCBSchemaGen paper. This site is unrelated to all three. Please cite this benchmark as “PCBBench (pcbbench.org)”.