How to build a fair PCB autorouter benchmark

Choose a varied board set

Include different package pitches, connector arrangements and congestion patterns. Keep a record of unrouted connections and any pre-existing violations in each input. An input defect should not unexpectedly become an obligation for one entrant while another router is judged only on newly created tracks.

Keep budgets and retry policies visible

Declare compute limits, thread counts and whether failed attempts can restart. Save every attempt, including crashes. If a tool chooses the best of several outputs, give every entrant the same opportunity and distinguish that policy from first-attempt performance.

Make the board checker independent

Refill zones and run the target EDA checks on the submitted artifact. Compare intended connectivity with the input netlist, not only the router's internal success flag. Freeze rule files outside the editable output where the evaluation environment permits it.

Practical checklist

1. Version every input board.
2. Publish the allowed edit policy.
3. Set equal retry and compute budgets.
4. Record crashes and incomplete routes.
5. Compare quality only after validity gates.

A concrete example

Run three routers on a dense connector board.

Evidence to accept

The report includes clean completion, runtime, vias and trace length for each attempt.

Evidence that is insufficient

The best seed is reported for one router while the others receive one attempt.

Common questions

Primary references and further reading

Use the matching device documentation and project configuration when applying this checklist. These references explain the underlying tools and evaluation approaches.

• KiCad PCB editor documentation
• KiCad command-line interface documentation
• PCBWorld: engine-grounded PCB design automation