Season 1 in preparation.Rankings will be published after measured runs are complete. The pilot snapshot is available with its caveats.Pilot results
Skip to content
PCBBench

Proposed Season 1 protocol

PCB symbol and footprint benchmark

Can the system turn the exact package documentation into correct native library artifacts?

What the task receives

A manufacturer pin table and package drawing for one explicitly selected device variant.

What counts as evidence

A valid symbol and footprint with correct pin mapping, pad geometry and documented assembly assumptions.

Grade pin identity separately from geometry. Check the generated library files in a real project and compare measured dimensions against the selected source. Keep unit conversions and tolerances explicit; a visually similar package should not receive credit for a different land pattern.

Failure cases the evaluation should catch

  • Mirroring the pad pattern from an incorrect drawing view.
  • Omitting an exposed pad or swapping pad numbers.
  • Optimizing visible outlines while leaving the electrical mapping wrong.

Illustrative task brief

From the attached datasheet, create the KiCad symbol and footprint for the 4×4 mm QFN-24 package, including the exposed pad.

This example describes the intended task format. It is not a published model result, and the exact task acceptance contract must accompany its frozen inputs.

Metrics to report

  • Pin-map accuracy
  • Pad IoU
  • KLC violations

Limits of this evidence

An isolated library pass does not establish component fit or routing quality in a finished board.

Read the proposed methodology and dated pilot snapshot before interpreting any result. Partial pilot averages cover different task mixes and do not establish a model ordering.

Related guides