Electronics reasoning benchmark

What the task receives

A self-contained electrical problem with specified device conditions, tolerances and an answer format.

What counts as evidence

A numerically correct answer, units, and the assumptions necessary to reproduce the calculation.

Separate arithmetic from device interpretation. Record acceptable rounding and unit conversions, and accept mathematically equivalent expressions. Include cases where the obvious nominal calculation fails at the stated input or tolerance limit.

Failure cases the evaluation should catch

• Confusing milliamps and amps while producing plausible prose.
• Using an ideal equation outside the device's stated operating conditions.
• Changing the requested tolerance or choosing a convenient nominal case.

Illustrative task brief

TPS62130 (adjustable), VFB = 0.8 V. Pick E24 resistors for a 3.3 V output with R2 ≤ 100 kΩ. Report R1, R2 and the resulting Vout.

This example describes the intended task format. It is not a published model result, and the exact task acceptance contract must accompany its frozen inputs.

Metrics to report

• Variant-group accuracy
• Single-question accuracy

Limits of this evidence

A correct calculation does not establish that the system can create a valid schematic or board.

Read the proposed methodology and dated pilot snapshot before interpreting any result. Partial pilot averages cover different task mixes and do not establish a model ordering.

Related guides

• Reproducible benchmarks for AI hardware design
• Data contamination in AI PCB benchmarks
• Pass rate, pass@k and reliability in PCB benchmarks
• PCB design benchmarks vs electronics question answering
• Reproducible hardware benchmark checklist