Researchers released a new benchmark for analog circuit design
The SGAnalog benchmark uses open-source silicon designs to test how well AI models can transcribe schematics and size electronic components.
Designing analog circuits is a complex task that requires both precision and deep knowledge of physics. A new study recently introduced SGAnalog, a benchmark designed to evaluate how AI models handle these hardware tasks.
The benchmark pulls from 273 distinct designs submitted to Tiny Tapeout, a project that makes manufacturing silicon chips more accessible. By using real-world schematics and their corresponding SPICE netlists—files that describe the electrical connections of a circuit—the researchers created a way to measure if AI truly understands circuit design or if it is just memorizing patterns.
The two tasks produce different model rankings, exposing distinct visual and design capabilities and, in one family, a policy rather than capability limit.
Researchers tested seven models on two specific challenges: schematic-to-netlist transcription and device sizing. In transcription, the best model achieved 56.1% accuracy in graph isomorphism, which measures how well the model maps the structural connectivity of the circuit.
For device sizing, the top-performing model reached a score of 91.2 out of 100 compared to human references. However, the study revealed that some newer models sometimes refuse to perform tasks they are otherwise capable of completing, suggesting that internal safety policies may interfere with their technical utility.
Could these benchmarks eventually help engineers automate the more tedious parts of hardware design?
It is worth noting that this study only tests specific transcription and sizing tasks. The benchmark does not measure whether these designs would function perfectly under every possible physical manufacturing condition.
In this story
Source: arXiv cs.AI