tscircuit community
← Back to community index
# contributor·Active

mohan-bee - Guys i have been doing some one sho...

Guys i have been doing some one shot prompt board and capture their benchmark to see how the AI working with kicad and tscircuit here is the results https://tscircuit gpt benchmark.vercel.app/

Started by mohan-beeSep 5, 202638 messages

Discussion

Last active last month · plain text
mohan-beeoriginal post
Guys i have been doing some one shot prompt board and capture their benchmark to see how the AI working with kicad and tscircuit here is the results https://tscircuit-gpt-benchmark.vercel.app/
image.pngimage.png
mohan-bee
we are doing good in schematic ig
image.pngimage.png
! Rushabh | Linux Shill
looks good
! Rushabh | Linux Shill
but what is this
image.pngimage.png
mohan-bee
Easyeda issue
! Rushabh | Linux Shill
https://github.com/tscircuit/easyeda-converter/pull/542
Seve
it's a bit confusing with the time thing
Seve
we need to measure times well because we're doing really well compared to openai astra for time at the moment
mohan-bee
Time is not accurate for sure
mohan-bee
True
Seve
our schematic seems much worse
Seve
it's spread out etc
Seve
good comparison tho
mohan-bee
yea
0hmx
we need to expand this more! maybe an official benchmark were we make model give score and see if we improve or go down i think if we write the rules right it should be deterministic ish most of the time
Seve
yep 100%
0hmx
also our auto router being more successful might be making the model do the placement worse
Seve
for sure
mohan-bee
seve what can we do to avoid that spread out in schematic placement.. because I am noticing this pattern in every one shot prompted board.
Seve
do we have a warning emitted if there isn't a schematic sheet or if components are outside the schematic sheet?
Seve
We could also warn on long schematic traces 🤔
mohan-bee
you mean we need to default to sheets ?
Seve
no, but we can warn if there's a missing sheet
mohan-bee
hmm
Seve
because then it won't spread out components because it'll throw outside-of-sheet errors
Seve
idk
Seve
or we just warn on long schematic traces
Seve
i think that might also be ok but the issue there is it's kind of a schematic trace solver issue
mohan-bee
🤔 leme try to figure this out to
mohan-bee
i tried to use the schematicsheet for that sheet .. but the ai just wrapped the large schematic with the sheet using a custom size ..
mohan-bee
i am not sure what could improve this
mohan-bee
<@757706909351411845>
mohan-bee
i think autoloayout is doing some good job.
! Rushabh | Linux Shill
i think we should avoid using explicit schX and schY then
Seve
why are we vertically stacking caps like that... that's quite bad haha
mohan-bee
that is done by astra with schx and schy .

Want to add to the conversation?

Reply in Discord so your notes stay connected to the source.
Continue in Discord ↗