← Back to community index
# contributor·Active

anil - tscircuit/core doesn't update in tscircu...

tscircuit/core doesn't update in tscircuit/eval.

Started by anilJul 29, 202665 messages

Discussion

Last active last month · plain text
aniloriginal post
tscircuit/core doesn't update in tscircuit/eval.
techmannih
Yes we are looking into same
techmannih
Seems like fanout-solver dep issue in core
techmannih
Can you fix it?
Seve
is it using github style installtion
Krishna
yeah i taught why format-si-unit is not auto-updated, then i done by myself by updating to core and it started throwing failled buntest .
Seve
Shared an attachment
Seve
yea
techmannih
Yes
Seve
we can only use that in the autorouter
Seve
(or anything that gets bundled before release- core doesn't get bundled)
Seve
CC <@1525187702029549659> another random dep issue, just adding some complexity 😉
astrimid
could be a bug in bun, issue with github networking or some permission issue with github actions. maybe even that git revision is unavailable for some reason. `resolve` hints a DNS issue
Seve
i think whenever a repo isn't bundled we have to avoid using github deps
Seve
the autorouter is bundled so can use github deps
astrimid
suggested fix range is changing DNS in the workflow image: ``` container: image: node:20 # or whatever options: --dns 1.1.1.1 --dns 8.8.8.8 ``` As both npm and github are owned by microsoft and probably hosted by Azure, I suspect resolution problems trace to the same infrastructure issue. Maybe GitHub runners having trouble to resolve domain names because they're hosted on the same internal network. Say dns lookup resolves to some private ip address, because injected by kubernetes or something, but the container only allowed access to public ip addresses. Or something like that, just speculation.
Seve
haha idk about that haha
Seve
that would make the issue much bigger than us
astrimid
maybe it only reproduces in combination with how bun does DNS resolving or ipv6 issue
astrimid
networking is very strange in general
astrimid
found open issue that might be related or might be red herring: https://github.com/oven-sh/bun/issues/28817
astrimid
but I suspect it's DNS if that git commit really exists
astrimid
there's some serious work on bun: https://github.com/oven-sh/bun/pull/36295 probably related to that controversial rust rewrite
astrimid
also, "resolve" could be either "can't resolve DNS name" or "can't resolve npm package", need to check bun source code to be sure. But yeah, might be simpler to just get rid of that git sha1 dep and publish an npm package
astrimid
but npm also might have DNS resolution problems due to reasons above
Seve
yea the npm package is now published
Seve
nightmare haha
astrimid
It doesn't look like core update workflow is working.
astrimid
the very bun install is not working
astrimid
why there's no lock file in core?
astrimid
no Update @tscircuit Package triggers since yesterday
astrimid
Why this workflow wasn't autodeleted when the workflow file was deleted https://github.com/tscircuit/eval/actions/workflows/update-deps.yml
astrimid
Ok. eval is a downstream of core: https://github.com/tscircuit/eval/actions/workflows/update-tscircuit-core.yml
astrimid
what triggers core?
astrimid
calculate-packing, circuit-json, schematic-trace-solver, props, ngspice-spice-engine, matchpack - any of those npm releases should trigger core
astrimid
ok, those packages is where the pattern breaks. apparently, core isn't being auto-updated by its dependencies
astrimid
core triggers eval: https://github.com/tscircuit/core/blob/f69f522dbd485bf115370e55bd3d9d34cdddb298/.github/workflows/bun-pver-release.yml#L34-L41
Seve
yea core generally can't auto-update due to too many snapshots
astrimid
I think renovate/dependabot are the only reliable way to sync up automatically without hardcoding dependency tree in all the workflow files
astrimid
There are two alternatives: - polling: every period of time a workflow runs that checks new releases of any of depdendencies (requires looking into package.json/bun.lock and comparing with npmjs released version) - external server: I guess the original purpose of release-tracker? https://github.com/tscircuit/core/blob/main/.github/workflows/on-merge-inform-release-tracker.yml could be extended to track latest npm versions and triggering subscribers, but that just pushed the dependency graph to that external server Could be a hybrid: external server funs out any release events to all the packages on tscircuit org and each repo individually tracks whether it cares about this repo.
astrimid
Another idea: ``` # Using GitHub CLI (gh) to find all org repos depending on '@tscircuit/core' gh api -X GET search/code \ -f q='org:tscircuit filename:package.json "@tscircuit/core"' \ --jq '.items[].repository.name' | sort -u ```
astrimid
or a custom bot
astrimid
but this is overengineering. how about trying out dependabot or renovate?
astrimid
renovate seems superior, but not sure what their pricing policy
astrimid
the problem with dependabot and renovate, they have a limit on frequency of update. we want instant trigger with cron fallback:
astrimid
alternatively, each repo just does its own auto-update loop
astrimid
the trade off is the more repos you have, the more API workload is spent vs simplicity of maintaining dep versions
astrimid
so instead of having `M x N` trigger workflows, we have `M to 1` (notify) and `1 to N` (fun out) workflows.
astrimid
As a POC, I could write a workflow that searches tscircuit npm/github org every hour and submits PRs for any outdated dependencies owned by tscircuit. Or, for starter just a bash script you run locally. I wonder if such already exists?
Seve
we already switched from renovate/dependabot, did not work fast enough or well enough
Seve
we're already essentially in a custom bot situation
astrimid
Shared an attachment
astrimid
massive number. Might actually need caching and nighly run to build dep graph
astrimid
unless you already have something like that
astrimid
Shared an attachment
Seve
we're not trying to auto-sync everything tho fwiw
Seve
a lot of dependencies are just dynamically loaded like circuit-json-to-*
astrimid
can easily add exceptions
astrimid
or opt-in list
astrimid
the current situation of browsring through heterogenous intermixed workflows is absolutely unmanageable, imho
astrimid
could filter by repository topics, hardcode the auto-sync/opt-out list or whatever criteria you like
astrimid
https://github.com/astrimid/depcirk/actions/runs/30485086940/job/90688637113#step:4:50
astrimid
you could clone the repo or I could submit it whatever a best location for this script/workflow. Maybe `release-tracker` is a logical place?
astrimid
Here's a taste of how it could look like. What I like that it should be cleear what is being released, what are downstream dependents on this upstream dependency, who triggered what and a single place in code where you could debug the dependency tree for the whole org. Next steps: - amending npm release workflows to trigger this central workflow - gradual opt-in or opt-out approach for downstream dependents - implement actual step that do required package.json/bun.lock creates PR and enables auto-merge - proper tracking of existing open PRs and closing superceeded ones - finding a new place for it in tscircuit org Does it sound reasonable or I'm wasting my time?
astrimid
If it isn't obvious, I want to clarify the difference between lock file and package.json. The lock file is not a part of package release. So if the library has lockfile it's completely ignored during the resolution of app dependencies. So for dependents where only lockfile required (e.g.`"*"`), you can absolutely skip new release (if it's not an app), as it would be noop. You should still trigger downstream propagation though, one step below. This means the `update-packages.yml` needs to be extended to either trigger downstream (`bun.lock` only), release app (`bun.lock` only, but is not a library), release library (full `package.json` bump). So there would be 3 types of PR: 1. package.json bump 2. app bun.lock-based release 3. transient dependency update (pass down the package that initiated the release to downstream many hops below) But for POC, this is probably overthinking.

Want to add to the conversation?

Reply in Discord so your notes stay connected to the source.
Continue in Discord ↗