(or anything that gets bundled before release- core doesn't get bundled)
Seve
CC <@1525187702029549659> another random dep issue, just adding some complexity 😉
A
astrimid
could be a bug in bun, issue with github networking or some permission issue with github actions. maybe even that git revision is unavailable for some reason. `resolve` hints a DNS issue
Seve
i think whenever a repo isn't bundled we have to avoid using github deps
Seve
the autorouter is bundled so can use github deps
A
astrimid
suggested fix range is changing DNS in the workflow image:
```
container:
image: node:20 # or whatever
options: --dns 1.1.1.1 --dns 8.8.8.8
```
As both npm and github are owned by microsoft and probably hosted by Azure, I suspect resolution problems trace to the same infrastructure issue. Maybe GitHub runners having trouble to resolve domain names because they're hosted on the same internal network. Say dns lookup resolves to some private ip address, because injected by kubernetes or something, but the container only allowed access to public ip addresses. Or something like that, just speculation.
Seve
haha idk about that haha
Seve
that would make the issue much bigger than us
A
astrimid
maybe it only reproduces in combination with how bun does DNS resolving or ipv6 issue
A
astrimid
networking is very strange in general
A
astrimid
found open issue that might be related or might be red herring: https://github.com/oven-sh/bun/issues/28817
A
astrimid
but I suspect it's DNS if that git commit really exists
A
astrimid
there's some serious work on bun:
https://github.com/oven-sh/bun/pull/36295 probably related to that controversial rust rewrite
A
astrimid
also, "resolve" could be either "can't resolve DNS name" or "can't resolve npm package", need to check bun source code to be sure.
But yeah, might be simpler to just get rid of that git sha1 dep and publish an npm package
A
astrimid
but npm also might have DNS resolution problems due to reasons above
Seve
yea the npm package is now published
Seve
nightmare haha
A
astrimid
It doesn't look like core update workflow is working.
yea core generally can't auto-update due to too many snapshots
A
astrimid
I think renovate/dependabot are the only reliable way to sync up automatically without hardcoding dependency tree in all the workflow files
A
astrimid
There are two alternatives:
- polling: every period of time a workflow runs that checks new releases of any of depdendencies (requires looking into package.json/bun.lock and comparing with npmjs released version)
- external server: I guess the original purpose of release-tracker? https://github.com/tscircuit/core/blob/main/.github/workflows/on-merge-inform-release-tracker.yml
could be extended to track latest npm versions and triggering subscribers, but that just pushed the dependency graph to that external server
Could be a hybrid: external server funs out any release events to all the packages on tscircuit org and each repo individually tracks whether it cares about this repo.
A
astrimid
Another idea:
```
# Using GitHub CLI (gh) to find all org repos depending on '@tscircuit/core'
gh api -X GET search/code \
-f q='org:tscircuit filename:package.json "@tscircuit/core"' \
--jq '.items[].repository.name' | sort -u
```
alternatively, each repo just does its own auto-update loop
A
astrimid
the trade off is the more repos you have, the more API workload is spent vs simplicity of maintaining dep versions
A
astrimid
so instead of having `M x N` trigger workflows, we have `M to 1` (notify) and `1 to N` (fun out) workflows.
A
astrimid
As a POC, I could write a workflow that searches tscircuit npm/github org every hour and submits PRs for any outdated dependencies owned by tscircuit. Or, for starter just a bash script you run locally. I wonder if such already exists?
Seve
we already switched from renovate/dependabot, did not work fast enough or well enough
Seve
we're already essentially in a custom bot situation
you could clone the repo or I could submit it whatever a best location for this script/workflow. Maybe `release-tracker` is a logical place?
A
astrimid
Here's a taste of how it could look like. What I like that it should be cleear what is being released, what are downstream dependents on this upstream dependency, who triggered what and a single place in code where you could debug the dependency tree for the whole org.
Next steps:
- amending npm release workflows to trigger this central workflow
- gradual opt-in or opt-out approach for downstream dependents
- implement actual step that do required package.json/bun.lock creates PR and enables auto-merge
- proper tracking of existing open PRs and closing superceeded ones
- finding a new place for it in tscircuit org
Does it sound reasonable or I'm wasting my time?
If it isn't obvious, I want to clarify the difference between lock file and package.json. The lock file is not a part of package release. So if the library has lockfile it's completely ignored during the resolution of app dependencies.
So for dependents where only lockfile required (e.g.`"*"`), you can absolutely skip new release (if it's not an app), as it would be noop. You should still trigger downstream propagation though, one step below.
This means the `update-packages.yml` needs to be extended to either trigger downstream (`bun.lock` only), release app (`bun.lock` only, but is not a library), release library (full `package.json` bump). So there would be 3 types of PR: 1. package.json bump 2. app bun.lock-based release 3. transient dependency update (pass down the package that initiated the release to downstream many hops below)
But for POC, this is probably overthinking.
Want to add to the conversation?
Reply in Discord so your notes stay connected to the source.