chore: merge next into merge-train/spartan - #25376
Closed
benesjan wants to merge 12 commits into
Closed
Conversation
Deletes the in-tree copies of the labs components now built from the `labs/` submodule: `yarn-project/`, `noir-projects/labs/`, `docs/`, `playground/`, `spartan/`, `aztec-up/`, `release-image/`, `labs-aztec-toolchain/` (~10.8k files). Goes with them: the workflows that only operated those components (network/RPC deploys, metrics, docs typesense, the CLI acceptance test, the rollup-upgrade payload, the AVM circuit-inputs nightlies, the spartan merge-train check), the `ci-docs` CI mode, the labs-operations claude skills and agents, and the `ci.sh`/`bootstrap.sh` subcommands behind them. Scripts that built the in-tree yarn-project (chonk/ultrahonk bench inputs, `bench_ivc`, socket-fix) now build `labs-yarn-project`. Stale references in CI.md, CLAUDE.md, CODEOWNERS, skills and the merge-train workflow are updated. `labs-patches` tooling: a labs commit touching a file the submodule marks `-diff` (its `yarn.lock`) is now recognised as exported (patch-id computed with `--binary`); lifecycle test added for it.
…idate marker (#25334) Removes the v5 → `next` forward-port flow; `mark-v5-port-candidates.yml` (labels `v5`/`v5-next` PRs `port-to-aztec-node`) is the only thing kept. - Deleted: `auto-port-v5-merge-trains.yml` (auto-labelled v5 trains `port-to-next`), `port-v5-next-to-next.yml` + `scripts/port_to_next.sh` (the superseded whole-branch sweep). - `backport.yml` / `scripts/backport_to_staging.sh`: `backport-to-*` backports only; the `port-to-next` target and its release-artifact skip list are gone. - `merge-train-auto-merge.yml`, `merge-train-update-pr-body.yml`: no `port-to-next-staging` branch. - ClaudeBox backport prompt and the `merge-train-infra` skill updated to match. Why: the labs half of the tree now lives in aztec-node, so on `next` a v5 port either applies to nothing or conflicts; the real destination is aztec-node (as in aztec-labs-eng/aztec-node#96). The `private-port-next` label on `v<N>-next` trains (private-repo driver) is untouched.
… that never reached it (#25333) Moves the labs pin from `9b17bcbe` (2026-08-25) to `1f14e9a6` (aztec-node `main`, 2026-08-27). The series re-applies cleanly on the new base; `l1-contracts/foundry.lock` records the new gitlink. Also carries the two in-tree changes that never reached aztec-node, found by auditing the deleted in-tree labs directories (as of `38fd5fc6e9`) against the submodule by blob history — of 447 differing files everything else is upstream already, deleted upstream on purpose, or foundation-relative wiring: - `0004` — the world-state IPC tests from #25196 (`ipc_churn_correlation.test.ts`, `wsdb_sigpipe_death.test.ts`). - `0005` — the 2026-08-19 docs sweep (`references:` frontmatter on 157 pages, `extract_doc_references.sh`, `docs/package.json`). Neither has an open aztec-node PR; both are ready as `fnd/port-ipc-tests` / `fnd/port-docs-references` branches for upstreaming, after which they drop out of the series on the next bump.
…#25363) ## Problem `bootstrap.sh build_and_test` runs the test engine **concurrently** with `make`, and each target streams its test commands to `/tmp/test_cmds` as soon as it builds. Those tests outlive the target that emitted them, so any later target that rewrites the same tree yanks files out from under a running `node`: - `Cannot find module '@aztec/foundation/config'`, `Tests: 0 total`, `Time: 0.104 s` — a `dest/` was deleted mid-resolution - `Bus error (core dumped)` with no jest output — a mapped file was replaced underneath the process Neither names a cause, and both land on whichever test happened to be running, so they read as unrelated flakes. The run halts on first failure, so one of these ends a 10,000+ test run. Two targets violated the ordering. ### 1. `yarn-project` built twice (labs) `labs-full` chained onto `labs-fast`, making two sub-make invocations. `make` only de-duplicates targets **within a process**, so the second invocation rebuilt `yarn-project` — a prerequisite of `playground`, `docs` and `yarn-project-benches`. Nothing stopped it: `yarn-project` is an existing directory that is not `.PHONY`, its prerequisite `labs-aztec-toolchain` *is* phony (so always "newer"), and `$(call build,...)` writes no stamp. That rebuild re-enters `yarn-project/bootstrap.sh`, which starts with `clean-lite` (`rm -rf` of every gitignored path, including all of `dest/`) and `npm_install_deps` (unconditional `node_modules` re-extraction) — while `labs-fast`'s tests were still reading that tree. Observed: `labs-fast` left at 23:53:12, `labs-full` entered at 23:53:12 and re-ran `yarn-project` at 23:55:02; a test container started 23:55:04 and died at 23:55:06. Both goal sets now run in a single sub-make, so `yarn-project` builds once. `docs`, `playground` and `yarn-project` each have their own `yarn.lock`, so no full goal writes the `yarn-project` workspace. Introduced by #25318, which split the single Makefile into root + submodule sub-makes. Pre-split this was one process with `yarn-project: bb-ts ... bb-avm-sim bb-cdb`, which gave both properties for free. ### 2. `bb-avm-sim` unordered against `bb-ts` Both install into the same `barretenberg/ts` `node_modules` — the hazard the existing `bb-cdb` comment already describes. `bb-ts` was never added to the chain, and it is the one that emits the bb.js test commands, so its tests run out of that `node_modules` while `bb-avm-sim`'s `npm_install_deps` re-extracts it. Observed: `build_bb_avm_sim` started 00:06:04; `native_socket.test.js` and `crs/net_crs.test.js` both died `Bus error` at 00:06:17, ~1s after launch. `net_crs.test.js` only downloads CRS, so no test-level bug explains both. Chain is now `bb-ts -> bb-avm-sim -> bb-cdb -> bb-avm-sim-cross-copy`. ## Verification `make -p` confirms the resolved graph: ``` labs-full: labs-use-local # no longer chains through labs-fast bb-avm-sim: ipc-codegen ipc-runtime bb-cpp-native bb-ts bb-cdb: ipc-codegen ipc-runtime bb-avm-sim ``` `make full` now makes one labs sub-make call covering both goal sets. dmesg from the failing runs shows no OOM and no filesystem error, consistent with SIGBUS from a file replaced under an mmap rather than resource exhaustion. ## Related, not fixed here - `labs/Makefile` omits `yarn-project` from `.PHONY` — the only one of its 8 directory-named targets missing, against the file's own stated rule. Separate PR: aztec-labs-eng/aztec-node#111. - `ci3/npm_install_deps` re-extracts `node_modules` on every call with no "already installed" check. Making it a no-op when the tree already matches `nm_hash` would remove this whole class rather than individual instances. ## Disabled: `labs-patches/0005` Renamed to `*.patch.disabled`. `apply` globs `*.patch`, so it stays in the queue but is not applied. **It is not deleted and nothing in it is lost.** The docs it adds carry `references:` frontmatter pointing at `l1-contracts/`, `noir-projects/fnd/`, `barretenberg/cpp/` and `avm-transpiler/src/`. Those live in this repo, but `docs/scripts/check_doc_references.sh` resolves references against `git rev-parse --show-toplevel`, which inside the submodule is `labs/`. So the docs build fails on paths that are perfectly valid one directory up. Verified against the pinned labs tree: the original patch reproduces the CI failure, and without it the checker passes (43 references, all resolve). **The references are correct — the checker is what's wrong.** It has no notion of the parent repo. Stripping the references would delete roughly a third of the patch's 157 reference lines (I tried it: removing just `l1-contracts` emptied 16 lines and left 18 failures across the other three trees). The proper fix is in the labs repo: resolve against the outer repo when one exists, or skip references whose first path segment is absent. Re-enable the patch then — by fixing the checker, not by deleting the metadata. This is unrelated to the races above; it surfaced only because fixing them let full runs get far enough to reach `labs/docs`.
) The ci3 dashboard (rkapp on ci.aztec-labs.com) is a single shared service, but it was vendored and deployed from **two** repos: - **aztec-packages** — auto-deployed on push to `next` touching `ci3/dashboard/**` (added in `58dc521480`). - **aztec-node** — same workflow, but push trigger commented out (manual `workflow_dispatch` only). Both run the identical `deploy.sh` → rsync to the same `ci.aztec-labs.com:rk` and `systemctl restart rkapp`. Same target, last-writer-wins — so aztec-packages' auto-deploy silently overwrote whatever aztec-node deployed, and aztec-node's copy (which now carries the multi-org dashboard sections + TLS/compose work) never stuck. This removes the aztec-packages deploy workflow. The dashboard is deployed manually from **aztec-node** for now (its `workflow_dispatch`), which owns the current dashboard code. Longer term the dashboard should live in one place rather than be vendored into two product repos. No effect on any other CI: this only deletes a deploy workflow.
…is fully absorbed (#25369) All five patches — toolchain portal, TEST_CMD_PREFIX, fnd-hashes identity, the ipc request-id envelopes, and the (disabled) docs sweep — are upstream byte-identically (verified by patch-id), so the series empties to nothing. aztec-node also moved `noir-projects` out of `labs/` (`labs/noir-projects/labs` → `labs/noir-projects`); the foundation files naming that layout are updated, and the labs entry in `l1-contracts/foundry.lock` follows the pin. Upstream adapted the swept docs `references:` itself (aztec-node@da44802b4): labs-local paths remapped, foundation paths deliberately dropped — so no docs-checker patch is needed at this pin.
The docs build — which runs `check_doc_references.sh` — is full-only, so a change that breaks a docs reference passes PR (fast) CI and fails only in the merge queue (that is how the docs sweep patch blocked the queue). The check itself is seconds of bash: `make labs-docs-refs-check` runs it in fast CI so a bad reference fails the PR instead. The checker-fix and re-enabled sweep patches this PR previously carried are gone: aztec-node absorbed the sweep and adapted the references to its own layout itself, so at the bumped pin no patch is needed and the check passes as-is.
…ed publishing (#25326) Stacked on #25321. The EC2 release build now only **packs** the foundation npm packages; a GitHub-hosted job publishes them with npm trusted publishing (OIDC + `--provenance`). The build machine never holds npm credentials and `NPM_TOKEN` is not used anywhere on this path. - `ci3/deploy_npm`: pack mode (`NPM_PACK_DIR`) — same version stamping and `workspace:^` rewriting, output is an `npm pack` tarball instead of a publish. - `bootstrap.sh release`: sets `NPM_PACK_DIR=npm-release`, then `cache_upload npm-release-<tag>.tar.gz` (force-uploaded so a re-run refreshes it; dry runs pack but do not upload). - `ci3/publish_npm_release`: publishes every tarball in a directory with the existing dist-tag policy (`latest` / `prerelease`) and idempotency (already-published version → `dist-tag add`); `DRY_RUN=1` supported. - `ci3.yml`: `publish-npm` job — `needs: ci`, tags only, `AztecProtocol/aztec-packages` only, `environment: release`, `id-token: write`, npm ≥ 11.5.1; `cache_download` + `publish_npm_release`. The private repo is unchanged: `private_release` keeps publishing to the internal GCP Artifact Registry from EC2. Verified in the devbox: pack mode on `barretenberg/ts/bb.js` and `wsdb/ts` (deps rewritten to the stamped version) and a `DRY_RUN=1` publish of the result. Not in this PR: the `@aztec` → `@aztec-foundation` scope rename (it has to rewrite import specifiers in the dist output and the labs pins too) and the npmjs-side trusted-publisher configuration, which needs the first publish of each package. **Audit follow-ups** ([AztecBot audit](https://gist.github.com/AztecBot/11a9abdb587e55c0ceb248256b899ff3)): the publish job downloads the bundle with its AWS OIDC credentials (`aws s3 cp`, not the plain-HTTP cache endpoint), extracts it outside the checkout, and `publish_npm_release` refuses any tarball whose package name is not ours or whose version differs from the tag. A re-run reports an already-published version instead of attempting `npm dist-tag add` (OIDC only authorizes `npm publish`). The service-account env var is `GCP_PRIVATE_NPM_DEPLOY_KEY` so `ci3/redact` masks it (the GitHub secret keeps its name), and ci3's own tests — including `redact_test` — now run in fast CI. `NPM_TOKEN` is no longer injected anywhere. `private-fork-release.yml` is deleted: its API-created tag never triggered ci3.yml, so a fork-dispatched release could never reach `publish-npm`, and the private repo has its own release flow. Deferred: current `barretenberg/docs` still installs `@aztec/bb.js` — it can only move to `@aztec-foundation/bb.js` after the first release publishes that package (tracked).
… processes (#25371) On a shared CI machine every concurrent build runs denoise and color_prefix, so these tests' machine-global pgreps match other jobs' tails, awks and sleeps — wired into fast CI by #25326, they fail on any busy runner (and `pkill -f "sleep 60"` can kill other jobs' processes). denoise gains a `DENOISE_TMPDIR` override so its test watches a private directory; the color_prefix test uses per-run unique sleep durations and label. Verified red/green in the devbox with decoy processes simulating a busy machine.
…scope (#25328) Stacked on #25326. Renames every npm package published from this repository from `@aztec/*` to `@aztec-foundation/*` — the scope npm trusted publishing is being set up for (see #25326). Labs packages keep `@aztec` and are published from the labs repo. - Packages: `bb.js`, `bb-avm-sim` (+ 4 platform packages), `cdb`, `ipc-runtime`, `wsdb` (+ 4 platform packages), `noir-*` (from `@noir-lang/*`, renamed at build in `noir/bootstrap.sh` and at release in `ci3/release_prep_package_json`), `l1-artifacts`, `constants-codegen`, `protocol-circuits-artifacts`, `mock-protocol-circuits-artifacts`, `protocol-contracts-artifacts`. - In-tree consumers follow: portals (`barretenberg/ts`, `wsdb`, `acir_tests`), imports, `ipc-codegen` defaults (`--package-name`, `ipcRuntimeDependency`), `bootstrap.sh` private-release npmrc scope (`@aztec-foundation:registry`), READMEs/comments. - Lockfiles: renamed workspace/portal entries only (`barretenberg/ts`, `wsdb`, `ipc-runtime/ts`, `protocol/constants-codegen`, `barretenberg/acir_tests`, `l1-contracts`); no transitive drift. - `labs/` unchanged: yarn portals resolve by path and accept a target manifest with a different name (verified: `@aztec/bb.js` → `portal:` to a package named `@aztec-foundation/bb.js` installs and imports), so `labs-use-local` keeps working until labs migrates its imports/pins. - Left alone on purpose: `barretenberg/docs` (pinned to the already-published `@aztec/bb.js@0.87.9`), CHANGELOG, the Solidity `@aztec/` remapping in `l1-contracts` (unrelated). Consequence to plan for: private releases now publish `@aztec-foundation/*` to the internal Artifact Registry, so labs' private pins (`@aztec/*`) need to move at the same time.
…y in SSM mode (#25349) Port of aztec-packages-private#713 (`cb/ci3-no-runner-ssh-key`) — the four CI files; the `aztec-up` hunk has no target here any more. SSM mode authenticates to the build instance with an instance profile, so `BUILD_INSTANCE_SSH_KEY` is now written only on the direct-SSH path (`CI_USE_SSH=1`). Without a key on the runner, `source_redis` skips the bastion tunnel instead of failing the run, and `denoise`/`source_cache` tolerate redis being absent. Motivation: the private repo's tag builds died at `ubuntu@ci-bastion: Permission denied (publickey)` once that key stopped matching. The build log is reachable as before — `bootstrap_ec2` posts the `ci/<job>` commit status with the dashboard link, and the instance logs to redis from inside the VPC (verified on both repos' runs). Two small fixes so that holds without redis on the runner: - `denoise` prints its captured output when it has no log store and `CI=1` — the orchestration output, including the `CI Log:` link, used to vanish on success; - `cache_log` persists the instance's top-level log to S3 synchronously (the background upload was killed with the container before it landed).
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
benesjan
requested review from
nventuro
and removed request for
a team,
charlielye and
nventuro
September 1, 2026 13:40
benesjan
marked this pull request as draft
September 1, 2026 13:41
Contributor
Author
|
Superseded by #25377, which reverts #25292 on the train instead — a 4-file diff rather than merging |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves the merge conflicts blocking #25375.
nextdeleted the in-treedocs/in #25321 (those components are now built from thelabs/submodule), while the train carried #25292, which modified four of those files — a modify/delete conflict on:docs/docs-operate/operators/prerequisites.mddocs/docs-operate/operators/setup/staking-provider.mddocs/network_versioned_docs/version-v5.2.0/operators/prerequisites.mddocs/network_versioned_docs/version-v5.2.0/operators/setup/staking-provider.mdResolved by accepting the deletions: those paths no longer exist in this repo. The operator docs content from #25292 is re-landed in aztec-node against
labs/docs/instead.The resulting tree is identical to
origin/next.Merge with the merge commit method (not squash) to preserve the resolution.