Skip to content

build: rework the turbo task graph and lint the repo in one pass - #1928

Merged
joshunrau merged 8 commits into
DouglasNeuroInformatics:mainfrom
joshunrau:chore/turbo-task-graph
Nov 19, 2026
Merged

joshunrau merged 8 commits into
DouglasNeuroInformatics:mainfrom
joshunrau:chore/turbo-task-graph

Conversation

@joshunrau

@joshunrau joshunrau commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Turbo's cache now survives between checkouts, git worktrees and CI runs, and pnpm lint takes 12 s cold instead of 22–25 s (11 cores). Stacked on #1927: until that merges, the diff here also shows its two commits.

Changes

  • pnpm lint runs one oxlint --type-check --fix over the whole repo instead of tsc && oxlint --fix src in each workspace. No workspace has a lint script any more except apps/outreach, which keeps astro check. pnpm --filter <pkg> lint no longer exists; pnpm exec oxlint --type-check <path> checks part of the repo.
  • That pass also lints files the per-package scripts skipped: vite configs, build scripts, runtime/v1's tests. Their findings are fixed here.
  • pnpm format is one oxfmt run over the repo. No workspace has a format script.
  • .env is no longer a turbo global dependency, so editing it reruns nothing and a cache entry is valid in another checkout. A task still sees a variable only through its env key.
  • NODE_ENV is no longer passed to turbo tasks from the calling shell, and NODE_OPTIONS is passed through without entering the hash. The build env list applies to apps/web and apps/playground only, each in its own turbo.json.
  • TURBO_CACHE_DIR is gone from .env.template. An existing .env that still carries the line keeps a second cache directory for the scripts that load it.
  • A turbo cache hit on runtime-core's build now restores lib/ as well as dist/.
  • Editing a *.md file or a unit test no longer rebuilds anything downstream; editing a file under /docs or /blog now invalidates outreach's build and lint, which it did not before.
  • pnpm lint no longer runs the gateway's prisma db push, and neither does the gateway's build. dev, dev:test and test:e2e still do.
  • turbo run test:e2e --filter=@opendatacapture/testing now builds everything the suite needs. Before, only an unfiltered run did.
  • pnpm test and pnpm test:coverage build the packages that export only build output before running vitest.
  • pnpm start builds the app it starts.
  • A production build with no RELEASE_VERSION takes the root package.json version instead of failing, so root pnpm build works on a development machine.
  • The api, gateway and web builds are rebuilt when the root package.json changes, since that is where the fallback version comes from.
  • CI restores turbo's cache before linting, and the release workflow's validate job saves one on every push to main. Entries older than seven days are evicted. That job no longer sets GATEWAY_DATABASE_URL.

Decisions

  • One lint pass rather than per-package tasks. Per-package tasks cache separately, so a change confined to one app re-linted only that app, which was quicker than the single pass: about 10 s whatever changed, and nothing when nothing did. In exchange a cold run halves, a change to a shared package no longer fans out to every dependent, and every file is linted rather than each workspace's src.
  • The type check relies on oxlint's --type-check, which oxlint labels experimental. It still builds one program per tsconfig.json. A file outside every include is checked with compiler defaults rather than skipped, which is the one way it differs from tsc.
  • apps/api's Prisma client stays uncached, so prisma generate runs on every turbo invocation. libnest imports @prisma/client itself, so the client cannot move to an output turbo can restore.
  • The release version falls back to package.json rather than failing. The release workflow already passes that same value, so the variable and the file cannot disagree in a release build.
  • The root package.json is a declared build input for three apps only. The release script's bump already changed their cache keys, but only because it also rewrites a package they depend on. Declaring it for every build would rebuild the libraries on any root script edit.
  • test:e2e keeps ^build beside its named prerequisites. One playground spec runs the built playground-url CLI, and ^build is what builds it; the first version of this change dropped it and the suite failed in CI on that one test.
  • The adversarial review used chore: migrate to oxlint, oxfmt and typescript 7 #1927's head as its base, not upstream main, so it covers this change alone. It ran before the last two commits, which change turbo.json files and one doc and none of the files the review read.

Risks

  • The release workflow's validate job changes here and cannot run before this merges. It gains a cache step and loses an environment variable that only the removed db:push read.
  • A task that read NODE_ENV from the calling shell would now see it unset. Every dev, dev:test, start and app build script sets it inline, and the library builds do not read it.
  • The gateway image is built without a db push at build time. scripts/run.sh applies the schema on every container start, as before.

Notes

  • All 12 build tasks pass on this branch, with and without RELEASE_VERSION set.
  • Deleting packages/runtime-core/{lib,dist} and rebuilding from cache restores both.
  • turbo prune for web and gateway followed by a dry-run build in the pruned output resolves, so the Docker builds accept the new root turbo.json and the per-package ones.
  • A type error injected into a src file, a build script, a *.test-d.ts file, a checked .js file and an e2e spec was reported identically by oxlint --type-check and by tsc.
  • With packages/playground-url/dist deleted, a filtered test:e2e run of the spec that failed in CI builds the CLI first and passes.

Adversarial review

Reviewed at commit bfc9345. pnpm check: green.
Follow-up 0c62e69: restores the ^build prerequisite of test:e2e after the end-to-end failure in CI; no new code.

Findings

CLEAN

Attestation

I, Claude Fable 5.2, attest that the review block above reports the adversarial review as it ran on the commit named, with every finding it returned.

🤖 Generated with Claude Code

joshunrau and others added 7 commits November 18, 2026 20:02
Caching was defeated by how tasks were keyed, and lint paid for 28
separate type-checks.

Cache keys:
- .env is no longer a global dependency. It is generated with random
  secrets per clone, so no cache entry was ever valid in another
  checkout, and every edit to it reran every task. A task still sees a
  variable only through its `env` key.
- NODE_ENV leaves globalEnv and NODE_OPTIONS becomes pass-through, and
  the build `env` list moves to the two apps that read it, so the
  library builds hash the same under `pnpm lint`, `pnpm build` and
  `pnpm dev` instead of three different ways.
- TURBO_CACHE_DIR leaves .env.template. It reached turbo only through
  env-cmd, which split the cache across two directories and switched
  off the cache sharing turbo does between git worktrees.
- runtime-core's build declares `lib/**` as an output. A cache hit
  restored `dist` alone, leaving the `./constants` export unresolved.
- `build` hashes neither `*.md` nor `__tests__/`, `db:generate` hashes
  the schema alone, and outreach hashes the /docs and /blog trees its
  content links to, which turbo saw only as symlinks.

Graph:
- `lint` no longer depends on `db:push`, and nothing depends on the
  never-matching `^db:generate`. `db:push` is defined in the gateway,
  the one workspace that has it, and its build no longer runs it.
- `//#generated` names what a whole-repo tool reads that source control
  does not hold. `test:e2e` depends on it explicitly, so the task is
  correct under `--filter` rather than only when every workspace is in
  scope.
- `pnpm test` and `pnpm test:coverage` build the packages that export
  only build output first, instead of reading whatever `dist` is on
  disk.
- `start` depends on its own `build`, and persistent tasks are
  uncached.

Lint:
- `pnpm lint` runs one `oxlint --type-check --fix` over the repo in
  place of `tsc && oxlint --fix src` in each workspace; outreach keeps
  `astro check`. A cold run drops from 22-25s to 12s on 11 cores.
- That pass also covers the files the per-package scripts skipped
  (vite configs, build scripts, runtime/v1's tests), whose findings
  are fixed here.
- `pnpm format` is one `oxfmt` run rather than 26 turbo tasks.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Every CI run started from an empty cache, and until .env left turbo's
global hash a saved one could not have matched anyway. A follow-up push
now reruns only the tasks whose inputs it changed.

The release workflow's lint no longer invents a GATEWAY_DATABASE_URL:
lint stopped running the gateway's `db:push`.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A production build read its version from RELEASE_VERSION alone, which
only the Dockerfiles set, so root `pnpm build` failed at the api build
on every development machine. It now takes the root package.json
version when the variable is unset, which is the value the release
workflow passes anyway. A malformed value still fails the build.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
CI runs only on pull requests, and a pull request can read a cache
saved on its base branch but not one saved by another pull request, so
the first run of each started cold. The release workflow's lint job
now saves turbo's cache on every push to main.

Each run saves what it restored plus its own entries, so both jobs cap
the age of an entry at seven days.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The playbooks and AGENTS.md files still named per-package `lint`
scripts, a `lint` that ran the gateway's `db:push`, a runtime-core
`lib/` that turbo did not restore, and `.env` as a global dependency.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A production build of the api, the gateway or the web app now reads its
version from the root package.json when RELEASE_VERSION is unset, and
nothing in their cache keys named that file. A version bump made by the
release script changed the keys only because it also rewrites a package
those builds depend on.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions github-actions Bot added Status: Waiting on Review Automated: unreviewed, pushed since the last review, or review re-requested Status: CI Failing Automated: a check failed; the author fixes it and removed Status: Waiting on Review Automated: unreviewed, pushed since the last review, or review re-requested labels Nov 19, 2026
Naming the suite's prerequisites explicitly dropped `^build`, which was
what built packages/playground-url/dist/cli.js, and one playground spec
runs that file. The suite passed on a machine that had built the CLI
before and failed in CI, which had not.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

This pull request was closed because its body does not meet .agents/skills/opening-pr/SKILL.md:

  • The review block names no commit matching the head 0c62e69c2dbc9b72328acdf8a73fbda7cc8769cc.

Fix the body with gh pr edit <number> --body-file <path>, then gh pr reopen <number>.

@github-actions github-actions Bot closed this Nov 19, 2026
@joshunrau joshunrau reopened this Nov 19, 2026
@github-actions github-actions Bot added Status: Waiting on Review Automated: unreviewed, pushed since the last review, or review re-requested and removed Status: CI Failing Automated: a check failed; the author fixes it labels Nov 19, 2026
@joshunrau
joshunrau merged commit 3156373 into DouglasNeuroInformatics:main Nov 19, 2026
6 of 7 checks passed
@joshunrau
joshunrau deleted the chore/turbo-task-graph branch November 19, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Waiting on Review Automated: unreviewed, pushed since the last review, or review re-requested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant