Skip to content

ci: cut test time and skip e2e on PRs that cannot affect it - #419

Merged
danii1 merged 5 commits into
mainfrom
ci/faster-tests
Nov 14, 2026
Merged

danii1 merged 5 commits into
mainfrom
ci/faster-tests

Conversation

@danii1

@danii1 danii1 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

CI's Test step took about 4m50s (run on #418). Nearly all of it was the offline e2e suite (packages/e2e/tests/local, 292s). Turbo runs it in parallel with the code unit tests (92s), so on its own it set the step's wall time. Two automation tests caused 165s of it by waiting for real cron minute boundaries.

These commits were pushed to fix/automations-respect-usage-limit after #418 merged, so this PR brings them to main.

Changes

CI

  • tests/local runs in its own e2e-local job, in parallel with verify.
  • On pull requests, the job runs turbo --affected. The suite runs only when e2e, packages/code, or a package code depends on changed since the base branch; otherwise the job passes without running any tests. Pushes to main always run the full suite. TURBO_SCM_BASE is set to origin/<base>, because the checkout has no local main branch; without it, turbo assumed every file had changed.
  • @getdevintern/code is now an e2e devDependency. turbo.json already built code before e2e#test, but turbo's package graph didn't link the two, so --affected ignored code changes.
  • ci.yml joins turbo's globalDependencies, so a workflow change runs everything. This PR edits it, so its own run executes the full e2e suite.

Faster tests

  • Automation e2e: schedules use a cron that never fires on its own (0 0 1 1 *). The test stops the worker, moves the durable cursor into the past, and restarts it, so the startup tick claims the occurrence at once. One real * * * * * wait stays, for the live timer after a config reload. About 165s → 37s.
  • worker add --wait tests: the poll interval can be injected (waitPollIntervalMs), and the tests poll every 20ms. 23s → 5.5s.
  • actioned-state tests: the re-read retry delay can be injected (rereadDelayMs), and the tests pass 0. 3.7s → 0.2s.

Both new options leave production defaults unchanged.

Bug fix: report redaction

  • The email regex was quadratic on long runs with no @. Redacting 100k characters took 2.2s, and devintern report runs it on log attachments up to 64KB. A lookbehind now anchors each match to the start of a local-part run, so the output is the same and the scan is linear.

Tests

  • @getdevintern/code: 2902 pass, 0 fail. Locally the suite went from 60s to 39s.
  • Local e2e suite: 217 tests, 0 fail.
  • New redaction tests: emails next to punctuation are still redacted, and a 1MB line is redacted in linear time (the old regex would hit the test timeout).
  • Checked turbo's selection with --dry-run against past commits: a pm-desktop-only change skips e2e, and code or dashboard-ui changes run it.

🤖 Generated with Claude Code

danii1 and others added 5 commits November 14, 2026 13:05
The email pattern had no left anchor, so a long run with no @ (a base64
blob, a minified log line) was rescanned from every offset: 100k
characters took over two seconds, and devintern report applies it to log
attachments up to 64KB. A lookbehind anchors matches to the start of a
local-part run; leftmost-greedy matching already started there, so the
redacted output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- e2e worker-automations: register schedules with a never-firing cron,
  then stop, move the durable cursor into the past and restart so the
  startup tick claims them, instead of waiting for real minute
  boundaries. One real boundary stays for the live-reload timer.
  ~165s -> ~37s.
- worker add --wait: poll interval is injectable (waitPollIntervalMs);
  tests poll every 20ms and shorten restart-gap sleeps. 23s -> 5.5s.
- recordTaskActioned: re-read retry delay is injectable (rereadDelayMs);
  tests pass 0. 3.7s -> 0.2s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/local was the critical path of the verify job. It now runs in its
own e2e-local job (turbo builds packages/code first), and verify runs
every other package's tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e2e#test already built packages/code first via turbo.json, but the
package graph only linked e2e to agent-harness, so turbo --affected
never treated a packages/code change as affecting e2e. Declare
@getdevintern/code as an e2e devDependency, and run the e2e job with
--affected on pull requests (full history checkout for the merge base).
Pushes to main still run the full suite. ci.yml joins turbo's global
dependencies so a workflow change runs everything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
actions/checkout fetches the base as origin/main without a local main
branch, so turbo could not resolve the PR base, assumed every file
changed, and never skipped the e2e suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danii1
danii1 merged commit dd9b2fa into main Nov 14, 2026
2 checks passed
@danii1
danii1 deleted the ci/faster-tests branch November 14, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant