Skip to content

test: add coverage and improve tests - #218

Merged
Exeloo merged 3 commits into
feat/add-project-compatibilityfrom
test/improve-tests
Nov 5, 2026
Merged

Exeloo merged 3 commits into
feat/add-project-compatibilityfrom
test/improve-tests

Conversation

@Exeloo

@Exeloo Exeloo commented Oct 30, 2026

Copy link
Copy Markdown
Member

What does this PR do?

Add coverage and improve tests

How do you test this PR?

@Exeloo
Exeloo requested a review from Tchips46 as a code owner October 30, 2026 14:04
@github-actions github-actions Bot added apps:editor Related to Editor app packages:code Related to Code package packages:history Related to History package packages:kernel Related to Kernel package packages:layout Related to Layout package packages:meta Related to Meta package packages:project Related to Project package packages:protocol Related to Protocol package packages:registry Related to Registry package packages:rpc Related to RPC package packages:runtime Related to Runtime package packages:sdk Related to SDK package packages:server-core Related to Server core package packages:settings Related to Settings package packages:ui Related to UI package plugins:code-editor Related to Code editor plugin plugins:command-palette Related to Command palette plugin plugins:console Related to Console plugin plugins:ecs Related to ECS plugin plugins:file-manager Related to File manager plugin plugins:git Related to Git plugin plugins:history-panel Related to History panel plugin plugins:inspectors Related to Inspectors plugin plugins:packages Related to Packages plugin plugins:scene Related to Scene plugin plugins:settings-ui Related to Settings ui plugin plugins:viewport Related to Viewport plugin tooling:vite-plugin Related to Vite plugin tooling labels Oct 30, 2026
@Exeloo
Exeloo added this pull request to stack #212 October 30, 2026 14:04
@Exeloo
Exeloo force-pushed the test/improve-tests branch 2 times, most recently from a5d62c0 to bd13bb6 Compare November 1, 2026 14:27
@Exeloo
Exeloo force-pushed the test/improve-tests branch from bd13bb6 to 60e2bb6 Compare November 1, 2026 14:52
@Exeloo
Exeloo force-pushed the test/improve-tests branch from 60e2bb6 to 0b5e61b Compare November 1, 2026 15:39
@Exeloo
Exeloo force-pushed the test/improve-tests branch from 0b5e61b to 778045e Compare November 5, 2026 11:41
- code: checks wait with vi.waitFor and longer timeouts under coverage
- e2e: the tutorial copy gets its own nf, check-build watches tutorial
- knip: mkfifo used by the fs tests
- command-palette: cover the keyboard shortcuts rows
@Exeloo
Exeloo force-pushed the test/improve-tests branch from 778045e to 9a299b7 Compare November 5, 2026 11:55

@Tchips46 Tchips46 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: tests and coverage (9a299b7)

Line numbers are at this PR's head. Most of the new specs are sound, and splitting the long git e2e test into independent ones is a good change. The findings below are about the CI wiring, the e2e fixtures, and some production code that ships inside this test PR.

Should fix

1. Account sync status hides errors, and saved() can return before anything was pushed (packages/settings/src/account-sync/account-sync-store.ts:112, apps/editor/src/lib/editor/follow-saved.ts:15)

  • write() now sets the status to pending unless it is syncing. When the status was error (for example "too large to sync") or conflict, any edit replaces it with "Changes waiting to sync", and the error text, which only shows while the status is error, disappears.
  • followSaved counts offline and error as saved. In e2e, a failed or offline push makes saved() return at once, which is the stale-account leak between tests that this helper exists to prevent.
  • Fix: switch to pending only from idle. Treat offline and error as not saved, so saved() times out loudly.

2. A PR that only moves the engine or CLI pin never runs e2e (.github/workflows/e2e.yml:9-19)

  • scripts/** is not in the paths list, but scripts/pinned-refs.json now picks the engine and CLI that e2e builds, and scripts/engine-bridge.mjs drives the workspace setup.
  • Example: a PR that bumps only the pin is merged with no e2e run, and a bad pin first fails in the nightly run on main.
  • Fix: add 'scripts/**' to paths.

3. Game cleanup failures are swallowed, and the scene spec now cleans up while the game plays (apps/editor/e2e/test.ts:35-47, apps/editor/e2e/scene.spec.ts)

  • Every step of the new stopsTheGame fixture ends in .catch(() => undefined). When Stop hangs, the test passes, and the next test fails with the "server exits at once" cascade that CLAUDE.md warns about, which hides the real cause.
  • scene.spec.ts dropped its own stop hook. Playwright runs afterEach hooks before fixture teardown, so the spec now restores the breakout sources, resets the layout and calls saved() while the game is still playing.
  • Fix: drop the .catch on the wait for the Play button. Keep the scene spec's own stop, or stop the game first in its afterEach.

4. The keymap test asserts on its own mock (plugins/command-palette/test/keymap/keymap-rows.spec.ts:13-31)

  • The mock of @nanoforge-dev/editor-sdk/ui rewrites bindingsConflict and findConflicts as a.key === b.key. The real code (packages/ui/src/workbench/keybinding/resolve-keymap.ts:89) also checks when overlap and chord prefixes (Mod+K against Mod+K S).
  • The conflict assertions pass whatever the real functions return, so a regression in conflict detection ships with green tests.
  • Fix: have the mock factory re-export the real pure functions from resolve-keymap.ts and keystroke.ts.

5. One turbo cache key per commit is shared by four jobs, and the cache only grows (.github/actions/prepare/action.yml, "Cache turbo")

  • Lint, typecheck, unit and e2e all save under turbo-<ref_name>-<sha>, and only the first job to finish wins: the other jobs' task results are lost and rerun next time.
  • Each run restores the previous .turbo/cache and saves it again with more in it. Near GitHub's 10 GB limit this evicts the engine, Playwright and pnpm caches.
  • Fix: add the job name to the key, and clear .turbo/cache before saving (or drop the cross-branch restore fallback).

6. Production changes are hidden under a test: commit

  • This PR also changes production code: the diff editor fix (plugins/code-editor/src/monaco/dispose-diff-editor.ts), the SettingsService.pending/hold counters with holds in LayoutController and WorkbenchService, data-nf-saved on every page, the sync status change from point 1, and about 50 removed exports.
  • cliff.toml files ^test commits under "Testing", so the Monaco fix never shows as a fix in the changelog. CLAUDE.md also asks for core changes first, in their own commits.
  • Fix: split out fix(code-editor), feat(settings) / feat(ui) and refactor commits.

Nice to have

  • check-build (apps/editor/e2e/check-build.ts:6-13, 57) counts tooling sources as editor sources, but the command it prints (--filter=@nanoforge-dev/editor...) does not rebuild tooling/example-plugin. After an edit there, the check keeps failing or passes with a stale dev plugin. Fix: add --filter=./tooling/* to the printed command.
  • Duplicates and leftover sleeps: plugins/code-editor/test/document/editor-document.spec.ts:9-27 and type-libraries.spec.ts:8-40 rebuild the fake model that plugins/code-editor/test/fixtures/fake-monaco.ts (added in this PR) already provides. inspectors.spec.ts:31-37 repeats the new stopsTheGame fixture. inspectors.spec.ts:28 and project.spec.ts:37 still sleep 900 ms where saved() now covers it.

Not checked: nothing was run locally. CI is green on this head (e2e: 102 passed, 0 skipped).

- settings: a write only turns an idle sync status pending, so an error or
  conflict stays shown; follow-saved counts only idle and conflict as saved,
  so saved() times out on a failed or offline push
- ci: run e2e when scripts/ change (engine and CLI pins); one turbo cache per
  job and month
- e2e: the stopsTheGame fixture fails when Stop does not stop the game
  (shared stopGame helper); the scene spec stops the game before restoring
  sources; inspectors drops its duplicate stop hook and waits for saved()
  instead of 900 ms
- e2e: check-build prints a rebuild command that includes tooling/
- code-editor: editor-document spec uses the shared fakeModel fixture

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Exeloo

Exeloo commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Thanks for the review. Addressed in 1d26f8e (fix: address the tests review).

Fixed

  • 1. write() switches to pending only from idle, so error and conflict stay shown. followSaved counts only idle and conflict as saved, so saved() times out on error or offline. A new spec writes while in error. Note: the error text was only hidden until the next push, which sets error again; the saved() part was the real leak.
  • 2. scripts/** added to the e2e paths.
  • 3. A shared stopGame helper: only the visibility check may fail quietly, and the Stop click and the wait for Play throw. The scene spec stops the game first in its afterEach. The duplicate hook in inspectors.spec.ts is gone, and its 900 ms sleep is now saved().
  • 5. The turbo cache key now includes the job and the month (<os>-turbo-<job>-<YYYY-MM>-<ref>-<sha>). Restores stay within the same job and month, so the cache starts over each month.
  • check-build: the printed commands add --filter='./tooling/*'. It is quoted because zsh refuses the glob unquoted.
  • Duplicates: editor-document.spec.ts uses the shared fakeModel.

Not changed

  • 4. The real bindingsConflict/findConflicts are covered in packages/ui/test/workbench/keybinding/keybindings.spec.ts (when overlap and the Ctrl+K / Ctrl+K S prefix), so a regression there fails CI. The plugin spec tests the plugin's own wiring, and plugins only import the SDK, so mocking /ui is the convention (9 plugin specs do it).
  • 6. Splitting the commit would mean rewriting a pushed branch that two other PRs of the stack are built on. These PRs are squash-merged, so the merged commit's title and changelog entry can be set at merge time.
  • type-libraries.spec.ts: not a duplicate: it records the addExtraLib calls, and the shared fake does nothing with them.
  • project.spec.ts:37: that sleep waits for the Project screen's file writes, which saved() doesn't track, so it stays.

Checked at the top of the stack: lint, typecheck and unit tests pass, and e2e passed 109/109 (0 skipped) against the CI-pinned engine and CLI.

@Exeloo
Exeloo merged commit 69a8a81 into main Nov 5, 2026
6 checks passed
@Exeloo
Exeloo deleted the test/improve-tests branch November 5, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

apps:editor Related to Editor app packages:code Related to Code package packages:history Related to History package packages:kernel Related to Kernel package packages:layout Related to Layout package packages:meta Related to Meta package packages:project Related to Project package packages:protocol Related to Protocol package packages:registry Related to Registry package packages:rpc Related to RPC package packages:runtime Related to Runtime package packages:sdk Related to SDK package packages:server-core Related to Server core package packages:settings Related to Settings package packages:ui Related to UI package plugins:code-editor Related to Code editor plugin plugins:command-palette Related to Command palette plugin plugins:console Related to Console plugin plugins:ecs Related to ECS plugin plugins:file-manager Related to File manager plugin plugins:git Related to Git plugin plugins:history-panel Related to History panel plugin plugins:inspectors Related to Inspectors plugin plugins:packages Related to Packages plugin plugins:scene Related to Scene plugin plugins:settings-ui Related to Settings ui plugin plugins:viewport Related to Viewport plugin tooling:vite-plugin Related to Vite plugin tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants