Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .agents/docs/architecture/auth-and-permissions.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,10 @@ that a grantable pair used to reach belongs in the same predicate.
reads the hash; `AssignmentsService.findPrivateKeyById`, called only by the gateway synchronizer,
is the only read of the key.
- **Password policy is `UsersService.validatePassword`**: strength via libpasswd, not equal to the
username, and a Have I Been Pwned check that **fails open** when the API is unreachable. Failures
username, and a Have I Been Pwned check that **fails open** when the API is unreachable and is
skipped when `PASSWORD_BREACH_CHECK_ENABLED=false` (the e2e suite and the api vitest project). Failures
carry a `code` from `PASSWORD_ERROR_CODES` (`packages/schemas/src/user/user.ts`); the web client
localizes it, the API does not.

Row scoping is invisible to a unit test with a mocked model, so a change to who can see what needs an
end-to-end test in `testing/`.
integration test against the real database in `apps/api/test/suites/` (see `08-authorization.suite.ts`).
34 changes: 22 additions & 12 deletions .agents/docs/architecture/testing-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@ What is tested where, and what CI blocks a merge on. Writing an e2e test is

## Tiers

| Tier | Runner | Where | Command |
| ---------- | ---------- | --------------------------------------------------------------------- | -------------------- |
| Unit | vitest | `__tests__/<file>.test.ts` beside `<file>.ts`, or a package's `test/` | `pnpm test:coverage` |
| End-to-end | Playwright | `testing/src/specs/` | `pnpm test:e2e` |
| Type-check | oxlint | code that fails to compile if the type breaks, e.g. `*.test-d.ts` | `pnpm lint` |
| Tier | Runner | Where | Command |
| ----------- | ---------- | --------------------------------------------------------------------- | -------------------- |
| Unit | vitest | `__tests__/<file>.test.ts` beside `<file>.ts`, or a package's `test/` | `pnpm test:coverage` |
| Integration | vitest | `apps/api/test/suites/`, run through `apps/api/test/app.test.ts` | `pnpm test:coverage` |
| End-to-end | Playwright | `testing/src/specs/` | `pnpm test:e2e` |
| Type-check | oxlint | code that fails to compile if the type breaks, e.g. `*.test-d.ts` | `pnpm lint` |

**There is no integration tier.** `apps/api` unit tests mock the Prisma layer entirely, so Playwright
is the only code path that touches a real database: a permissions or row-scoping change is untested
until an e2e spec covers it.
`apps/api` unit tests mock the Prisma layer entirely. The integration suites boot the real app on an
in-memory replica set (`apps/api/AGENTS.md` §Tests), so a permissions or row-scoping change is tested
there, at a few milliseconds a case; an e2e spec is for what needs the browser and the other apps.

## Scoping a run

Expand All @@ -24,6 +25,12 @@ from the `name` field in each package's `vitest.config.ts`, e.g. `api`, `web`, `
single file with `pnpm exec vitest path/to/file.test.ts`. A scoped run with `--coverage` measures
only what it ran against the whole coverage scope, so add `--coverage.thresholds.100=false` to it.

**`pnpm test:coverage` and `pnpm test:e2e` take one lock per machine.** `scripts/with-test-lock.sh`
holds a directory in the git common directory, which every worktree shares, so a second heavy run —
from another session or worktree — waits and says whose run it is waiting on, rather than exhausting
memory or colliding on the e2e ports. A lock whose holder has exited is taken over. `vitest` without
`--coverage` and the scoped runs above take no lock.

## A test nothing collects passes silently

Root `vitest.config.ts` collects only `.github/vitest.config.ts`, `apps/*/vitest.config.ts`,
Expand Down Expand Up @@ -108,9 +115,12 @@ a lint failure and every run may rewrite the working tree. To check part of the

`pnpm lint` is `turbo run lint lint:root`. `//#lint:root` depends on `//#generated`, a task with no
script that names what a tool run over the whole repo reads and source control does not hold: both
Prisma clients, `testing`'s route types, and `instrument-library`'s build, which reaches `runtime-v1`
and `runtime-core` through `^build`. `test:e2e` depends on the same task, the gateway's `db:push`, and
`^build`, which builds the `playground-url` CLI that `testing/src/specs/playground.spec.ts` runs.
Prisma clients, `testing`'s route types, `apps/outreach`'s content collection types (`gen:types`,
`astro sync`, into the git-ignored `.astro/`), and `instrument-library`'s build, which reaches
`runtime-v1` and `runtime-core` through `^build`. Without the content types, the type-check reads every
`astro:content` import as `any` on a fresh checkout such as CI's. `test:e2e` depends on the same task, the gateway's `db:push`, the
`build` of gateway, web and playground, whose production builds the suite serves, and `^build`, which
builds the `playground-url` CLI that `testing/src/specs/playground.spec.ts` runs.
`dev` and `dev:test` depend on `["^build", "db:generate"]`, to which `apps/gateway/turbo.json` adds
`db:push`; `build` depends on `["^build", "db:generate"]` and `start` on `build`. `dev`, `dev:test`,
`start`, `test:e2e` and `db:push` are `cache: false`. A workspace whose task differs from the root
Expand All @@ -137,7 +147,7 @@ counts their functions twice in the merged report.

`.github/workflows/ci.yaml`, one job, on `pull_request` to `main` or a `release/**` branch and
`workflow_dispatch` only — a push alone runs nothing. In order: `./scripts/generate-env.sh`, `pnpm install --frozen-lockfile`,
`pnpm lint`, `pnpm test:coverage`, Playwright install (chromium + firefox, capped at 10 minutes, after
`pnpm lint`, `pnpm test:coverage`, Playwright install (chromium, capped at 10 minutes, after
lint and unit tests so a hung download cannot hide them), `pnpm test:e2e`. `actions/cache` restores
`.turbo/cache` before the first of them: a pull request reads the caches its own earlier runs saved
and the one the Release workflow's `validate` job last saved on its base branch.
Expand Down
10 changes: 7 additions & 3 deletions .agents/docs/playbooks/add-e2e-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ Conventions and fixtures live in `testing/AGENTS.md`; read it first. This file i
operations.

**Once per machine:** a repo-root `.env` (`pnpm generate:env`) and the browsers,
`pnpm --filter @opendatacapture/testing exec playwright install chromium firefox`. Nothing else needs
starting — `playwright.config.ts` boots api, gateway, web and playground itself.
`pnpm --filter @opendatacapture/testing exec playwright install chromium`. Nothing else needs
starting — `playwright.config.ts` boots api, gateway, web and playground itself, serving the last
build of the last three: `pnpm test:e2e` builds them first, `test:dev` and a bare `playwright test` do
not.

## Steps

Expand Down Expand Up @@ -39,7 +41,9 @@ starting — `playwright.config.ts` boots api, gateway, web and playground itsel
8. **Set the role and first-run state** with `test.use({ actingRole: 'ADMIN' })` (default
`GROUP_MANAGER`) and `test.use({ appState: {...} })` (`testing/src/specs/disclaimer.spec.ts`).

9. **Tag `@smoke` only a critical flow**; those are the only tests firefox runs.
9. **Check whether a cheaper tier proves it first.** A rule in one component, hook or route is a unit
test; a query's row scoping is an `apps/api/test/suites/` test. An e2e spec is for what needs the
browser and the other apps at once.

## Verify

Expand Down
2 changes: 1 addition & 1 deletion .agents/docs/playbooks/add-web-route.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Conventions live in `apps/web/AGENTS.md`; read it first. This file is only the o

**Plan around this:** `apps/web/src/route-tree.ts` is generated by the `tanstackRouter` plugin in
`apps/web/vite.config.ts`, and **only the user regenerates it**. Every Vite run of `apps/web` rewrites
it — `pnpm dev`, `vite build`, and the `pnpm dev:test` that `pnpm test:e2e` boots. Until the user
it — `pnpm dev`, `pnpm dev:test`, and the `vite build` that `pnpm test:e2e` runs first. Until the user
regenerates, `tsc` fails on your new file with:

```
Expand Down
4 changes: 2 additions & 2 deletions .agents/skills/troubleshooting/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ that is deliberately left alone is recorded in the owning workspace's `AGENTS.md
## Loud and correct — leave it standing

- **TS2345 on a route literal you just added** is expected until the user regenerates `route-tree.ts`
(`.agents/docs/playbooks/add-web-route.md`). Every Vite run in `apps/web` (`dev`, `dev:test`, the Playwright
webServer, `vite build`) rewrites that tracked file; if `git status` shows it after a debugging run, revert it.
(`.agents/docs/playbooks/add-web-route.md`). Every Vite run in `apps/web` (`dev`, `dev:test`, `vite build`, which
`pnpm test:e2e` runs first) rewrites that tracked file; if `git status` shows it after a debugging run, revert it.
- **`pnpm lint` passed and the tree changed**: `oxlint --fix` sorted your new code. Keep the rewrite.
- **`Failed to find .env file at default paths`**: run `pnpm generate:env` (root `AGENTS.md`).
- **ENOENT while a Vite config resolves, or `/runtime/v1/*` unresolved**: `runtime/v1/dist` is unbuilt. Turbo
Expand Down
3 changes: 3 additions & 0 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,9 @@ GATEWAY_API_KEY=
# Optional server-wide GitHub token used to sync instrument repositories. Only needed for private
# repos that were registered without their own access token; leave blank otherwise.
GITHUB_TOKEN=
# Whether a password being set is checked against the Have I Been Pwned API, which needs outbound
# internet access (default: true). The end-to-end suite turns it off so it never depends on that server.
PASSWORD_BREACH_CHECK_ENABLED=
# Whether 'debug' level logs should be enabled
DEBUG=false
# Whether 'verbose' level logs should be enabled
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,10 @@ jobs:
run: pnpm test:coverage
- name: Install Playwright
# Cap the step so a hang fails fast instead of hitting the 6h job limit, skip the interactive
# apt prompt, and install only the browsers the e2e suite uses.
# apt prompt, and install only the browser the e2e suite uses.
timeout-minutes: 10
env:
DEBIAN_FRONTEND: noninteractive
run: pnpm --filter "@opendatacapture/testing" exec playwright install --with-deps chromium firefox
run: pnpm --filter "@opendatacapture/testing" exec playwright install --with-deps chromium
- name: End-to-End Tests
run: pnpm test:e2e
2 changes: 1 addition & 1 deletion apps/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"build": "rm -rf dist && NODE_ENV=production libnest build -c libnest.config.ts",
"db:generate": "prisma generate --no-hints",
"dev": "NODE_ENV=development NODE_OPTIONS='--conditions=development' env-cmd -f ../../.env libnest dev -c libnest.config.ts",
"dev:test": "NODE_ENV=test env-cmd -f ../../.env env STORAGE_ENABLED=true libnest dev -c libnest.config.ts --no-watch",
"dev:test": "NODE_ENV=test env-cmd --no-override -f ../../.env env PASSWORD_BREACH_CHECK_ENABLED=false STORAGE_ENABLED=true libnest dev -c libnest.config.ts --no-watch",
"start": "NODE_ENV=production env-cmd -f ../../.env node dist/app.js",
"test": "env-cmd -f ../../.env vitest"
},
Expand Down
11 changes: 11 additions & 0 deletions apps/api/src/assignments/__tests__/assignments.service.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -650,6 +650,17 @@ describe('AssignmentsService', () => {
expect(gatewayService.deleteRemoteAssignment).not.toHaveBeenCalled();
});

it("should scope the existence check to the caller's groups, so a manager outside the group cannot kill its link", async () => {
assignmentModel.exists.mockResolvedValueOnce(false);
await expect(
assignmentsService.updateById('assignment-1', { status: 'CANCELED' }, userAt('GROUP_MANAGER'))
).rejects.toBeInstanceOf(NotFoundException);
expect(assignmentModel.exists.mock.lastCall?.[0]).toEqual({
AND: [{ OR: [{ groupId: { in: [GROUP_ID] } }] }],
id: 'assignment-1'
});
});

it('should delete the gateway copy of an assignment the caller cancels, so its link stops working', async () => {
assignmentModel.exists.mockResolvedValueOnce(true);
assignmentModel.update.mockResolvedValueOnce({ groupId: GROUP_ID, id: 'assignment-1' });
Expand Down
7 changes: 7 additions & 0 deletions apps/api/src/core/schemas/__tests__/env.schema.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,11 @@ describe('$Env', () => {
it('should not require storage variables while storage is disabled', () => {
expect($Env.safeParse({ ...testEnv, STORAGE_ENABLED: 'false' }).success).toBe(true);
});

it('should check passwords against known breaches unless it is turned off, so a deployment is protected by default', () => {
expect($Env.parse(testEnv).PASSWORD_BREACH_CHECK_ENABLED).toBe(true);
expect($Env.parse({ ...testEnv, PASSWORD_BREACH_CHECK_ENABLED: 'false' }).PASSWORD_BREACH_CHECK_ENABLED).toBe(
false
);
});
});
1 change: 1 addition & 0 deletions apps/api/src/core/schemas/env.schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export const $Env = $BaseEnv
GATEWAY_REFRESH_INTERVAL: $NumberLike.pipe(z.number().positive().int()),
GATEWAY_SITE_ADDRESS: $UrlLike.optional(),
GITHUB_TOKEN: z.string().min(1).optional(),
PASSWORD_BREACH_CHECK_ENABLED: $BooleanLike.default(true),
STORAGE_ACCESS_KEY: z.string().min(1).optional(),
STORAGE_BUCKET: z.string().min(1).optional(),
STORAGE_ENABLED: $BooleanLike.optional(),
Expand Down
20 changes: 20 additions & 0 deletions apps/api/src/sessions/__tests__/sessions.service.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,26 @@ describe('SessionsService', () => {
expect(sessionModel.createMany.mock.lastCall?.[0]).toMatchObject({ data: [{ groupId: 'group-1' }] });
});

// The batched path once handed over only the ids, so a new subject lost the demographics its
// first session named.
it('should hand every subject over with the demographics its session names', async () => {
const subjectData = {
dateOfBirth: new Date('1990-05-17T00:00:00.000Z'),
firstName: 'Ada',
id: 'subject-1',
lastName: 'Lovelace',
sex: 'FEMALE' as const
};

await sessionsService.createMany({
entries: [{ date: new Date(), subjectData }],
groupId: null,
type: 'IN_PERSON'
});

expect(subjectsService.createMany).toHaveBeenCalledExactlyOnceWith([subjectData]);
});

it('should associate the batch with the group in a single write', async () => {
groupsService.findById.mockResolvedValueOnce({ id: 'group-1' } as any);

Expand Down
13 changes: 12 additions & 1 deletion apps/api/src/users/__tests__/users.service.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { CryptoService, getModelToken } from '@douglasneuroinformatics/libnest';
import { ConfigService, CryptoService, getModelToken } from '@douglasneuroinformatics/libnest';
import type { Model, RequestUser } from '@douglasneuroinformatics/libnest';
import { MockFactory } from '@douglasneuroinformatics/libnest/testing';
import type { MockedInstance } from '@douglasneuroinformatics/libnest/testing';
Expand Down Expand Up @@ -38,6 +38,7 @@ const baseUser = {
describe('UsersService', () => {
let usersService: UsersService;
let userModel: MockedInstance<Model<'User'>>;
let configService: MockedInstance<ConfigService>;
let cryptoService: MockedInstance<CryptoService>;
let groupsService: MockedInstance<GroupsService>;
let permissionSetsService: MockedInstance<PermissionSetsService>;
Expand All @@ -49,20 +50,23 @@ describe('UsersService', () => {
UsersService,
MockFactory.createForModelToken(getModelToken('User')),
MockFactory.createForService(AuditLogger),
MockFactory.createForService(ConfigService),
MockFactory.createForService(CryptoService),
MockFactory.createForService(GroupsService),
MockFactory.createForService(PermissionSetsService)
]
}).compile();
userModel = moduleRef.get(getModelToken('User'));
usersService = moduleRef.get(UsersService);
configService = moduleRef.get(ConfigService);
cryptoService = moduleRef.get(CryptoService);
groupsService = moduleRef.get(GroupsService);
permissionSetsService = moduleRef.get(PermissionSetsService);

userModel.exists.mockResolvedValue(false);
userModel.create.mockResolvedValue({});
cryptoService.hashPassword.mockResolvedValue('hashed-password');
configService.get.mockReturnValue(true);
// By default the password is strong and has not appeared in a breach.
(estimatePasswordStrength as Mock).mockReturnValue({ feedback: {}, score: 4, success: true });
(pwnedPassword as Mock).mockResolvedValue(0);
Expand Down Expand Up @@ -290,6 +294,13 @@ describe('UsersService', () => {
expect(userModel.create).toHaveBeenCalledOnce();
});

it('should not contact Have I Been Pwned when the breach check is disabled, so an offline instance never waits on it', async () => {
configService.get.mockReturnValue(false);
await usersService.create({ ...baseUser });
expect(pwnedPassword).not.toHaveBeenCalled();
expect(userModel.create).toHaveBeenCalledOnce();
});

it('should persist mustResetPassword, so a generated password forces a reset at first sign-in', async () => {
await usersService.create({ ...baseUser, mustResetPassword: true });
expect(userModel.create.mock.lastCall?.[0]).toMatchObject({ data: { mustResetPassword: true } });
Expand Down
7 changes: 6 additions & 1 deletion apps/api/src/users/users.service.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { CryptoService, InjectModel } from '@douglasneuroinformatics/libnest';
import { ConfigService, CryptoService, InjectModel } from '@douglasneuroinformatics/libnest';
import type { Model, RequestUser } from '@douglasneuroinformatics/libnest';
import { estimatePasswordStrength } from '@douglasneuroinformatics/libpasswd';
import {
Expand Down Expand Up @@ -35,6 +35,7 @@ export class UsersService {
constructor(
@InjectModel('User') private readonly userModel: Model<'User'>,
private readonly auditLogger: AuditLogger,
private readonly configService: ConfigService,
private readonly cryptoService: CryptoService,
private readonly groupsService: GroupsService,
private readonly permissionSetsService: PermissionSetsService
Expand Down Expand Up @@ -353,6 +354,7 @@ export class UsersService {
* `BadRequestException` if the password is too weak, matches the username, or has
* appeared in a known data breach. The breach check fails open: if the Have I Been
* Pwned API is unreachable (e.g. an air-gapped deployment), the password is allowed.
* `PASSWORD_BREACH_CHECK_ENABLED=false` skips the check, so nothing waits on that server.
*/
private async validatePassword(password: string, username: string): Promise<void> {
if (!estimatePasswordStrength(password).success) {
Expand All @@ -361,6 +363,9 @@ export class UsersService {
if (password.toLowerCase() === username.toLowerCase()) {
throw this.passwordError('PASSWORD_MATCHES_USERNAME', 'Password must not be the same as the username');
}
if (!this.configService.get('PASSWORD_BREACH_CHECK_ENABLED')) {
return;
}
let breachCount: number;
try {
breachCount = await pwnedPassword(password);
Expand Down
23 changes: 23 additions & 0 deletions apps/api/test/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,26 @@ export function defineSuite(name: string, fn: (this: TestSuiteThis) => Promise<v
});
};
}

/**
* A form with one number field, as the bundle the api stores. A bundle normally imports zod from
* `/runtime/v1`, which does not resolve under vitest (#1653), so this one imports the api's own zod
* by its file URL. The instrument id derives from `name`, so each bundle needs a name of its own.
* `measured` makes the score a measure, which is what an export writes a row for.
*/
export function scoreFormBundle(name: string, { measured = false }: { measured?: boolean } = {}) {
return `(async () => {
const { z } = await import(${JSON.stringify(import.meta.resolve('zod/v4'))});
return {
__runtimeVersion: 1,
content: { score: { kind: 'number', label: 'Score', variant: 'input' } },
details: { description: 'A form with one number field', license: 'Apache-2.0', title: ${JSON.stringify(name)} },
internal: { edition: 1, name: ${JSON.stringify(name)} },
kind: 'FORM',
language: 'en',
measures: ${measured ? "{ score: { kind: 'const', label: 'Score', ref: 'score' } }" : 'null'},
tags: [],
validationSchema: z.object({ score: z.number() })
};
})()`;
}
Loading