Skip to content

feat: let an administrator define the fields that identify subjects - #1935

Open
joshunrau wants to merge 19 commits into
DouglasNeuroInformatics:mainfrom
joshunrau:session-changes
Open

joshunrau wants to merge 19 commits into
DouglasNeuroInformatics:mainfrom
joshunrau:session-changes

Conversation

@joshunrau

@joshunrau joshunrau commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

An administrator can now define the fields that identify a subject, such as an animal ID and a cage number, and choose which identification methods staff are offered. Ids are derived from those fields in the browser, so no typed value is stored. The admin settings page also splits into tabs that take in the mail server page, and two existing subject-id rules are corrected.

This PR is stacked on #1933. Its first four commits are #1933's, so until #1933 merges, review from 2204b3ede onward.

Closes #1546. Closes #1684. Closes #1751.

Changes

  • Settings tabs.
    • /admin/settings now has Features, Interface, Mail and Subject Identification tabs. The first tab in translated-label order opens by default, and the chosen tab is kept in the address as ?tab=.
    • Mail holds what was /admin/mail. That route is removed with no redirect, and the sidebar's Instance group keeps only Settings and Branding.
    • On an instance without the gateway, the Remote Assignments section is hidden; there, its switch did nothing.
  • Custom identifying fields.
    • On the Subject Identification tab, an administrator defines a label for the method and one to ten fields. Each field has a label, optional help text, a pattern and an optional error message, and a test box shows how a typed value normalizes and whether the pattern accepts it.
    • Saving the first definition asks for confirmation. GET and PUT /v1/subjects/identification/settings read and write it, and the server adds a random 32-byte salt and 600,000 PBKDF2 iterations.
    • Once any subject has a cf_ id, the stored definition is locked: removing it or changing its number of fields is refused with 409, and the editor stops adding, removing or reordering fields. Labels, help text, messages and patterns stay editable. With no definition stored there is nothing to lock, so a cf_ id that arrived some other way, such as a CSV upload, blocks no save.
  • The CUSTOM_FIELDS method.
    • Start Session and the data hub's subject lookup ask for every field and check each pattern against the normalized value. The browser then derives cf_ plus the PBKDF2-SHA-256 digest in hex.
    • Only that id leaves the browser, with the optional date of birth and sex. A returning subject's details fill in and lock, as they do for a custom id.
    • A group can make the method its default. A cf_ id is never scoped to a group and is kept out of custom-id suggestions.
  • Offered methods.
    • The tab also chooses which methods staff are offered. An instance that never chose offers Custom ID and Personal Info, as before. At least one method must be offered, and custom fields only once they are defined.
    • A method that is not offered disappears from Start Session, the data hub lookup and the group default-method select. A form whose group's default is not offered opens on the first offered method, and saving the group's settings keeps the stored default unless a manager picks another.
    • The tab lists its two sections, and the method checkboxes, by translated label, as the other tabs do.
    • Before a method is turned off, the tab shows how many subjects it identifies, from GET /v1/subjects/identification/counts (admin only).
  • Name hash (Subject ID hash keeps spaces, punctuation and Unicode hyphens that the form says are removed, so one person gets several subject IDs #1684).
    • The name help text and docs/en/4-concepts/4.2-hash-based-identification.md now say what the hash ignores: letter case, plain hyphens and accents, with other scripts romanized. They also say what still counts: spaces, apostrophes and other punctuation.
    • The hash itself is unchanged. Tests now pin its real digests, and transliteration is pinned to exactly 2.6.1, the version the lockfile already resolved, because its table is part of every stored Personal Info id.
  • Scoping rule (CSV upload scopes personal-information subject hashes to the group, creating a duplicate subject instead of adding to the existing one #1751).
    • How a typed id becomes a stored one moves into subject-utils as toStoredSubjectId, and CSV upload now uses it too.
    • A pasted name hash is no longer stored as Group$<hash>, which was a second subject beside the real one.
    • An uppercase 64-character hex value now counts as a custom id and is scoped.

Decisions

  • What can never change once an instance uses the method:

    • the normalization steps and the letter table;
    • the JSON-array encoding and the field order;
    • PBKDF2-SHA-256 at 600,000 iterations with a 256-bit output;
    • the cf_ prefix with lowercase hex.

    The typed values are never stored, so ids cannot be recomputed after a change.

  • The salt is served to every user who can read subjects, because the browser derives the id. It is never on the public setup route. With the salt, a short identifier can be brute-forced offline: about 15 hours on one CPU core for six digits.

  • Forms hold the method as a string validated by z.enum(<offered methods>). libui types a select's options as covering every value of the field's type, so a precisely typed select could not leave out a method that is not offered.

  • The default tab depends on the language. The first tab in translated-label order opens by default, as the brief asked, so Mail opens first in Spanish and French. Making Features the default instead is a one-line change.

  • A typed Custom ID is always scoped. Start Session and the data hub lookup scope it even when it is shaped like a name hash, because the user chose Custom ID. They do not go through toStoredSubjectId.

Risks

  • Old bookmarks. /admin/mail now lands on the not-found page.
  • Offered methods are enforced only by the forms. The server still accepts a session whose subject id has the shape of a method that is not offered.
  • Field order is locked only by the UI. Fields have no stored identity, so the server checks the number of fields but not their order. A page opened before the lock could reorder the fields, keep the count, and save. Closing this needs a stored field identity, which is an API change.
  • The bulk wizard's identifying-columns mode stays visible when Personal Info is not offered.
  • ẞ (U+1E9E) is not in the letter table and normalizes to itself. Once an instance uses the method, adding it would change ids.
  • The iteration count should be confirmed against current OWASP guidance before release. A later change affects only definitions saved afterwards.

Filed: #1934

Notes

  • This PR's adversarial review covers its own commits, from feat: let a group require a form to be completed when a session starts #1933's head (d5d749dc7). feat: let a group require a form to be completed when a session starts #1933 carries the review of the rest.

  • packages/subject-utils/AGENTS.md changes only its export list (the brief approves this edit):

    -Derives and formats clinical subject identifiers. Four exports: `generateSubjectHash`,
    -`encodeScopedSubjectId`, `isSubjectWithPersonalInfo`, and `removeSubjectIdScope` (re-exported from
    +Derives and formats clinical subject identifiers. Ten exports: `generateSubjectHash`,
    +`encodeScopedSubjectId`, `isSubjectWithPersonalInfo`, `inferSubjectIdentificationMethod`,
    +`toStoredSubjectId`, `CUSTOM_FIELDS_SUBJECT_ID_PREFIX`, `DERIVED_SUBJECT_ID_PATTERNS`,
    +`normalizeCustomFieldValue`, `deriveCustomFieldsSubjectId`, and `removeSubjectIdScope` (re-exported from

Adversarial review

Reviewed at commit b45ce85. pnpm check: green.
Follow-up 80a9834: regenerates apps/web/src/route-tree.ts, which the review excludes; no new code.

Findings

  1. apps/api/src/subjects/subject-identification.service.ts:60 — A cf_ subject created while no definition was stored, by a CSV upload say, made every save of the settings fail with a 500 until that subject was deleted — Answer that state without failing, keeping the offered methods editable.
  2. apps/web/src/routes/_app/group/manage.tsx:928 — Saving the group form for any change overwrote a group's stored default method with the first offered method whenever the stored one was not offered — Send the stored value unless the manager picks another.
  3. apps/web/src/components/SubjectIdentificationSettingsForm/SubjectIdentificationSettingsForm.tsx:105 — The Subject Identification tab showed its sections and method checkboxes in a fixed order, not by translated label as the brief requires — Sort both with sortByLabel.
  4. apps/api/test/suites/11-subject-identification.suite.ts:129 — The counts integration test checked only that the response parses, so it would pass with every id in the wrong bucket — Seed one subject of each shape and assert the counts.
  5. apps/web/src/components/ApplicationSettings/tests/ApplicationSettings.test.tsx:77 — The default-tab test ran in English, where written order and label order agree, so it would pass if the default were the first tab written — Run it in French.
  6. apps/web/src/utils/subject-identification-draft.ts:79 — The client put the offered methods into the fixed order the server imposes again on every save, a step that changed nothing — Send them as drafted.
  7. .agents/docs/architecture/auth-and-permissions.md:48 — The authorization inventory omitted the new read Subject route that serves SetupState, and the two new unscoped service paths — Add them.

Fixed

    1. The definition locks only while one is stored, so a cf_ id that no stored definition derived no longer blocks a save.
    1. The default method is sent only once the manager changes the select; left as seeded, the stored one stays.
    1. Both sections and the method checkboxes sort by translated label.
    1. The test starts a session for one subject of each shape and asserts that each count rose by exactly one, since the suites share a database.
    1. The test runs in French and asserts that the Mail panel opens.
    1. The offered methods are sent as drafted.
    1. Added the route to the table, and both service paths to the unscoped list.

Attestation

I, Claude Opus 5.5, attest that the review block above reports the adversarial review as it ran on the commit named, with every finding it returned.

🤖 Generated with Claude Code

joshunrau and others added 18 commits November 19, 2026 10:33
…e page loaded

The Start Session form computed the current date once, at module scope, and used it as the
in-person session date, the upper bound for a retrospective date and the reference for the
group's minimum age. Every session started later in the same page load was backdated to when
the page first opened. Each check now reads the clock when the form is validated, and the
in-person default is built at submission.

Closes DouglasNeuroInformatics#1675

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A retrospective session's date is now a date and time, so the time of collection is kept
(for MouseODC, the time of day is a proxy for hormonal variation). The date is required: a
blank one used to be read as the moment the form was submitted, and a time typed without a
date was lost.

Closes DouglasNeuroInformatics#1029

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… time of day

The export projected each row's timestamp from the day the record was inserted, so a
retrospective session, a bulk upload or a remote assignment exported with the wrong date, and
every row lost its time. It now takes the record's own date, and both it and the session date
keep their full ISO timestamp.

Closes DouglasNeuroInformatics#1717

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A group manager can choose one form instrument as the group's Session Instrument. Start
Session then shows that form after the session details, with a Back control, and the session
is created only once the form is submitted: POST /v1/sessions carries the answers and creates
the session and a record of them, dated as the session is, or neither. If the record fails to
save, the session is deleted, and so is the subject if this request created it. The server
refuses a session in such a group without the answers, and answers for a group that requires
none.

The setting follows a new edition of the form once that edition is saved, an update that would
take the form away from the group is refused, and a group is never created with one. The
gateway synchronizer, CSV upload and demo seeding create sessions in process and are not asked
for it.

The group's manage page now reads the group fresh rather than as it was at sign-in, since a
save sends back every setting it was seeded with and so could undo a change made since.

Refs DouglasNeuroInformatics#1546

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No test pinned an actual digest: the existing ones compare
generateSubjectHash with itself, so a change to its output would pass.
Each new case hard-codes the digest the current code produces, checked
against an independent SHA-256 of the string it hashes, and covers what
the normalization really does: accents and ligatures are spelled out,
plain hyphens are removed, while spaces (trailing ones included),
apostrophes and a non-breaking hyphen are kept, 李 and 里 both become
"Li ", and the date is the UTC calendar day.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The name hash transliterates its input, so the library's mapping table
is part of every stored Personal Info id. A caret range let a minor
release change that table, and with it what an existing subject's
details hash to. The lockfile already resolved 2.6.1; only the
specifier changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first-name and last-name help text on Start Session and the datahub
lookup said white space and special characters are removed. The hash
removes only plain hyphens: spaces, apostrophes and other punctuation
are part of it, so "Jean Paul" and "Jean-Paul" are two subjects. The
text now says that accents, letter case and plain hyphens are ignored,
and that spaces, even leading or trailing ones, apostrophes and other
punctuation count.

The hash itself is unchanged: its output is every Personal Info
subject's stored id.

Refs DouglasNeuroInformatics#1684

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hash-based identification page said white space and special
characters are removed, that non-Latin names are refused, and that a
private key held by the server is added to the input. None of that is
what generateSubjectHash does. The page now lists what is ignored
(letter case, plain hyphens, accents and ligatures, with other scripts
romanized), what still counts (spaces, apostrophes, other dashes and
punctuation), says no secret is added, and replaces the example with the
real input and digest for René Tremblay.

The French docs mirror only the introduction, so there is no French
page to update.

Refs DouglasNeuroInformatics#1684

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
How an id a person typed or pasted becomes the id it is stored under
lived in the bulk assignment wizard as toStoredSubjectId and
GENERATED_ID, so CSV upload never used it. It moves to subject-utils
beside the classifier it is built on, inferSubjectIdentificationMethod:
a bare 64-character lowercase hex digest is a name hash and everything
else a custom id. Only a custom id without a `$` is scoped to the group.

GENERATED_ID ignored case, the new rule does not. A name hash is always
lowercase, so an uppercase digest is a custom id and is now scoped. Bulk
matching is unchanged: an uppercase value never matched a stored
lowercase hash, and the prefix match compares unscoped ids, so a scoped
paste still finds the same subjects.

The wizard always has a group, so resolveSubjectIds and resolveEmailList
now require its name, and the branch that passed ids through without one
is gone.

Refs DouglasNeuroInformatics#1751

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CSV upload scoped every subject id to the current group, a Personal Info
name hash included. A hash pasted from the datahub was stored as
Group$<hash>, a second subject beside the real one, and the uploaded
records went to it. Upload now applies the same rule as bulk
assignments, toStoredSubjectId: a bare name hash is left as it is, and
only a custom id is scoped.

Refs DouglasNeuroInformatics#1751

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… into it

`/admin/settings` now has Features, Interface and Mail tabs. Features holds a
Remote Assignments section (default validity and the remote assignments
switch) and an Uploader section; Interface holds coloured table text, the
group switcher position and the languages; Mail holds the mail server settings
that had their own page. Features and Interface still save each change as it
is made, and Mail still saves through its own buttons. No section is called
"Settings" any more.

Tabs, sections and the settings within them are ordered by their translated
label, as the language list already was, and the first tab is the default. The
chosen tab is held in the address as `?tab=`, so a reload or a shared link
reopens it. Every tab stays mounted while hidden, so a mail form not yet saved
survives a look at another tab.

`/admin/mail` is removed with no redirect, and the sidebar's Instance group
keeps only Settings and Branding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the gateway

Without the gateway the server reports remote assignments as disabled whatever
is saved, so the Enable Remote Assignments switch did nothing there. The whole
Remote Assignments section is now hidden on such an instance, as the default
validity already was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Groundwork for the CUSTOM_FIELDS identification method (DouglasNeuroInformatics#1546), which
derives a subject's id in the browser from fields an admin defines.

- normalizeCustomFieldValue uppercases, applies NFKD, spells out the
  Latin letters NFKD leaves whole (Œ, Æ, Ø, Ł, Đ, Ð, Þ) and keeps only
  letters and digits. It does not use transliteration, which spells
  李 and 里 alike.
- deriveCustomFieldsSubjectId normalizes each value itself, refuses an
  empty list or a value that normalizes to nothing, and returns cf_ plus
  the lowercase hex of PBKDF2-SHA-256 over the UTF-8 JSON array of the
  normalized values, with the given iteration count and a 32-byte salt
  passed as 64 lowercase hex characters. It uses crypto.subtle, so it
  runs in the browser.
- isCustomFieldsSubjectId checks the cf_ shape, and toStoredSubjectId
  never scopes a cf_ id to a group.

Every expected id in the tests was computed separately with Node's
pbkdf2Sync, one of them at 600,000 iterations. The normalization steps,
the letter table, the encoding and the cf_ prefix cannot change once an
instance has used them: the typed values are never stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The package guide named four exports. It now names the five added for
the shared scoping rule and the custom-fields method as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y subjects

GET and PUT /v1/subjects/identification/settings read and write a definition of custom
identifying fields: a label for the method and an ordered list of one to ten fields, each with a
label, optional help text, a pattern and an optional error message. Patterns are checked by
compiling them as the browser will test them: as a whole-value match with the u flag.

The server gives a first definition a random 32-byte salt and 600,000 PBKDF2 iterations and keeps
both. Once any subject id carries the cf_ prefix, the definition is locked: it cannot be removed
and its number of fields cannot change, though labels, help text, messages and patterns can. The
salt is served to every user who can read subjects, since the browser derives the id, and never
on the public setup route. Subjects with a cf_ id are kept out of the custom-id suggestions.

Refs DouglasNeuroInformatics#1546

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once an administrator has defined custom identifying fields, Start Session and the data hub's
subject lookup offer them as a third identification method, CUSTOM_FIELDS, under the
administrator's label. The forms ask for every field, test each pattern against the normalized
value, derive the id in the browser (cf_ and the PBKDF2 digest) and send only that id with the
optional date of birth and sex; no name and no typed value leaves the browser. Once every field
is valid, Start Session looks the subject up by the derived id and fills in and locks what they
record, as it does for a custom id.

A group can make the method its default, a cf_ id is classified by its shape and never scoped to
a group, and the session card shortens it to the group's display length.

Refs DouglasNeuroInformatics#1546

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a Subject Identification tab to /admin/settings where an admin
defines the CUSTOM_FIELDS method: its label in each active language and
an ordered list of one to ten fields, each with a label, optional help
text, a pattern and an optional error message. A test box beside each
pattern shows what a typed value normalizes to and whether the pattern,
compiled as the forms compile it, accepts it.

The tab saves one draft with one button. The first definition asks for
confirmation, since its field list is fixed once a subject has been
identified with it. Once locked, the fields can no longer be added,
removed or reordered and the definition cannot be removed; their text
and patterns stay editable. A save the lock refuses reloads the saved
definition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Subject Identification tab now lists the methods staff are offered, saved with the custom
fields by the tab's one save. An instance that never chose offers Custom ID and Personal Info,
as before; at least one method must be offered, and custom fields only once they are defined,
so saving a definition does not offer it by itself.

A method that is not offered disappears from Start Session, the data hub lookup and the group
default-method select, and a form whose group prefers one falls back to the first offered.
Saving the group's settings keeps that preference, unless a manager picks another method.
Turning a method off changes no id, but the tab first says how many subjects it identifies,
counted by the shape of their ids through GET /v1/subjects/identification/counts.

The definition is locked only while one is stored: a cf_ id saved without a definition matches
none, so it no longer makes every save of these settings fail.

Closes DouglasNeuroInformatics#1546

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the Status: Waiting on Review Automated: unreviewed, pushed since the last review, or review re-requested label Nov 19, 2026
@github-actions

Copy link
Copy Markdown

This pull request was closed because its body does not meet .agents/skills/opening-pr/SKILL.md:

  • The review block names no commit matching the head 80a9834a4cac8f0896f0e58821e6a145c00acc93.

Fix the body with gh pr edit <number> --body-file <path>, then gh pr reopen <number>.

@github-actions github-actions Bot closed this Nov 19, 2026
@joshunrau joshunrau reopened this Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Waiting on Review Automated: unreviewed, pushed since the last review, or review re-requested

Projects

None yet

1 participant