[anth-volk]

Fixes #4045

Summary

Plan item 1.17: /v2/parameters/children for the parameter tree of the app's newer workspace (v3). It is stacked on #4042 (1.16), and v3 does not use it until it moves to the v2 routes.

• Tree table. The new parameter_tree_children table holds one row per node and parameter of a catalog version, under its parent's path ("" for the top level). Each row has:

  • its label;
  • its position among all its siblings;
  • its position among the siblings a reform can change (null if a reform cannot change it);
  • for nodes, its direct child counts in both views;
  • for parameters, the parameter's ID and its economy and household flags.

  Two unique indexes, (version, parent path, position) and (version, parent path, reform position), serve one level in one read.

• Revision 3869eb814a90 is chained on f5c1d9b4685b (1.15). It is autogenerated with no corrections and creates the table, its two listing indexes and an index on parameter_id. It writes no rows. Rows are deleted with their version or their parameter.

• Writing at publication.

  • Catalog publication writes, in its transaction and under its lock, the tree of each version in the catalog it publishes that has none. That covers the versions it publishes now, and the installed version when an earlier release stored it with catalog format 2 but without a tree.
  • An installed version stored before catalog format 2 gets no tree: its parameter rows store no economy or household flags, so a tree built with the catalog's flags would not match them. It returns the 503.
  • The rows are built from the installed packages' catalog records. Only the stored parameters' IDs are read, by name, so the rows reference them, after publication has checked that the stored rows match the catalog.
  • It is a row-only, idempotent insert of package-derived rows, which the Alembic skill's deployment-seeder passage allows.
  • Publication fails if a version in the catalog with catalog format 2 is still without a tree. The evidence adds parameter_trees_written (model, version, rows).

• Tree rules.

  • A [ starts a child as a . does, so a bracket schedule is one node whose children are its brackets, labelled "Bracket 1", "Bracket 2" and so on, in bracket order. A bracket's own children (rate, threshold) are unchanged.
  • Labels take the form v3 shows: the stored label with underscores as spaces and its first letter upper-cased. A path without a stored label gets the last part of its name in that form.
  • A reform can change a parameter under gov. whose economy or household flag is set and whose path names neither taxsim, gov.abolitions nor pycache, and every node above such a parameter. These are v3's rules (app/src/libs/buildParameterTree.ts).

• The route reads one level from the table, with the leaf summary the parameter routes return (including economy and household).

  • reform_changeable=true lists only what a reform can change, sorted by label, with child counts that count only those.
  • Responses get the same caching as the other metadata routes: an ETag, Cache-Control (a day for a named version, five minutes for the running one), Vary and a 304. A version's tree never changes. The 503 is not cached.

Route contract changes

/v2/parameters/children has no callers in this repository outside tests, and none in the app's main.

• Schedules: a bracket schedule is one node, instead of its brackets appearing beside its parent's groups under their raw names.
• Labels:
  • every child has its stored label in v3's display form;
  • children without a stored label get a generated label ("Property" instead of the raw property; an unlabelled schedule …rates.uk is "Uk");
  • brackets are "Bracket N".
• New query parameter: reform_changeable. The route's query goes through the shared strict models, so an unknown or repeated parameter, an unsupported country and a malformed policyengine_version return 422. An unsupported country and a malformed version returned 400 before 1.16.
• 503 for a version without a tree. A catalog version without a tree returns 503 with a message that names it. The route's OpenAPI 503 says so.

Measured

Locally, against the installed catalog with format 2 fields (the .py branch containing PolicyEngine/policyengine.py#701, 6.10.0), service level, 20 requests each:

Measure	Value
Tree rows	US 131,961; UK 2,425
One level (for example gov, gov.irs, a 42-bracket schedule, the reform view of gov)	3–7 ms median
EXPLAIN ANALYZE of a reform-view level	index scan on ix_parameter_tree_children_reform_listing, 0.07 ms
Writing the US tree	12.8 s on a loaded machine; republication with the tree write 32.1 s, without it 27.6 s
Peak traced memory writing the US tree	97.6 MiB (the production-scale publication test allows 256 MiB)

Staging times need a deploy; 1.16's release check records route times on each candidate.

Declared differences from v3's current tree

• A node with nothing a reform can change below it is left out of the reform view. v3 shows such nodes as empty groups, because v1's node entries carry their own flags.

• Brackets are ordered by number. v3 sorts them as text, so "Bracket 10" comes before "Bracket 2".

• Sorting by label approximates JavaScript's localeCompare with a case-folded comparison.

• Labels: an open decision for a person. The catalog stores .py's labels, and .py generates labels for parameters without a stored one (utils/parameter_labels.py, since December 2025):

  • bracket parameters become " (tier N amount)";
  • breakdown parameters become " ()".

  v1's /{country}/metadata instead uses the last part of the name with underscores as spaces, and "bracket N" for brackets. So 558 UK and 54,710 US reform-view labels differ from what v3 shows today, and the order differs at some levels. The plan labels children with their stored labels, and 1.14 deliberately publishes no second copy of each stored label, so this PR keeps .py's labels. The options are:

  1. accept .py's labels;
  2. shorten generated labels in the tree only;
  3. have .py also publish the raw stored label, which changes 1.14 and 1.15.

• Schedule labels. Schedules get their stored labels once the Public API depends on the .py release containing Publish the model metadata the Public API can only read through a private method policyengine.py#701, which publishes each bracket schedule as a node with its label. Until then a schedule has no node row and gets a label from its name ("Uk", "Al").

Trees for other versions stored earlier: left out

• What a backfill would do. On each publication, find every stored catalog version other than the installed one that has parameters or nodes but no tree rows. For each, read its stored nodes and parameters, build the rows, and insert them with COPY, insert-only and idempotent, inside publication's transaction and lock.
• Why it is left out. Those rows would be derived from data that earlier releases stored. The Alembic skill's "V2 application-data migrations" section reserves that for Alembic and a human migration decision. The deployment-seeder passage covers only package-derived rows, which the installed version's tree is.
• Consequence.
  • The installed catalog version gets its tree at the first deploy that includes this PR, including the version published at 1.15's .py bump, as long as it is stored with catalog format 2.
  • Every other stored version has no tree, and /v2/parameters/children returns 503 for it until a person approves that migration. So does an installed version stored before catalog format 2, for example if a .py release carrying the flags were published from master before this stack deploys.

Migration notes

• Route group and runtime path. The metadata route group; the route is served by FastAPI and reads Supabase (no change).
• Deploy order.
  • The seed job runs alembic upgrade head, then catalog publication, which writes the trees, before the new release takes traffic.
  • The previous release does not read the new table.
  • The revision's downgrade drops the table and keeps every catalog row.
• Privileges. The runtime role may read the table. The catalog writer may read and insert rows, and a test publishes as the catalog writer role with only its declared privileges.
• Environment variables. None are read or added, and no workflow changed.
• Suppressions and rules. No # noqa is added.
• Rule scope and statement change. rules/registry.toml adds tests/integration/test_v2_metadata_routes.py and tests/integration/test_v2_catalog_publication.py to the scope of api.v2.runtime-privileges-tested-as-runtime-role, because they now run a service as a managed role: the route as the runtime role, publication as the catalog writer. The rule's statement and the Alembic skill's privileges section now say the tests run the v2 user services and the parameter tree read as the runtime role and catalog publication as the catalog writer, and the skill's command runs the two modules. A review from PolicyEngine/core-developers is requested.

Waits for

• Serve v2 metadata in parts loaded on demand, cacheable per catalog version #4042 (1.16) and the PRs below it: Store the metadata v3 reads in the v2 catalog from PolicyEngine.py's public records (catalog format 2) #4023, Publish v2 catalog regions from checked-in reference files and the certified data release #4020, Rename the v2 table user_policies to user_policy_associations #4010 and Let saved reforms and households point at a new v2 policy or household #4005. Store the metadata v3 reads in the v2 catalog from PolicyEngine.py's public records (catalog format 2) #4023 (1.15) and the stack above it wait for the .py release containing Publish the model metadata the Public API can only read through a private method policyengine.py#701.
• Review.
• A person's decisions on the labels and on the migration for other stored versions (above).

Testing

CI on this branch is cancelled at the user's request. The evidence is local; each commit failed first for the expected reason and then passed, unless it only adds tests.

Commit	Failed first	Then passed
b8c8d751e Derive each catalog version's parameter tree	ModuleNotFoundError	11 tree tests
873218365 Store each catalog version's parameter tree in its own table	KeyError: 'parameter_tree_children', the revision chain, no child revision, NoSuchTableError	148 model, revision, privilege and migration tests; alembic check clean; privilege apply and verify
b35c854dc Write each stored catalog version's parameter tree during publication	no tree rows; evidence field missing	57 publication tests
8bd67742f Serve /v2/parameters/children from the stored parameter tree	8 route and 6 service tests; the Postgres schedule test against the old route	56 unit tests and 11 Postgres route tests, one as the runtime role
472a6286a Return parameter tree labels in the form v3 shows them	stored labels returned unchanged	103 tests
a38f6c018 Write the parameter tree only for the catalog versions a publication publishes	the older version was still written; the 503 message	98 tests
c5bb6301d Publish the parameter tree as the catalog writer role in a test	InsufficientPrivilege with the INSERT grant revoked	passes with the declared grant
20d9a419d Document the children route's statuses and page the reform view in a test	no route description in OpenAPI	57 tests
cc4489863 Test the parameter tree with records shaped like PolicyEngine.py's output	tests only	15 unit tests and 1 Postgres test
97904be68 Type a parameter tree row's child type with its two values	annotation was str	93 tests; alembic check clean
9983a3aed Drop the reform rule for parameters without flags	removes code that could not run; one service test exposed unflagged fixtures, now fixed	39 unit and 68 Postgres tests
bfe79da42 Declare exactly the errors the parameter tree route returns	the 503 description named no tree case	86 route tests
6c3bbcfe9 Give the installed catalog version its parameter tree when it has none	the installed version stored without a tree got none	54 publication and route tests; the rebuilt tree equals a fresh publication's, parameter IDs included
89156a2cb Test that parameter tree responses are cached like other metadata	tests only	88 route tests
4d14c0dc3 Write no parameter tree for a version stored before catalog format 2	a stored format 1 installed version got a tree	49 Postgres publication and route tests
03d081864 Test the parameter tree's link to stored parameters by name	tests only (the missing-name branch)	9 publication unit tests, 28 Postgres publication tests
b7300b3c0 State the parameter tree's real rule in its documents and messages	documents and messages only	150 unit tests
d6f4af31b Name the catalog writer in the runtime-role rule and the Alembic skill	documents only	rules validate, render and guard; 22 workflow and skill tests
37ab9482f Look up a stored catalog version's ID and format in one place	refactor, no behavior change	58 publication and route tests
2e41451ea Add the catalog format 2 condition to the tree writer's descriptions	documents only	ruff and format
edb0a8ec4 Test the tree's parameter IDs with stored IDs unlike the catalog's	tests only; both fail if the tree uses the catalog's IDs (a mismatch, and a foreign-key error)	2 tests
1cc6f620f Check that a version stored before catalog format 2 returns the tree 503	tests only; fails without the format 2 condition	1 Postgres route test
4519b8e96 Say which metadata read the runtime-role rule's tests cover	documents only	rules validate, render and guard; 22 workflow and skill tests

The commits that add a role-based test (8bd67742f, c5bb6301d) also add its module to the rule's scope, so test_every_managed_role_integration_module_runs_in_ci_and_is_in_its_rule passes at each commit.

On the final commit:

• make test set (without coverage), under the locked .py 6.10.0: 3,657 passed.
• Postgres tests on a schema recreated at head, grouped as the v2 integration workflow runs them after privilege apply and verify: publication and routes 50 passed; user identity and saved-policy view 20 passed; comparison runs and privilege reconciliation 9 passed; v2 migration lifecycle 10 passed.
• alembic check: no new operations.
• ruff, format, mypy, quality guards and rules checks: clean.
• Contract export: no change.
• No commit adds a .coverage file or names a private repository.

Review

Two review rounds were run on this branch before it was rebased onto 1.16.

• Round 1 found one bug. Stored labels were returned without v3's display rule while generated labels had it. This is fixed in 472a6286a. The same round's quality findings led to a38f6c018 (no trees from earlier releases' stored rows), c5bb6301d, and 20d9a419d.
• Round 2 reported the label difference against v1 described above. It is a product decision, so the labels are unchanged. Its quality findings are fixed in cc4489863, 97904be68, 9983a3aed and bfe79da42.

After the rebase, 6c3bbcfe9 gives the installed version its tree, and 89156a2cb adds the caching tests.

• Round 3, on those new commits, found one major bug. A version stored before catalog format 2 would get a tree carrying the catalog's flags, which its own parameter rows do not store. This is fixed in 4d14c0dc3: no tree for such a version, and the check that every version has a tree leaves it out. The same round's quality findings are fixed in 03d081864, b7300b3c0 and d6f4af31b.
• Final pass, after the rebase onto Serve v2 metadata in parts loaded on demand, cacheable per catalog version #4042's final commits (3885872ae), found no substantiated bugs. Its quality findings are fixed in:
  • 37ab9482f: one stored-version lookup shared with the stored-version comparison;
  • 2e41451ea: the format 2 condition in the remaining descriptions;
  • edb0a8ec4: tests that tell the stored parameter IDs from the catalog's;
  • 1cc6f620f: a test that the format 1 case returns the 503;
  • 4519b8e96: the rule statement narrowed to what CI checks.

🤖 Generated with Claude Code

[vahid-ahmadi]

Review at 4519b8e96e.

Verdict: the tree builder and the route look correct. One design gap to settle before the label decision lands. Stacked on #4042, so I reviewed only v2plan/1f-116-catalog-routes-on-demand...v2plan/1f-117-parameter-children. No must items.

1. should policyengine_api/data/v2/catalog/publication_parameter_tree.py:171: a version that already has a tree is skipped (_has_tree), and nothing records which tree rules built it. If the open label decision ends in option 2, or the reform rule, bracket ordering or label form changes later, every version published before that keeps its old tree and nothing fails. Serve a compact parameter search file for each catalog version at /v2/parameters/search-file #4049 avoids this for the search file with a file_format column and a hash check that fails publication when the stored file differs. A tree_format column (or the same compare-and-fail check on the installed version) would let a rule change either add new rows or stop the deploy, instead of serving two sets of rules side by side.
2. nit parameter_tree.py: _position_key orders the non-bracket siblings by raw path, while the reform view orders them by label. That matches the description, but it means /v2/parameters/children without reform_changeable lists children in a different order from v3's tree. A sentence under "Declared differences" would save a later bug report.
3. nit At about 132,000 US rows per catalog version, the table grows by that much with every published US release, and nothing prunes old versions. Fine for now; worth a line in the Stage 9 doc saying who cleans up.

Checked and fine: parent_path/bracket_index handle a.b[0].rate → a.b[0] → a.b; ancestors of reform-changeable parameters are marked changeable; the two unique listing indexes match the two ORDER BY columns; the 503 is not cached; the route is read-only, takes bound parameters, and caps parent_path at 512 characters.

What I ran: fresh clone, uv sync --frozen --extra dev, then pytest on the changed unit and contract test files: 238 passed. CI is skipped on stacked PRs here, so the integration, publication-scale and Alembic lifecycle tests have not run on this head. I did not run them (they need disposable databases and the full catalog), and I did not check the label counts in the description.

🤖 Generated with Claude Code