Skip to content

UK: take up #1095's actionable items on policyengine-uk 2.165.1 - #1422

Open
juaristi22 wants to merge 46 commits into
mainfrom
uk-1095-next
Open

juaristi22 wants to merge 46 commits into
mainfrom
uk-1095-next

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR takes the engine-free items of #1095 that María ruled on 29 October (Phase A), and the first re-bump past #1421 (Phase C): policyengine-uk 2.165.1. Main is merged in after #1420 and #1421, and the national and local target surfaces are regenerated on the pinned feed (Chronicle 53815b2). The spine and calibration items (Phase B) move to #1426, each measured in its own arm.

policyengine-uk 2.165.1

The lock moves from 2.164.2, which #1421 brought, to the newest release.

  • Other investment income in market, HBAI, net and capital income (2.164.3, Count other investment income in market and HBAI income policyengine-uk#2736; 2.164.4, Count other investment income in capital income, and test it in every income measure policyengine-uk#2746). The build writes it from National Savings and gilt interest on FRS rows and from the SPI's other investment income on SPI rows. The engine already taxed it, and now also counts it in market income, HBAI income, household net income and capital_income. The stages that read those incomes:
    • The LCFS, NEED and NTS stages band recipients on the engine's HBAI income with its deductions added back, so their gross income now counts it, as the surveys' gross incomes do.
    • was_wealth, and was_lisa through its predictors, read household net income against WAS's DVTotinc_bhcR8, whose investment income covers every account and investment WAS asks about, National Savings bonds and gilts included. was_wealth also reads capital_income as a predictor, which now matches the donor's gross income from all financial investments (DVGIINVR8).
    • etb_services reads the engine's HBAI net income, and etb_vat its HBAI gross income less the ETB's direct taxes, against the ETB's disinc. The ETB's original income includes investment income, and its records come from the Household Finances Survey from 2018. Whether that survey's investment income counts National Savings and gilt interest is still to check against its documentation.
  • Company car and car fuel benefits (2.165.0, Model the company car and car fuel benefits, with Class 1A NICs policyengine-uk#2750). The build writes none of the nine person inputs, so no one has a company car and both benefits and their Class 1A NICs are zero, as before. The inputs are signed at their default in UK_ENGINE_DEFAULT_INPUTS, for María's signature. The company car PR for the Autumn Budget dashboard supplies six of them.
  • Tax-Free Childcare over a smaller UC award (2.165.1, Keep Tax-Free Childcare over a smaller UC award, and take no deductions from awards only a reform creates policyengine-uk#2754). A family that would claim UC keeps Tax-Free Childcare instead when the top-up is worth more than the award. No debt deductions come out of an award only a reform creates. Every new variable is a formula.
  • No economic assumption moves. The vendored uprating pins and the bonus-sacrifice anchor change only their version stamps, and no gate digest moves.

Regeneration on feed 53815b2

The national and local references, membership and compile-parity receipts are regenerated on the merged tree, on main's feed pin (Chronicle 53815b2, #1420's re-pin). The vendored resources check against the full artifact unchanged. The local census, the local references, the local parity receipt and the coverage manifest regenerate unchanged.

  • National surface: 1,409 active references, up from 1,403, and no row left without a fact (four before). The new rows are the three OBR council tax nation rows (D14e) and SLC's three nation rows (R4). The DfE rows, the R7 windows and obr.council_tax move as described below. SLC's three nation rows differ from the incumbent's devolved metrics only by their uprating to 2025.
  • SPI uprating holds catch up with Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421: main's surfaces were generated before Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421 refreshed the HMRC uprating pins with the October EFO, so the regeneration moves 21 SPI holds. The eight self-employment amount band holds go from 1.1176 to 1.1115, as per-capita mixed income from 2023 to 2025 falls from 1.0384 to 1.0327. The 13 dividend holds by income band go from 1.088147 to 1.088151 (OBR per-capita GDP).
  • Local surface: 27,168 active references, every status count unchanged. The signed deferrals fall from 94 to 77, as the support-floor deferrals now follow each target's nations.
  • A window fix the regeneration found: R7's calendar-2025 window on dfi_ni.bus.passenger_receipts failed to compile, because its selector pins two services and the window counted series only from list-valued source keys. A dimension pinned to several values now names one series per value. The window of the sum is the sum of the windowed series, and a service short of a year, or with no facts at all, refuses the window. No other UK, US or NZ window reference pins several dimension values.

Targets

  • OBR council tax by nation (D14e): the three nation rows are pinned to the UK and held as measured diagnostics.
  • Declared geography wins (R8, R9): a selector's declared geography_id is returned before the inferred pin, and the authoring refuses a disagreeing pin.
  • SLC devolved repayments (R4): slc.repayments.devolved_total becomes one row per nation. Wales binds. Scotland and Northern Ireland are held out by María's ruling, as their plan thresholds are not modelled. The three rows carry SLC's 2024-25 outturn to 2025 by policyengine-uk's OBR average earnings index, since repayments are a share of earnings above the plan thresholds (María's ruling of 30 October). This replaces the matched-period hold of 29 October, which declared the frame's own year: the solve honours a declared period only at the calibration year, or at the frame's year for a stored column, so it would have refused the Welsh row.
  • Calendar-2025 window (R7): Scottish Child Payment, the TFC top-up, Northern Ireland bus receipts and the four SLC England repayment rows.
  • Ledger observation_period: a reference can select its facts at a declared period while the target keeps its own.
  • DfE (D5): the eight rows bind the January 2026 census at model year 2025. The three in_year_snapshot holds are retired.
  • Window-ratio complement: a numerator can be its denominator less the listed cells.
  • OBR council tax (D14f): binds the main-residence share: England's chargeable dwellings less second homes and empty homes, over chargeable dwellings (MHCLG, October 2025, about 0.968). Scotland and Wales take England's share, as a stated approximation.
  • Signed exclusions: D3 keeps Scotland's three-plus-children rows out with a corrected reason. D18a signs the MSOA income deferral. D18c keeps Northern Ireland's UC area cells out.

Spine and stage records

Gates

  • Zero-weight strata (Adjudicate the UK spine grain at the ingest/calibration seam #688): both SPI-synthetic strata are capped at 0. The June caps of 100,000 counted the retired ten-clone grain, and the national seam ships no zero-weight row.
  • Export surface (WS-E spine deferrals: known bugs and signed inconsistencies awaiting their increment #796): the all-zero incapacity_benefit_reported exclusion never matched the pinned reference, so it is dropped. The release boundary still drops the column.
  • macro_realism (Swap the microcosm-built UK spine into the national build and retire the incumbent feed #757): not declared, and the battery's policy text records why. The solve binds the income components directly, and the national accounts' household disposable income differs in scope. The release contract refuses a not_applicable entry in this spec vintage.
  • Take-up flags at their stages (María's ruling, "do both"): frs_take_up, frs_person_draws and frs_household_draws record their flags' shares at the stage's weights. Three assembled-phase gates hold them to their contract targets, at the terminal gate's bounds until a spine receipt measures them. The terminal uk_take_up_signal gate stays release-blocking.
  • The battery, spine-part, release-cut and dense digests in microcosm-data's contract move with the manifest.

Disclosure floor of 3 (A1)

María ruled on 29 October to move every UK disclosure floor from ten source households to three, CD171's absolute floor (recorded on #1095). One constant, receipt_counts.UK_DISCLOSURE_FLOOR, now sets it for the stage receipts, the stage-health gates and the report tools. A withheld figure reads "<3", and counts from three to nine read "3-9", rounded record counts included (Vahid's review). The SPI tape's two SN 9422 conditions are unchanged. A guard test fails on any "<10" label left in the UK runtime, the UK spec, the gate notes or the tools. The statistical floors stay: the zero-support floor, the private-rent tool's ten-row group rule, and the SPI coherence tool's floor for medians and quantiles. The signed rules in local_small_cell_rule.json defer published cells, not survey cells, so they are unchanged.

Text

Stale tracker text in notes, docstrings and the vendored bus resources' purposes. The Incapacity Benefit leaf note now names the FRS 2024-25. The charter-H2 fixture and the release input coverage manifest follow the stage contract digests.

Not in this PR

Tests

  • Engine tier on policyengine-uk 2.165.1: every UK engine test passes, run in full locally because CI stops at the first failure. Three needed updates: Tax-Free Childcare's top-up now lives in a shared helper, which still reads the spend the build routes; the company car's list price and capital contributions are person-level stocks that are no one's capital; and the graph build's stage records list each stage's rewrites beside their outputs (Carry manifest rewrites into the shared Stage/StageRecord producer graph #721). CI passed in full at 02df9df2b.
  • Full engine-free suite at b46be0044 (673 files, in 30-file chunks): 22,969 tests, 233 skipped, counted by --junitxml. Sixteen fail:
    • ten fixed since: the SLC nation rows' declared period (four test_uk_measurement_period tests) and the pins and fixture rows that follow the regenerated surface (six). At 652835c1f, the 60 files that read the regenerated surfaces pass: 3,920 tests, 62 skipped;
    • six only on this machine: four Orrery tests (a missing Node package), test_engine_is_reported_unavailable_without_the_uk_extra (the uk extra is installed here) and test_spawned_workers_score_identically_to_sequential_cli (needs Python 3.14.7; this machine has 3.14.6).
  • The "3-9" band (254be6d41): the tests of every rounded_count caller pass: 39 engine-free files (1,384 tests) and 8 engine-tier files (72 tests).
  • After the Promotion for the constant-id national line: pointer and certified-loader extension #823 revert (1f46653b6): the 18 files that resolve or load a UK dataset pass, bar the live Hub pointer test, which needs private credentials (CI skips it). Main's version of that test expects the old release naming; UK: make the promoted national line the default load (#823) #1432 fixes it.
  • Feed-reading tests on 53815b2 (the 15 feed-gated files, one process each, which CI skips without the feed): 235 tests, two skipped because the Scotland support and the authenticated national artifact are not on this machine. Two tests in test_uk_target_references.py still expected values from before D5 and R7; 02df9df2b moves them to DfE's January 2026 census and the calendar-2025 windows, each value checked against the feed's facts and the committed membership. The two drift checks that regenerate the vendored resources and the target surfaces from the feed were left out: the regeneration wrote those files, and the vendoring check passed in it.
  • After Vahid's second review: f83d7a3c2 links the ruling's record from receipt_counts and the changelog, and its tests pass. ddd82aa2f adds a test that a pinned service with no facts refuses the window; the 460 tests of test_ledger_targets.py pass.
  • Checks: tools/ci_test_plan.py verify, tools/spec_engine_coverage.py --check, the vendoring check against the full artifact, and ruff on the edited files.

🤖 Generated with Claude Code

juaristi22 and others added 21 commits October 29, 2026 12:59
OBR's table 4.1 is UK-wide, so Chronicle stamps its England, Scotland and
Wales council tax rows K02000001, and the substring rule's nation pins
compiled no fact. The rows now pin to the UK, as obr.council_tax and
obr.domestic_rates do; each binding still scopes the model column to its
nation. obr.council_tax binds their sum, so measure exclusions hold the three
as diagnostics, measured on every evaluation as evidence for CT-C (María's
ruling of 2026-10-29, microcosm#1095 D14e). The references and membership
regenerate with the PR's shared regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R8 and R9: a wrong geography pin reads as "no fact", and the
generator's inference (one-by-one pins, prefix pins, then the substring rule)
silently overrode any geography a target's selector declared, because the
authoring merged the pin over the selector. María ruled on 2026-10-29 that a
declared geography wins. The UK generator now returns a selector's declared
geography_id first and refuses a one-by-one pin that disagrees with it, and
the shared authoring refuses a pin that differs from a declared selector key
instead of overwriting it. No committed selector declares a geography yet, so
no reference changes; the devolved SLC rows are the first to use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R4. slc.repayments.devolved_total took one fact per devolved
nation, but the "slc." rule pinned it to Great Britain, where no fact sits, so
it never compiled. It becomes slc.repayments.wales_total, .scotland_total and
.northern_ireland_total, each declaring its nation's geography in its selector
(the R8 rule), measuring the frame's own year (measurement_period 2024)
against SLC's 2024-25 outturn, and filtering the model to its nation. The
incumbent registry's three per-nation metrics now map one to one.

María ruled on 2026-10-29: bind Wales, and hold Scotland and Northern Ireland
until their borrowers' plan thresholds are modelled (plan_4_imputation is
False; policyengine-uk#2636 closed as not planned). Measure exclusions hold
the two. The matched-period hold for the three rows waits for her signature,
so the membership assertion in test_uk_target_references fails until the
shared regeneration, which needs that hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follows the split of slc.repayments.devolved_total (microcosm#1095 R4): the
contract has 453 targets, the registry scope 393, and the incumbent's three
devolved metrics map one to one, so 185 mapped targets. geography_id joins
the national selector vocabulary, as the R8 rule lets a selector declare its
fact's geography.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R7: Scottish Child Payment spending, HMRC's Tax-Free Childcare
top-up, Northern Ireland's Translink bus passenger receipts and SLC's four
England repayment rows took their 2025-26 fact alone, outside María's rule of
2026-09-22 that fiscal-year facts take only the months to the end of the
calibration calendar year. She extended the rule to them on 2026-10-29, so
each declares calendar_year_window: three twelfths of 2024-25 and nine of
2025-26. The pinned feed has both years for all seven (Scottish Child Payment
has 2026-27 too). No hold or measurement period names them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D5: a target is measured at the model's period, but some
publications observe a period with another label. DfE's January census for
model year Y is January Y + 1, the spring term of the year's entitlements, so
the rows could only bind the census a year out: every selection path refused
a fact after the target period.

LedgerTargetReference gains observation_period. Fact eligibility, the
latest-not-after choice, the exact-period match and the resolved-fact guard
read it in place of the target period; the TargetSpec keeps the target period,
and its metadata records ledger_observation_period. Operations that compose or
pin periods themselves (the calendar-year windows and average, latest_plateau,
the two ratio translations) and the monthly windows refuse it, as does an
unparseable period. The authoring passes a target's observation_period into
its reference rows and checks eligibility at it. Nothing declares one yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D5. The eight dfe.funded_childcare rows bound the January 2025
census, but model year 2025's entitlement year runs from April 2025 to March
2026: its spring term starts on 1 January 2026, and the spring-term measures
count children by their age on 31 December 2025, the basis DfE's January 2026
census uses. The January 2025 census was a year out, held as an in-year
snapshot. María ruled on 2026-10-29 to bind January 2026.

Each row declares observation_period 2026-01, so its fact is selected there
(the universal-only difference's two operands too) while the target stays at
2025. A fact after the target period carries no uprating hold, so the three
in_year_snapshot holds retire. The reference and membership assertions wait
for the shared regeneration; the childcare take-up refit follows on the arm's
spine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D14f needs OBR's council tax receipts restated for the main
residences households hold: England's chargeable dwellings less its second
homes and empty dwellings, over the chargeable dwellings. No published cell
is that remainder, and a window-ratio operand can only sum cells.

A calendar_year_window_scaled_by_ratio numerator may now declare complement:
true. Its cells are taken off the denominator before the quotient, which
keeps the (0, 1] subset-share check. complement is operand metadata, never a
selector field; only a numerator may declare it, and only as true. The spec
records ledger_value_numerator_rule, and the formula reads
"(denominator - numerator) / denominator".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D14f. obr.council_tax bound OBR's Great Britain receipts,
which include council tax on second homes and empty dwellings. Households do
pay it on property they own, but the frame's council_tax is each household's
main-residence bill only and the engine has no input for council tax on
other property, so the solve met the whole line by pushing main-residence
bills up. María ruled on 2026-10-29 to bind the main-residence share.

The row now scales its calendar-2025 window by England's chargeable dwellings
less its second homes and empty dwellings, over its chargeable dwellings
(MHCLG Council Taxbase, October 2025), through the window ratio's new
complement numerator. A private compile of the row against the pinned feed
resolves the window and the three taxbase cells once each. The count-based
share and England standing for Scotland and Wales are stated in the notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095, María's rulings of 2026-10-29:
- D3: ons.population.scotland_households_3plus_children stays signed out,
  but its reason was wrong: a fact exists, Chronicle's Scotland cut, and it
  counts couple households only, so binding it would fix part of the group
  and leave lone parents free. The register entry takes the corrected reason
  (couples_only_composition_fact), her approval of 2026-10-29 and an
  adjudication line; it tracks #1095, as #736 closed.
- D18a: ONS's small-area income rows (BHC and AHC means, housing costs) were
  deferred "pending the signed mean-aggregation design". ONS models them
  from the FRS, so binding them would partly fit the FRS to itself: they are
  kept out as circular (frs_model_based_circularity).
- D18c: Northern Ireland's UC area cells stay out; the reason now records
  that DfC publishes claimants, counting a joint claim twice (uk-data#590).

The membership and local surfaces carry the new texts at the shared
regeneration; test_uk_target_references pins the D3 text to the new reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095's "stale register and gate text" items that need no signature:
- weight_year's docstring quoted 0.72% growth from 2024 to 2025, the figure
  before policyengine-uk 2.155.0's population; it now points at the pinned
  dataset_projection block, which every lock re-pins.
- The certifier and the calibration run said local-candidate gates stay out
  of national certification "until microcosm#146"; #146 closed when the local
  variant shipped, so they now say the local line certifies them.
- was_wealth's bedrooms note pointed at the closed #145; the question is
  #1095's "num_bedrooms as a WAS predictor" (uk-data#463).
- dwp.uc.households_lcwra_element's note quoted a September baseline and an
  open #882 question; it now records pe-uk#1989's proxy and María's NS6
  ruling of 2026-10-29 to keep it.
- obr.universal_credit's note said both "UK incl. Northern Ireland" and
  "Scoped to Great Britain" (CF1); it now says Great Britain.
- obr.sdlt's note called pe-uk#2322 unreleased; it has been in the lock since
  #1400, and the note now states how its projection compares with the row's.

Signed register text (the JSA and ORR reasons, tracking fields of closed
issues) waits for the renewals PR, so each entry is re-signed once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#721 (tracked in #1095): a stage can declare rewrites, columns an
earlier stage produced that it replaces, but StageRecord kept only the
declared outputs, so provenance credited a rewritten column to its first
producer, and the UK spine's final nonzero-share list kept a hard-coded
FRS_EDUCATION_GRANT_REWRITES carve-out (redundant, as frs_spine already
produces education_grants).

StageRecord gains rewrote, the stage's declared rewrites, and its
nonzero_share covers them as they stand after the stage. A rewrite absent
after the stage is refused, like a missing output. The spine manifest's
stage_records carry rewrote, and the final list reads every record's
outputs and rewrites, so the carve-out and its import go. #721 can close
once this lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 (from #796): the support-floor loop deferred E06000053 and
E09000001 for every local-authority target outside the bound families,
including the nine Welsh and eight Scottish families whose area_scope holds
no English authority, so their membership listed deferrals for cells they
never have. The loop now reads each target's local_authority gss_prefixes,
keeps only the areas in scope, and skips a target with none. Thirty-two
targets take the deferral (59 area entries, E06000053 in 27 and E09000001 in
32), down from 49 (93; 44 and 49). The local references and membership
regenerate with the shared regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 items from #688, #796 and #757.

- uk_zero_weight_strata: the two strata kept June's caps of 100,000 each,
  10,000 SPI-synthetic households times ten output-area clones, a grain the
  national seam no longer builds. Its calibrated control of 2026-10-28
  (4374e0bf) holds 78,093 households and no zero-weight row in any of the
  four SPI-synthetic and capital-gains-clone strata, because the SPI stage
  gives the stacked donors their prior mass before calibration. Both strata
  are re-declared at a cap of 0 under current names; they keep their
  selectors so that a zero-weight row there is named by stratum.
- uk_export_surface: the reviewed exclusion for
  person.incapacity_benefit_reported never matched. The pinned reference
  (efrs_parity_reference.json) lists the all-zero column among its zero-share
  inputs, not on its export surface, so the gate reported the exclusion as
  unused. The release boundary still drops the column, and the shared check
  refuses it as an unlisted extra, so the guard holds without it.
- macro_realism: María ruled on 2026-10-29 to drop it and record why. The
  plan declared it not_applicable, but the release contract's verifier
  refuses an excused entry in this spec vintage (test_gate_battery_contract_
  pins holds every UK entry to not_applicable None). So the battery's policy
  text records the reason: the solve binds the household income components
  directly, and the national accounts' household disposable income differs
  in scope (imputed rent and, in the headline series, non-profit
  institutions serving households).

The battery, part and dense digests in microcosm-data's contract move with
the manifest, as in a094f5d. The take-up stage-time checks (her ruling of
2026-10-29) follow in their own commit. The Carer's Allowance stage-health
gate waits on her: the take-up contract records her 2026-10-20 ruling of no
such gate until a caseload target binds the rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d2a1f9c gave StageRecord its rewrote field and read the spine's final
nonzero-share list from every record's outputs and rewrites, but only the
legacy plan path filled it. The graph build, the production path, projects
its records from node artifacts and left rewrote empty, so its records
dropped the rewrite provenance microcosm#721 asks for, and
test_driver_writes_spine_h5_sidecars_and_logbook lost education_grants from
the final list (the real roster still lists it through frs_spine's outputs).

A declared rewrite is an owned cell of its node, as an output is, so the
graph projection now reads its share from the node's artifact and records
the stage's declared rewrites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 on microcosm#1095 ("do both"): keep the
terminal uk_take_up_signal gate release-blocking on the calibrated shares,
and check the same flags at stage time.

- The twelve flags the terminal gate measures are drawn by three stages
  before the assembled boundary: frs_take_up (the four childcare flags, the
  UC childcare element and the under-2 usage), frs_person_draws (Marriage
  Allowance, Scottish Child Payment) and frs_household_draws (TV ownership,
  licence evasion, the first-home and purchase flags).
  take_up_signal_outputs selects each stage's flags, and a test holds the
  three sets to a partition of the terminal gate's.
- The terminal gate's per-flag measurement moves into
  _measure_take_up_signal, unchanged, so that uk_take_up_signal_receipt
  measures the same thing at the stage's weights. The receipt is
  aggregate-only: support is a band of distinct source households, and a
  share or target resting on fewer than ten is withheld.
- Each stage records the receipt in its checkpoint evidence, and the graph
  hashes frs_take_up and receipt_counts with the two draw stages.
- stage_health gains a take_up_signal check: the receipt must carry
  exactly the declared outputs; each must vary, sit within the declared
  bound of its target, and keep the universal hours nested; a withheld
  share fails, as it cannot be measured.
- gates.json declares uk_stage_frs_take_up_signal,
  uk_stage_frs_person_draws_take_up_signal and
  uk_stage_frs_household_draws_take_up_signal (assembled, release-blocking,
  evidence_absent_blocks, population_fact_check) at the terminal gate's
  bounds, 0.05 and 0.10 for the under-2 usage, until a spine build's
  receipt measures the stage-time deviations. The spine scope, the data
  contract's entry, evidence and spine-part mirrors and the battery, spine
  and release-cut digests follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#823 (tracked in #1095). The registry kept the UK 2024-25 national
line off the default variant until its first certified cut existed on the
Hub, so that resolve("uk") never chased an absent pointer; its comment named
the flip a one-line follow-up after promotion (review of #966, Max's #823
ruling). Checked on 2026-10-29: latest-national.json on
policyengine/populace-uk-private names
microcosm-uk-2024-25-national-20261025T091414Z-a6299504 (updated
2026-10-26), with its release certification, built on policyengine-uk
2.151.0, the version #1095 names for the flip.

The entry's variant becomes DEFAULT_VARIANT, so resolve("uk") and
load("uk", 2025) follow latest-national.json, and load("uk", 2023) stays on
latest.json. This changes the default UK load for every consumer of
microcosm.data; no other local repository names the retired national
variant. The registry, loader and assembler tests and the release runbook
follow; the live pointer test now branches on the line pointer, not the
variant, and resolves the promoted cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two items from microcosm#1095's stale-text list, both without a signature:

- The three vendored bus resources said that the LCFS and ETB bus-fare and
  bus-support rakes consumed them. microcosm#930 and María's ruling of
  2026-09-23 replaced those rakes with the bus fare and bus support pricing.
  The purposes in the selection register and in the vendored files now name
  the pricing, and dft_bus_value_anchors.json lists bus_support_pricing,
  which reads its net support, among its consumers.
- The Incapacity Benefit SPI leaf (FRS benefit code 17) was "a structural
  zero in the audited 2023-24 FRS". A private check of the FRS 2024-25
  benefits table found no code-17 row, so the note now names both vintages.
  The text is in sources.yaml, its source_stages.json projection and the
  contract's expected string.

Both move stage contract digests, of frs_hmrc_spine_leaves, lcfs_consumption
and etb_services. So tools/graph_uk_spine_fixture.py regenerated the
charter-H2 fixture (only those three digests and the leaf note change), and
the release input coverage manifest takes the new source manifest digest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8e3394b gave LedgerTargetReference an observation_period field. The
monetary binding serialises the reference with asdict, so every activated
and source reference receipt now carries the key (None unless declared), and
microcosm-calibrate's integrity check refused it as an unknown key: the full
engine-free run failed test_monetary_targets' two binding tests. The
verifier's reference keys gain observation_period, as #834 added the operand
keys, and the calibrate fixture's hand-built receipt carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fff37ea held the three OBR council tax nation rows (ruling D14e) and
ae1edf7 held SLC's Scottish and Northern Irish repayment rows (ruling R4),
which takes the packaged calibration measure exclusions from 40 to 45 and
the census's slc. marker from 3 to 5. test_uk_measure_simulation still
said 40 and 3; the full engine-free run caught it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R4. ae1edf7 split SLC's devolved repayments into one row
per nation; each resolves SLC's 2024-25 outturn, the latest each nation
publishes, at the 2025 target period and measures the frame's own year
(measurement_period 2024). The compile needs a signed matched_period hold
for that, and test_every_committed_hold_is_signed refuses a proposed one,
so the hold waited for María's signature. She signed it on 2026-10-29
(approved_by juaristi22, expires 2027-01-29). The shared regeneration can
now compile the three rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 on microcosm#1095 (A1): "move everything to 3".
CD171's absolute floor is three; the build used ten, CD171's working
minimum against secondary disclosure across outputs. Her ~/.claude/CLAUDE.md
now sets three for what people read too.

- receipt_counts gains UK_DISCLOSURE_FLOOR (3), UK_WITHHELD ("<3") and
  UK_BELOW_TEN_BAND ("3-9"). small_cell_share withholds one or two records;
  support_band and extreme_count print "<3" and "3-9" below ten;
  rounded_count prints three to nine exactly.
- Every stage receipt that withheld on fewer than ten sources now withholds
  on fewer than three, through the shared names: annual income, the
  apprentice draw, Carer's Allowance, the Move to UC and UC take-up draws,
  pension coherence, property purchases, the SPI band mean, the NTS fare
  mix, the gambling fits, the high-value home gates and the single years of
  age (diagnostics.UK_DIAGNOSTIC_MIN_SOURCE_HOUSEHOLDS). stage_health reads
  the new label and bands, and its detail key becomes
  regions_rewritten_under_floor.
- Tools: the private-rent receipt (its output key becomes
  cells_under_floor_source_households), the childcare realised take-up
  report and, through the diagnostics constant, the council tax liability
  comparison.
- Kept on purpose: the zero-support floor (1); local_small_cell_rule.json,
  whose three signed rules defer published cells too small to fit, not
  survey cells, so they need no re-signing; the private-rent tool's
  ten-row group rule (no summary describes one district); and the SPI
  coherence tool's cell floor for correlations, medians and quantiles,
  since a median of a few records is one record's own value.
- Six gate notes and four stage notes state the new floor; the battery and
  spine-part digests, the coverage manifest's source digest and the
  charter-H2 fixture's data bundle follow (its graph is unchanged).
- About 86 test expectations across 21 files: a withholding test keeps its
  purpose on one or two sources and expects "<3"; an incidental "<10" now
  reads the exact figure, "<3" or "3-9".

1,270 tests across the touched files, the gate and contract pins and the
coverage manifest pass; the charter-H2 parity test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 this is an early look at 5a944e449, not a full review, since the body says it isn't ready. It covers must and should items only, and nothing on your own pending list. I'll do a full pass after main merges in and the references regenerate.

  1. must: an engine-uk failure the body doesn't list. tests/engine/uk/test_uk_graph.py:84 still asserts set(nonzero_share) == set(stage.outputs). But _graph_stage_records now adds stage.rewrites to shares (Carry manifest rewrites into the shared Stage/StageRecord producer graph #721). So age_tail fails in engine-uk on both Pythons (run 40917095981, at this head). Regenerating won't fix it. The assertion needs set(stage.outputs) | set(stage.rewrites). The body's engine-tier line covers only the charter-H2 parity test, so this one was missed.
  2. should: rounded_count now prints exact counts from three to nine (receipt_counts.py:103). The module docstring still says "no exact count that ... narrows a small group by difference". extreme_count and support_band print "3-9" there instead. Printing exact counts just above the floor is where differencing bites. That matters more now that the floor is three rather than ten, CD171's working minimum against secondary disclosure across outputs, as the docstring says. I'd keep "3-9" in rounded_count too. I'm not reopening the ruling. Please link the UK: every open data item: uk-data parity, engine locks, the local surface, consumption and high-value homes #1095 comment that records it from the module and the changelog, and say there that the SN 9422 conditions (HMRC sees SPI-based material ten working days ahead) are unchanged.
  3. should: UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 adds new "<10" sites. The SPI pay-floor receipt in UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 hardcodes ten in several places. Whichever of the two lands second should move them onto UK_DISCLOSURE_FLOOR. Otherwise one receipt stays on the old floor without anyone noticing. I've raised it on UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 too.
  4. should: ship the Promotion for the constant-id national line: pointer and certified-loader extension #823 default-load flip on its own. Pointing load("uk") at latest-national.json and retiring variant="national" affects every consumer. It doesn't depend on Budget 2026 slower data fixes, PR C of #1418: electricity at the ONS level, pension income by area #1420, Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421 or the regeneration. As its own small PR it could land now, and it wouldn't be held behind this one. The old 823-national-variant-until-promoted.changed.md entry, which describes the opposite state, should go in the same change.

The D14f main-residence share (0.968) is covered on #1423: #1423 (comment).

What I ran. I read the diff at 5a944e449 against its base fddbc7073 and the failing CI logs. I didn't run anything locally. I didn't review the target, gate or ledger changes in detail, and I'll leave those for the full pass.

🤖 Generated with Claude Code

juaristi22 and others added 3 commits October 30, 2026 15:44
Main now carries #1420 (electricity at the ONS level, pension income by
area, Rural Fuel Duty Relief areas, the balanced Carer's Allowance draw)
and #1421 (policyengine-uk 2.164.2, ISA subscriptions, the Lifetime ISA
anchor, the additive ISA contract), with the feed at Chronicle 53815b2.

Hand-merged:
- uprating_holds.json: main's holds, re-pointed to microcosm#1095, less
  the three DfE in_year_snapshot holds this branch retires (D5), plus the
  signed SLC nation hold (R4) and main's ONS electricity hold.
- gates.json: the export surface keeps main's tax_free_savings_income
  exclusion (microcosm#1360) and drops the all-zero incapacity benefit
  layer as this branch does; its notes name both.
- sources.yaml: main's Carer's Allowance stage note, with its receipt
  floor restated at three source households.
- weight_year.py keeps the docstring that names no lock's growth figure.
- calibration_run.py, test_country_spec.py, contract.py and
  test_contract.py list both sides' new gates in gates.json's order.
- The tests take the merged counts: 458 targets (registry 396, profile
  62; mapped 185, unmapped 211), 44 measure exclusions, and 34 support
  floor rows over 61 area entries ({E06000053: 27, E09000001: 34}).
- The Carer's Allowance receipt test takes main's balanced placement at
  the floor of three; two stale comments restate that floor.

Regenerated on the merged tree with policyengine-uk 2.164.2:
source_stages.json, the release input coverage manifest (143 required,
2 reviewed exclusions), the gate battery, spine-part and release-cut
digests, and the charter-H2 fixture. The national and local references,
membership and compile-parity receipts regenerate next, on feed 53815b2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK lock moves from 2.164.2, which #1421 brought, to 2.165.1, the
newest release (microcosm#1095, Phase C's first re-bump):

- 2.164.3 (policyengine-uk#2736) counts other investment income in
  market income and in HBAI household net income, before and after the
  reset to zero. The LCFS, NEED and NTS stages band recipients on that
  HBAI income with its deductions added back, so their gross income now
  counts it, as the surveys' gross incomes do.
- 2.164.4 (policyengine-uk#2746) counts it in capital_income too. Its
  changelog says nothing reads capital_income; the WAS stage does, as a
  predictor, and the recipients' figure now matches the donor's
  DVGIINVR8, WAS's gross income from all financial investments.
- 2.165.0 (policyengine-uk#2750) adds the company car and car fuel
  benefits in kind and the Class 1A NICs on them, read from nine person
  inputs. The build writes none, so no one has a company car and both
  benefits are zero, as before. The inputs are signed at their default
  in UK_ENGINE_DEFAULT_INPUTS for María's signature; the company car PR
  for the Autumn Budget dashboard's #316 supplies six of them.
- 2.165.1 (policyengine-uk#2754) lets a family that would claim UC keep
  Tax-Free Childcare instead when the top-up is worth more than the
  award (forgoes_universal_credit_for_tax_free_childcare), and takes no
  debt deductions from an award only a reform creates. Every new
  variable is a formula.

Two lockstep tests follow the engine: Tax-Free Childcare's top-up is
one helper from 2.165.1, which reads the spend the build routes; and the
person-level GBP stocks gain the company car's list price and the
capital contributions towards it, which are no one's capital.

No economic assumption moves: the vendored uprating pins and the
bonus-sacrifice anchor change only their version stamps, and no gate
digest moves. The three pyproject floors, uv.lock (policyengine-uk
alone), the approved lock digest, the engine-core pin test, the
concept coverage report, the version mentions in code, notes and
tests, source_stages.json, the coverage manifest and the charter-H2
fixture follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
610a1bc records a stage's declared rewrites beside its outputs in the
graph build's stage records, as d2a1f9c does in the plan build
(microcosm#721). The engine-lane test of the record projection still
expected the outputs alone, so it failed on age_tail, which rewrites
age. It now expects both, and the record's rewrote field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the independent review of 0a69ef4b5. Every finding is addressed at c0dd6eae2, except finding 4, which belongs to whichever of #1422 and #1430 lands second.

Fix before merge

  1. Regional home values and "3-9": the gate reads any banded count at its bounds. A region of three to nine rows counts as three, the stage's own banded row count is read at its top, and only a region printed "<3" is flagged, as before. The fixture's counts are now ones the receipt can print, and a new test covers "3-9" (7a95f37a7).
  2. The pay-ratio clamp: it stays at ten, now PAY_CONCEPT_RATIO_MINIMUM_SUPPORT, as a statistical floor: on fewer households a ratio past its bounds is noise, and stopping the build on it is what Vahid's second review of Build the national + local-area pair on policyengine-uk 2.151.0, and drop UK model-figure withholding #1396 ruled out. The gate now reads a ratio whose support band is under ten as unmeasured, so a held ratio is flagged rather than judged (7a95f37a7).

Periods and windows

  1. observation_period across restamps: restamp_reference_period moves a declared observation period by the same number of years as the target period. The calibration compile, the local compile and the childcare fit tool restamp through it. So the DfE rows read the January census after their model year at any period, and --target-period and --vintage-override select the matching census. The test now restamps the row instead of nulling the field (42913ab5e).
  2. UK: company cars from the FRS job table (dashboard #316) #1430's feed: right; this falls to the second PR to land. The UK: company cars from the FRS job table (dashboard #316) #1430 session has the three nation rows to add to source_table_by_opening_year when it merges main.
  3. The engine-defaults guard: it also reads every stage's declared outputs and rewrites from source_stages.json. UK: company cars from the FRS job table (dashboard #316) #1430's company_cars stage will fail it until its six written inputs leave UK_ENGINE_DEFAULT_INPUTS (7a95f37a7).
  4. Series per pinned value: a window over several pinned members now needs exactly one complete series per member. Your case of two ulsterbus series and no metro_glider refuses, with a test (42913ab5e).
  5. The nation diagnostics' basis: the three nation rows now take obr.council_tax's main-residence share, so they sum to the bound row and each reads its nation on the same basis. The notes, exclusion reasons and changelog say so (c0dd6eae2).

Gates and tests

  1. Withheld cells in the take-up stage gates: they are flagged unchecked, never failed, as the sibling gates do under María's ruling of 26 October. The terminal gate still measures every flag at the calibrated weights, so nothing goes unmeasured (7a95f37a7).
  2. childcare_copy_take_up: it now records the five flags it re-draws, and a fourth assembled gate holds them to their targets. The changelog says so (7a95f37a7).
  3. Test gaps: frs_take_up's receipt is read in its stage test. A new test runs each take-up stage gate through the battery's stage_health binding, as the spine battery does. The copy stage's gate is tested, and so is "3-9" in the home values gate (7a95f37a7).
  4. The floor guard: it now scans tools/** and microcosm-data too. A second guard fails on any comparison, constant, default or prose that keeps a floor of ten outside the statistical floors it names (7a95f37a7).

Texts

  1. Floors of ten: moved to three are the high-value-homes support gate and its default, and the three tools' SDC defaults. Kept as statistical floors, each now saying so, are the pay ratio's ten, the SPI transport's ten rows, the regional renter minimum, the SPI coherence tool's cells and the private-rent tool's groups. The stale prose is restated.
  2. Replaced designs: restated are the R4 changelog, the parity tool's DfE and council tax rationales, the DfE contract note, the local census's MSOA entry (now D18a's reason) and the generator's SLC comment (c0dd6eae2). The regenerated receipts carry them.
  3. The complement's metadata: the formula now reads numerator / denominator, matching the recorded numerator, and the test checks that the recorded pieces give the value back (42913ab5e).

The national and local surfaces, both parity receipts and the local census are regenerated on Chronicle 53815b2 in a follow-up commit. Until it lands, CI's checks of the generated surfaces will fail.

juaristi22 and others added 2 commits October 31, 2026 17:35
The national references, membership and compile-parity receipt move only
for the three OBR council tax nation rows, now scaled by obr.council_tax's
main-residence share, and for the restated DfE and council tax rationales.
The local census carries D18a's circularity reason for ONS's small-area
income rows. The local references, the local parity receipt, the coverage
manifest and the H2 fixture regenerate unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…napshot

The restated note says each DfE row declares observation_period 2026-01
(D5); the test still looked for the replaced in-year snapshot's "one birth
cohort earlier".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Re-review of #1422 at 9d25e4579

This re-reviews the fixes in c0dd6eae2, and the regeneration in 43c3da74e and 9d25e4579, against my review at 0a69ef4b5. One reviewer re-ran each earlier finding's reproduction, a second read the fix commits for new defects, and I reproduced the blocker below. Paths are relative to packages/microcosm-build/src/microcosm/build/ unless they start with packages/, tools/ or changelog.d/.

Verdict: one new blocker, from the item 9 fix. The childcare copy gate is declared at a phase that runs before its stage, so it stops every licensed spine build (1). Item 4 is deferred as agreed, and the other round-1 items are fixed, with small residuals.

Fix before merge

1. The childcare copy gate runs before its stage.

  • uk/gates.json:1002 puts uk_stage_childcare_copy_take_up_signal in the assembled phase, and packages/microcosm-data/src/microcosm/data/contract.py:567 does the same. UK_SPINE_GATE_SCOPE arms it in the spine battery.
  • The assembled boundary is frs_brma, stage 13 of 50, but childcare_copy_take_up is stage 48. Each boundary offers evidence only for the stages that have run (uk_runtime/spine_build.py:1326, and uk_runtime/graph_evidence.py:251 in graph mode).
  • So the binding raises "stage_evidence has no receipt object for 'childcare_copy_take_up'" (uk_runtime/battery_bindings.py:327), and the gate fails closed. Because it is release-blocking, enforce stops the build at the boundary, on dev and release-candidate builds alike.
  • A synthetic smoke build records the failure and carries on, because the entry sets population_fact_check. So integration-uk passes.
  • Why the tests miss it:
    • the binding test skips the copy stage (packages/microcosm-build/tests/engine_free/uk/test_uk_take_up_gate.py:449);
    • the copy stage's own test checks only that no flag is omitted or undeclared (test_uk_childcare_copy_take_up.py:383). On that test's frame the gate would fail anyway ("max_free_entitlement_hours_used is constant"), since the frame's age-1 usage rate is 1.
  • No other assembled-phase stage gate reads a stage after the boundary.
  • Fix:
    • Move the gate to transferred, as the student_loans and isa_subscriptions stage gates are.
    • Carry the change through contract.py:567, packages/microcosm-data/tests/engine_free/shared/test_contract.py:387, the digests and the changelog's "four assembled-phase" gates.
    • Add a test that every assembled-phase stage_health gate reads a stage at or before UK_SPINE_ASSEMBLED_FINAL_STAGE, so this can't come back.

Round-1 items

# Status
1. Home values gate on "3-9" Fixed. _count_bounds (uk_runtime/stage_health.py:4641) reads each region's band at its bottom. 2, 3, 9 and 10 rows all pass, and a new test covers "3-9".
2. Pay-ratio clamp Fixed. A true ratio of 1.5 on 5 or 9 households is now flagged pay_concept_ratio_measured: False. Residual: the receipt still prints the held 1.1 as pay_concept_ratio (uk_runtime/frs_annual_income.py:943-945), and the gate repeats it beside the flag. Only the support band shows that it was held.
3. observation_period and restamps Fixed at the three named sites, which now call restamp_reference_period. tools/diagnose_uk_uc_matrix.py:389 still restamps the old way on its --references route. See also 2 and 3 below.
4. Nation rows on #1430's feed Deferred to whichever PR lands second, as your reply says. The rows are now scaled windows, still without a source_table_by_opening_year map. A trial merge with #1430 conflicts only in three notes hunks and five digest hunks. The rows read the same MHCLG facts that obr.council_tax already needs.
5. Engine-default guard Fixed. Run on #1430's stages and fixture, it flags the six company_cars outputs. So #1430 will fail it until they leave the defaults list, which is the point.
6. One series per pinned value Fixed and tested.
7. Nation basis Fixed. The three rows carry obr.council_tax's main-residence ratio, and 43c3da74e regenerates them at 0.96767 × the window. Scotland and Wales take England's share, which the notes state.
8. Withheld cells Fixed and tested.
9. Copy stage gate Added, but at the wrong phase: see 1 above.
10. Test gaps Fixed: the three named gaps are filled. The copy gate still has no end-to-end test (1).
11. Floor guard Partly fixed. It now scans tools/** and microcosm-data, and it catches < 10 on labelled lines. Still missed: unlabelled forms such as distinct_count(…) < 10, np.unique(x).size < 10, >= 10, minimum=10, "< 10" with a space, and "at least ten". On #1426's spi_income.py with only the labels fixed, it lets :961 and :1790 through. Allow-list: it matches by substring, so the uk/spec/sources.yaml:1717 needle also exempts any line containing it. Scope: of the uk/*.json files it reads only gates.json (see 4 below).
12. Floors of ten Moved, or labelled as statistical floors. Two texts remain: uk_runtime/spi_rank_transport.py:622-626 still withholds positive_rows under its 10-row constant, citing CD171 5.2.1, and uk/spec/sources.yaml:1717 still says a positive-row count under 10 is withheld. Both contradict the constant's new label. They are stricter, so nothing leaks.
13. Stale texts Fixed, the receipts included after 43c3da74e. One more of the same kind: tools/build_uk_ledger_compile_parity_signed_differences.py:617-622 still calls the ONS small-area income rows a "Signed deferral", pending MSOA-to-LA aggregation. The regenerated local parity receipt repeats that on 903 rows, while the local census now keeps them out as circular (D18a).
14. Complement metadata Fixed. The test checks the numerator, ratio and formula, but not the denominator.

New in the fix commits

2. restamp_reference_period shifts only a leading four-digit year (ledger_targets.py:4667).

  • "2024-25" becomes "2025-25", which the reference then refuses, and "FY2024" raises even at a zero shift.
  • The national compile calls it outside its try block (uk_runtime/ledger_targets.py:494 and :1430). So one such row aborts the compile instead of being recorded as unsupported.
  • No committed row has these forms, so this is latent.

3. The authoring route doesn't restamp.

  • target_reference_authoring.py:1177 and :1415 set period from --period, but :1202-1204 and :1429-1431 copy observation_period unchanged.
  • So authoring the DfE rows at --period 2026 gives "2026-01", where the committed rows restamped to 2026 read "2027-01".
  • With tools/diagnose_uk_uc_matrix.py:389 (item 3 above), that makes two routes that bypass the helper.

4. Old floor-of-ten texts in uk/*.json files the guard doesn't read. Four of them still justify a choice by the floor of ten:

  • uk/calibration_measure_exclusions.json:79 says twice that rows under 10 source households are withheld;
  • uk/target_reference_signed_exclusions.json:168 and :181, and uk/local_small_cell_rule.json:44, sign out the North East and Wales £2m rows partly because a public sign-off "would narrow an estimate expected to rest on fewer than 10 records".

Restate them, or list them as kept.

5. The home values gate reads rounded counts as exact (low; uk_runtime/stage_health.py:4641).

  • _count_bounds treats a count of 10 or more as exact, but the producer rounds it to tens. Ten regions of 15 rows print 20 each, and against 150 stage rows that fails as "more rows rewritten than the stage carries".
  • It needs nearly every row to be an England-and-Wales owner, so only synthetic frames hit it.

Checked and found true

  • Contract: the digests match the entries, and the gate battery and contract tests pass.
  • One-series check: it covers the four multi-value windows, and none collide.
  • The regenerated surfaces move only as described:
    • the three nation rows;
    • the DfE and obr.council_tax reasons in the parity receipt;
    • the ONS small-area income entry in the local census.
  • CI on 9d25e4579: lint, wheels and integration-uk pass. The engine jobs were still running.

Disclosure-control items went to the author separately.

🤖 Generated with Claude Code

- Blocker: uk_stage_childcare_copy_take_up_signal runs at the transferred
  phase. Its stage, childcare_copy_take_up, runs after the assembled boundary
  (frs_brma), so at the assembled phase the gate found no receipt and failed
  closed on every licensed spine build. The contract mirrors, the changelog
  and the digests follow. A test now fails on any assembled-phase stage gate
  that reads a stage after the boundary, and the copy gate passes end to end
  through the battery's stage_health binding on a frame whose flags vary.
- restamp_reference_period moves an annual range's two ends and a prefixed
  year, leaves any spelling alone at a zero shift, and a row it cannot move
  is recorded unsupported, not aborting the compile. The authoring tools
  move a contract's observation period with the authored period
  (observation_period_base, 2025 for the UK), and the UC matrix diagnostic
  restamps through the helper.
- The regional home values gate reads a rounded count at the counts that
  round to it. The annual income gate says when the pay concept ratio was
  held to its bounds.
- The floor guard catches a comparison with ten in any form, a keyword or
  default of ten, "< 10" and "at least ten", over the UK tools, microcosm-data
  and the hand-written UK JSON; each ten it keeps is allowed by the phrase
  that contains it, never by its line.
- The SPI transport rounds its positive-row count as every receipt count is;
  the spec says the ten rows are a statistical floor. The U3 adjudication
  states the floor of three, and the parity tool gives D18a's reason for the
  ONS small-area income rows. The complement test checks the denominator.

The gate battery and spine-part digests are re-pinned, and the coverage
manifest follows the spec.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the re-review of 9d25e4579. Everything below is at 5b693b736.

Fix before merge

  1. The childcare copy gate's phase: it now runs at the transferred phase, as the WAS, LCFS and ETB stage gates do, since its stage runs after the assembled boundary (frs_brma). The contract's mirrors, the changelog and the digests follow. A new test fails on any assembled-phase stage gate that reads a stage after the boundary. The copy gate now passes end to end through the battery's stage_health binding, on a frame with children under 1, so the under-2 usage varies.

Round-1 residuals

  • 2: the gate's details now say pay_concept_ratio_held_to_bounds when the stage held the ratio, beside pay_concept_ratio_measured: False. The receipt still prints the held value, with its support band.
  • 3: tools/diagnose_uk_uc_matrix.py restamps through restamp_reference_period.
  • 11: the guard now catches a comparison with ten in any form (< 10, >= 10, 10 <), a keyword or default of ten, "< 10" with a space, and "at least ten". It reads the UK tools, microcosm-data and the hand-written UK JSON. Each ten it keeps is allowed only by the phrase that contains that match, not by its line. A probe with your unlabelled forms fails it.
  • 12: the SPI transport now rounds its positive-row count as every receipt count is ("<3" for one or two rows), and the spec says the ten rows are a statistical floor for the distances.
  • 13: the parity tool now gives D18a's reason for the ONS small-area income rows. The local parity receipt picks it up at its next regeneration, which runs when the machine is free.
  • 14: the complement test checks the denominator too.

New in the fix commits

  • 2: restamp_reference_period moves both ends of an annual range ("2024-25" to "2025-26") and a prefixed year ("FY2024"), and leaves any spelling alone at a zero shift. A row it can't move is recorded as unsupported instead of aborting the compile.
  • 3: the authoring tools move a contract's observation period with the authored period. The UK generator declares the contract's base, 2025, so authoring at --period 2026 gives "2027-01", as the restamp does.
  • 4: the U3 adjudication now states the floor of three beside its history. The two signed £2m exclusions and the local small-cell rule are kept as they are: each estimate rests on about one household weight, so the reason holds at the floor of three too, and their texts are copied into 603 generated membership rows. The guard names them as kept.
  • 5: the home values gate now reads a rounded count as the range of counts that round to it, so ten regions printed 20 pass against 150 rows. A test covers it.

The touched test files pass locally, and CI runs the full engine-free and engine tiers.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Third review of #1422 at 5b693b736

This re-reviews 5b693b736 against my second review at 9d25e4579. As before:

  • one reviewer re-ran each earlier finding;
  • a second read the commit for new defects;
  • I checked the main points myself.

Paths are relative to packages/microcosm-build/src/microcosm/build/ unless they start with packages/, tools/ or docs/.

Verdict: the blocker is fixed, and nothing here blocks a build. Before merge:

  • regenerate the local parity receipt, since CI can't see that it's stale (item 13 in the table);
  • reconsider the new try/except in the compile; my last review pointed the wrong way there (1 below);
  • tighten the rounded-count bounds, which the commit loosened along with a test (2 below).

The rest are latent edge cases and texts.

Round-2 items

# Status
1. Copy gate phase Fixed. uk/gates.json:1002 is transferred, and the mirrors, digests and changelog follow. The new test (test_uk_stage_health.py:3387) would have failed at 9d25e4579. Through the battery, the copy gate now resolves at the transferred boundary and passes on the infant frame. One gap: live spine gates run in graph mode, which hard-codes "frs_brma" (uk_runtime/graph_evidence.py:243-245). The test reads UK_SPINE_ASSEMBLED_FINAL_STAGE, whose only user, _run_plan_with_spine_sampling (uk_runtime/spine_build.py:1286), has no callers. So the test guards the live boundary only while the two agree. Having graph_evidence.py read the constant would tie them together.
2. Restamp helper Fixed for the named forms: "2024-25", "FY2024" and "2026-01" all move correctly. Latent edges are in 3 below.
3. Authoring route Fixed for the national generator. With base 2025, the 8 DfE rows give "2025-01", "2026-01" and "2027-01" at 2024, 2025 and 2026. The local generator declares no base, which is harmless while no local row has an observation period.
4. Floor-of-ten texts Partly fixed. :79 is restated, and the kept texts are named. Still wrong: the support-floor reason at uk/calibration_measure_exclusions.json:76 still gives the floor as ten ("fewer than 10 distinct source households"), and the guard misses it (4 below).
5. Rounded counts Fixed for the round-2 case, but loosened elsewhere (2 below).
Item 2 residual: pay ratio Fixed. But: the new pay_concept_ratio_held_to_bounds is set from the support band alone (uk_runtime/stage_health.py:1317-1327), and the stage clamps only a ratio outside the bounds. So an in-bounds ratio the stage never moved is also marked held: 1.02 on 5 households, the stage test's own case (test_uk_frs_annual_income.py:544).
Item 3 residual: diagnose tool Fixed. No other UK restamp bypasses the helper.
Item 11: floor guard Partly fixed. A labels-only fix to #1426's spi_income.py now fails it, which is the case that mattered. The remaining gaps are in 4 below.
Item 13: parity reason Tool fixed, receipt stale. uk/ledger_compile_parity_local_incumbent_2025_signed_differences.json keeps the old "Signed deferral" reason on 903 rows. No CI test compares reasons: the local receipt replay matches names, kinds and values, and the only regeneration test is national and skips without the feed. So it merges stale unless it is regenerated first.
Item 14: complement Fixed.

New in 5b693b736

1. The compile's new try/except turns a contract error into an unsupported row (uk_runtime/ledger_targets.py:494-500, :1436-1442).

  • The same function says at :507-510 that a contract error should "refuse the whole compile rather than report it as an unsupported row". An observation period that can't be moved is one.
  • At the calibration year, full_targets.py still refuses, so a build stops. At validation or parity periods, though, the row becomes ledger_absent, and a regeneration would sign it with the generic reason.
  • My round-2 point 2 pointed the wrong way here. The abort was the right outcome; it only needed to name the row. Raising with the reference's name, as :507-510 does, fits the codebase's rule.
  • No test covers the branch.

2. The rounded-count bounds are too wide at 10 and 1,000, and a test was changed to fit (uk_runtime/stage_health.py:4659-4662).

  • rounded_count prints 10 only for counts of 10-14, since 5-9 print "3-9". It prints 1,000 for 995-1,049. But _count_bounds returns (5, 14) and (950, 1,049).
  • So impossible receipts now pass. For example, ten regions printed 10 (at least 100 rows) against a stage printed 90 (at most 94). That failed at 9d25e4579.
  • test_uk_stage_health.py:2802 moved its mutation from rows=90 to rows=40 to keep failing. The comment and assertion at :2844-2849 say nine regions printed 10 plus one "3-9" pass against rows=50, though they need at least 93 rows.
  • Fix: use max(v - 5, 10) below 1,000 and max(v - 50, 995) from 1,000. Only synthetic frames reach these values.

3. _shift_period_years disagrees with the ledger's own period parser (ledger_targets.py:4711-4718; latent).

  • Month or range? For 2000-2011, YYYY-MM with MM equal to the next year's two digits is read as an annual range. So "2011-12" (December) moves to "2012-13", and "2005-06" moves to "2006-07".
  • Refused spellings: the ledger accepts "2024-2025", "2024_2025" and "2026-1", but the helper refuses them at any nonzero shift.
  • Unparseable target period: "2025Q1" still turns "2026-01" into "1-01".
  • Suggestion: classify the spelling with _normalize_period_value (ledger_targets.py:5119). All committed rows are "2026-01" at 2025, so nothing is affected now.

4. The floor guard still misses common forms and has generic allow-list needles (packages/microcosm-build/tests/engine_free/uk/test_uk_receipt_counts.py:173-183).

  • Prose: in digits, prose matches only when one of seven nouns follows "10". So it misses:
    • the live text at :76 ("10 distinct source households");
    • "Fewer than 10 FRS source households" (uk/uk_population_targets.json:682, :35132);
    • "fewer than 10 awards" (uk_runtime/frs_spine.py:2016).
  • Code: it misses keywords not named minimum* (min_rows=10, min_count=10, threshold=10), > 9 and <= 9, and YAML min_rows: 10.
  • False positives: it flags < 10_000, < 10% and format specs such as f"{x:<10}".
  • Allow-list: the needles are generic, so a new if value < 10: in receipt_counts.py or stage_health.py passes.
  • Dead needle: "about one household weight" (:142) contains no ten, so it can never match.

So the reply's "a comparison with ten in any form" is overstated. Either tie each kept entry to its file and exact phrase and widen the prose match, or say in the test which forms it checks.

5. Small items:

  • No changelog fragment describes restamping observation periods or the authoring base.
  • The base 2025 is hand-set in tools/generate_uk_target_references.py:147-149, and no test ties it to the committed references' period.

Checked and found true

  • Phase move: every boundary (driver and graph) offers the copy stage's receipt at the transferred phase. Certification checks each entry's phase, and the spine scope and all mirrors include the gate. No other assembled-phase gate reads a later stage, in this tree or UK: company cars from the FRS job table (dashboard #316) #1430's. The old digest appears nowhere.
  • Coverage manifest: its 23 changed lines are the new source_stages.json hash, and tools/build_uk_release_input_coverage_manifest.py --check reports it current.
  • Compile: an unsupported reference at the calibration year still makes full_targets.py refuse, so no bound row is dropped silently.
  • Tests: the touched test files all pass, along with the contract and digest pins.
  • CI on 5b693b736: lint, wheels and integration-uk pass. The engine jobs were still running.

Disclosure-control items went to the author separately.

🤖 Generated with Claude Code

juaristi22 and others added 4 commits October 31, 2026 21:17
…LI test

The CLI test runs without --sdc-minimum-count, so its one differing row now
reads "< 3" (7a95f37 moved the default to the floor of three); the tests
that pass minimum=10 still read "< 10".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 903 rows of ONS's small-area income estimates now give the reason the
local census and references give: kept out as circular, not a signed
deferral pending MSOA-to-local-authority aggregation. Nothing else moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5b693b7 changed modules the spine graph fingerprints (the SPI rank
transport's receipt among them), so the fixture's node implementation
hashes moved; tools/graph_uk_spine_fixture.py --check now regenerates it
byte-identically, and the H2 parity test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…texts

- A reference whose observation period cannot move stops the compile,
  naming the row, as the function's other contract errors do: an unsupported
  row would let a validation or parity compile drop it (finding 1).
- The rounded-count bounds start where rounded_count starts printing a count:
  10 stands for 10 to 14 and 1,000 for 995 to 1,049; the home values test
  is restored to the impossible receipts it caught (2).
- _shift_period_years classifies a spelling as the ledger's own parser reads
  it, so "2011-12" is a month and "2024-2025" a range, and a target period
  without a year refuses (3).
- The floor guard names each kept ten by its exact phrase and count, finds
  prose before any noun, keywords and keys named min*, minimum* or
  threshold*, "> 9" and "<= 9", and skips format specs, 10_000, 10% and
  10 * x; its docstring lists the forms it checks (4).
- UK_TARGET_REFERENCE_PERIOD names the period both generators author at
  and the base of the contract's observation periods; a test ties it to the
  committed references, and the observation-period changelog describes the
  restamp and the authoring base (5).
- The support floor's reason states the floor of three; the receipt says
  whether the stage held the pay concept ratio, so an unmoved ratio is not
  reported held; and the graph's spine gates read UK_SPINE_ASSEMBLED_FINAL_STAGE
  (now in graph_evidence), the boundary the stage gate test checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the third review, of 5b693b736. Everything below is at 821e342d1.

Before merge

  • The local parity receipt (item 13): regenerated in ce9344c68. Its 903 ONS small-area income rows now give D18a's reason, and nothing else moves. bd9839bf2 also regenerates the H2 fixture, which 5b693b736 had made stale.
  • 1. Restamp errors: reverted to stopping the compile. A reference whose observation period can't be moved now raises, naming the row, as the function's other contract errors do. A test covers the branch.
  • 2. Rounded-count bounds: a printed count now stands only for counts that print as it: 10 for 10 to 14, and 1,000 for 995 to 1,049. The home values test is back to rows=90 for its mutation. The "3-9" case now says that nine regions printed 10 and one of three need 93 rows, so it fails against a stage printed 80 and passes against one printed 90.

Round-2 items

  • 1, the graph boundary: UK_SPINE_ASSEMBLED_FINAL_STAGE now lives in graph_evidence.py, and the graph's spine gates read it. So the stage-gate test checks the boundary the build runs.
  • 4: the support floor's reason now gives the floor of three.
  • Pay ratio: the receipt says whether the stage moved the ratio onto a bound (pay_concept_ratio_held, withheld beside a withheld ratio), and the gate reports pay_concept_ratio_held_to_bounds from it. So 1.02 on five households is unjudged, but not reported as held.

New in 5b693b736

  • 3. Periods: _shift_period_years now classifies a spelling as _normalize_period_value reads it:

    • "2011-12" and "2005-06" are months, so only their year moves;
    • "2024-2025" and "2024_2025" are ranges, so both ends move;
    • "2026-1" moves its year;
    • a target period without a year, such as "2025Q1", refuses.
  • 4. The floor guard: each kept ten is now named by its exact phrase and the number of matches it covers in its file. A new if value < 10: fails, and so does an entry that matches nothing, so the dead one is gone. The guard now finds:

    • "fewer than", "under", "below", "less than" or "at least" before "ten" or "10", whatever noun follows ("10 distinct source households", "10 awards");
    • keywords, keys and defaults named min*, minimum* or threshold* set to ten, in Python, YAML and JSON;
    • > 9 and <= 9.

    It skips format specs, 10_000, 10% and 10 * x. Its docstring lists the forms it checks, rather than claiming every form. The two "Fewer than 10 FRS source households" notes and "fewer than 10 awards" are named as facts that hold at the floor of three.

  • 5: UK_TARGET_REFERENCE_PERIOD (2025) names both the period the two generators author at and the base of the contract's observation periods. A test ties it to the committed references, and the observation-period changelog now describes the restamp and the authoring base.

The touched test files, the H2 parity test and the H2 fixture check pass locally. CI runs the full tiers.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Fourth review of #1422 at 821e342d1

This re-reviews 8f7d69204, ce9344c68, bd9839bf2 and 821e342d1 against my third review at 5b693b736. As before:

  • one reviewer re-ran each earlier finding;
  • a second read the commits for new defects;
  • I checked the main points myself.

Paths are relative to packages/microcosm-build/src/microcosm/build/ unless they start with packages/ or tools/.

Verdict: the round-3 items are fixed, and so are the two CI failures since 5b693b736. Nothing here blocks a build. Two of the fixes regress latent cases, and in both the author followed my round-3 suggestion: periods under a typed selector (1) and the floor guard on a sentence-final "ten." (2). Item 3 is a trap for whoever next moves the reference year.

Correction to my last review: it said the engine jobs at 5b693b736 were still running. They then failed:

  • engine-free on the payload comparator's CLI test, stale since 7a95f37a7 and also failing at 9d25e4579;
  • engine-uk on the H2 spine parity test.

8f7d69204 and bd9839bf2 fix them. engine-free runs with --maxfail=1, so each run shows only its first failure.

Round-3 items

# Status
Local parity receipt Fixed. Only the 903 reasons changed, and they now equal the tool's text. Can recur: nothing in CI compares the local receipt's reasons with the tool.
H2 fixture Fixed. One stage-contract hash and the transport text changed, and every node's hash recomputes.
1. Contract errors Fixed. Both compiles raise and name the row, and the national branch's test fails if that is reverted. Gaps: the local branch has no test, and its message says "UK target reference" where its sibling says "UK local reference".
2. Count bounds Fixed. I checked to 20,000: the bounds now equal exactly the counts each printed value stands for, and rows=90 is back. Gap: nothing pins the 1,000 edge; reverting it to v - 50 passes every test.
3. Period spellings Partly: a regression under typed selectors (1 below).
4. Floor guard Partly: a regression on sentence-final tens (2 below).
5. Base and changelog Partly (3 below).
Graph boundary Fixed. The graph's spine gates read the constant from uk_runtime/graph_evidence.py. Residuals: uk_runtime/graph.py:1719-1720 still names "frs_brma" for the checkpoint node, though a divergence fails loudly. The test imports the constant through spine_build's re-export (test_uk_stage_health.py:3394), which breaks if the dead plan path is removed.
Support floor reason :76 now gives the floor as three.
Pay ratio Fixed. 1.5 on five households reads as held, 1.02 on five doesn't, and ten or more behave as before. A receipt from before this commit just lacks the flag.

New in these commits

1. Periods shift wrongly under a typed selector (ledger_targets.py:4716, against :5063-5068).

  • The helper now classifies a spelling without a type. The exact-match path, however, reads an observation period with the selector's period_type.
  • Under a fiscal_year selector, "2000-01" to "2011-12" are fiscal ranges. So "2011-12" +1 now gives "2012-12", which no fiscal-year fact matches; round 3 gave "2012-13".
  • My round-3 suggestion to classify with _normalize_period_value caused this. The helper should classify with the reference's period_type when it has one.
  • A target period of "FY25" still turns "2026-01" into "26-01".
  • It's latent: every committed observation period is "2026-01", with no typed selector.

2. The floor guard's new lookahead misses a sentence-final ten (packages/microcosm-build/tests/engine_free/uk/test_uk_receipt_counts.py:213).

  • (?![\d_.%]) rejects a following full stop. So these now pass, though round 3 caught them:
    • "fewer than ten." or "under 10." at the end of a sentence;
    • constants and keys set to 10.0.
  • Fix: (?![\d_%]|\.\d) would still skip "10.5" without losing them.
  • Wrapped prose: it still misses prose split across a line. tools/uk_private_rent_receipt.py:416-417 still says "fewer than ten" across a line break, so the guard can't see it.
  • False positives: it still flags f"{count:>9}" and minutes=10. These fail loudly, so they're harmless.

3. One constant plays two roles (uk_runtime/ledger_targets.py:480-484).

  • UK_TARGET_REFERENCE_PERIOD is both the generators' default --period and the contract's observation base.
  • The trap:
    • Moving the references to 2026 the right way, with --period 2026, gives "2027-01" but fails the new test (test_uk_target_references.py:2239).
    • Bumping the constant instead passes the test, but it moves the base too. The DfE rows then read "2026-01" at 2026: the model year's own census, the error D5 fixed.
  • Suggestion: keep the base separate, since it belongs to the contract, and check that each observation period is the January after it.
  • The test also checks only the national references. The changelog's "classified as the ledger's own period parser reads its spelling" doesn't hold for typed selectors (1).

Checked and found true

  • Regenerated files: the receipt and fixture diffs contain only the stated changes. The touched modules import with no cycle, and the graph's stage order matches the declared roster.
  • Restamps: the try wraps only the restamp, so fact gaps are still recorded as unsupported.
  • Manifest: the coverage manifest is current.
  • Tests: the touched test files pass.
  • CI on 821e342d1: every job passes.

Disclosure-control items went to the author separately.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Nov 1, 2026
María plans to merge #1422 first, so #1430 builds on
it (uk-1095-next at 0a69ef4).

- The lock is #1422's policyengine-uk 2.165.1, with its stamps, pins and
  version mentions.
- #1422's three OBR council tax nation rows declare
  source_table_by_opening_year, as the other windowed OBR rows do on the
  08962f1 feed. The council tax, SDLT and UC notes #1422 restated keep the
  edition sentence.
- UK_ENGINE_DEFAULT_INPUTS signs only the three company car inputs the
  stage leaves at the engine's defaults, with a reason that states the
  full-year approximation. The six the stage writes are no longer signed.
- Both sides' gates are in the contract mirrors, and the gate battery and
  certification digests are re-pinned.
- Both sides' calendar-year window tests are kept.
- The generated surfaces keep one side here: the national references,
  membership and compile-parity receipt, the coverage manifest, the H2
  fixture and the concept coverage. The commits that follow regenerate
  them on the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
The review is issuecomment-6545997069 on #1430. María's rulings of 31
October 2026: carve upstream, fuel first, and measure the EPB shortfall
before correcting it.

- Items 2 and 10: company_cars runs right after hmrc_spi_income_spine,
  so every later stage reads SPI-channel pay without the car benefits.
  That covers salary_sacrifice and its bounds, the take-up draws'
  earnings rules, the gainful self-employment flag and the CGT stages,
  and the identity tool's salary-sacrifice carrier holds. The carve no
  longer keeps room for pension deductions; salary_sacrifice bounds them
  on the carved pay.
- Item 11: only tape rows (person_is_spi_income_recipient) carve. The
  SPI channel's undrawn copies and dependants keep their twin's pay and
  car as FRS rows do, and the pay identity is checked on the tape rows.
- Item 7: where the car and fuel benefits do not fit inside EPB together,
  the fuel goes first, so a car that fits without it is always kept. An
  unknown-fuel holder who reports private fuel draws petrol or diesel.
- Item 3: the receipt records the weighted holders and list price the
  tape rows lose for want of EPB.
- Item 4: the receipt records the list price the scaling takes off and
  that of the cars it drops. The docs say the electric range and RDE2
  assumptions set a scaled car's price, so they move the 2030-31 base.
- Item 5: the receipt states the CO2 shares of the cars the output holds
  beside the draw's.
- Item 6: the docs say the receipt's figures are at the stage's weights,
  and the results measure the base on the calibrated file.
- Item 1: obr.ni_employer binds ni_class_1_employer. OBR's line is table
  4.4 row 18, Class 1 only; Class 1A is row 21. Three notes no longer say
  that no household carries Class 1A.
- Item 8: the employment income count rows filter on pay plus the car
  benefits, as the amounts read.
- Item 9: band 1's uniform draw starts at the declared minimum list
  price, GBP 5,000.
- Item 12: company_cars gains a coverage-manifest family.
- Item 13:
  - the annual simulation makes a car available for the share of the
    year the pay covers;
  - the fact fence's count is compared with the vendored rows;
  - CARVAL's bounds and FUELTYP's codes are defined once in frs_spine;
  - the stage's helper modules gain the engine adapter, spi_income and
    stochastic_assignment;
  - the stage notes and graph comments are restated.
- Cross-PR: the guard test counts every stage's declared outputs. The
  policyengine-uk 2.165.0 fragment gives way to #1422's 2.165.1.

The generated surfaces are regenerated on 08962f1 in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's head is now 43c3da7 (its review commits 7a95f37, 42913ab
and c0dd6ea, and their regeneration on 53815b2).

- The three OBR council tax nation rows are now
  calendar_year_window_scaled_by_ratio with obr.council_tax's MHCLG
  operands. They keep source_table_by_opening_year beside them, and their
  restated notes keep the edition sentence.
- The engine-defaults guard is #1422's: it counts every stage's declared
  outputs and rewrites. One assertion is added, that it sees the company
  car inputs.
- Both sides' gates are in the contract mirrors, and the gate battery and
  certification digests are re-pinned.
- The generated surfaces keep this branch's side, to be regenerated on
  08962f1 next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 9d25e45 checks the DfE contract note for its observation
period, not the replaced snapshot sentence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
Run on the merged tree, one job at a time, from the 08962f1 cuts:

- The vendored resources: re-vendored on the full artifact. All 36 are
  byte-identical to the merge.
- The national references and membership: 1,409 active, 30 signed out.
  Relative to this branch's last regeneration, only #1422's changes move:
  - the three OBR council tax nation rows resolve, scaled to main
    residences, with their edition maps;
  - obr.council_tax moves -3.2% (main residences);
  - the DfE funded childcare rows take their 2026-01 observation period;
  - the SLC devolved total splits into Scotland, Wales and Northern
    Ireland;
  - the TFC top-up, the NI bus receipts and Scottish Child Payment are
    restated.
- The national compile-parity receipt follows.
- The local census, references, membership and compile-parity receipt are
  byte-identical to the merge.
- The coverage manifest, the H2 fixture (company_cars after the SPI income
  stage) and the policyengine-uk 2.165.1 concept coverage are regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 5b693b7 moves the childcare copy take-up gate to the transferred
phase and restates restamps, counts and floors. The gate battery and
certification digests are re-pinned; the coverage manifest is regenerated
next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
- The national references, membership and compile-parity receipt are
  unchanged.
- The local compile-parity receipt takes #1422's restated rationale for its
  903 ONS small-area rows; no value moves.
- The coverage manifest and the H2 fixture follow the round's stage notes
  and receipts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 8f7d692 expects the comparator's default SDC minimum of three in
its CLI test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's ce9344c regenerates its local parity receipt with D18a's
reason on feed 53815b2. This branch's receipt on 08962f1 already
carries that reason, and the two files are identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's bd9839b regenerates its H2 spine parity fixture on #1422's
code. This branch keeps its own, regenerated after merging 5b693b7
on code that adds the company_cars stage. It already carries the SPI
receipt wording bd9839b brings, and it records this branch's own
hmrc_spi_income_spine contract digest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 821e342 answers its third review: restamp failures raise and
name the row, the reference period is one constant
(UK_TARGET_REFERENCE_PERIOD), the spine gates read the assembled
boundary from graph_evidence, and the stage-health count bounds and the
floor-of-ten guard are tighter. It merges without conflicts, and
company_cars stays in the transferred phase, after frs_brma.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion base

- A restamp, and the authoring tools, read an observation period under the
  selector's period_type when it declares one, as the exact-match path does:
  under fiscal_year "2011-12" moves to "2012-13", untyped or under month to
  "2012-12", and a spelling the type cannot read refuses. A target period
  must give a four-digit year, so "FY25" refuses (1).
- The floor guard reads "10." at a sentence's end and 10.0 as ten, still
  skipping 10.5, 10% and 10_000; prose matches across a line break, a
  comment marker or a string's quotes; the constant pattern is
  case-sensitive and skips MINUTE, so minutes=10 and {count:>9} no longer
  match. The rent receipt tool's docstring still said fewer than ten: it
  gives the floor of three it uses. The pay ratio's wrapped "one to nine
  households" (statistical) and the SPI tape's "at least ten working days"
  (licence) are kept by phrase (2).
- UK_OBSERVATION_PERIOD_BASE holds the contract's base apart from
  UK_TARGET_REFERENCE_PERIOD, the generators' default period. Tests check
  that the contract's observation periods are the January after the base,
  and that each national and local reference reads the January after its
  own period; the changelog says the period is read under the selector's
  type (3).
- Round-3 gaps: a test replays the tool's reasons over both committed
  parity receipts; the local compile's restamp error names a local
  reference, with a test; a test pins every printed count's bounds up to
  20,000, the 1,000 edge included; UK_SPINE_ASSEMBLED_FINAL_STAGE moves to
  uk_runtime/graph.py, which builds the checkpoint from it, and the
  stage-gate test imports it from there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the fourth review, of 821e342d1. Everything below is at 0da33d0cd. Noted on the engine jobs: 8f7d69204 and bd9839bf2 fixed both failures.

New in the round-3 commits

  • 1. Periods under a typed selector: the restamp and the authoring tools now read an observation period under the selector's period_type when it declares one, as the exact-match path does.

    • Under fiscal_year, "2011-12" moves to "2012-13". Under month, or with no type, it moves to "2012-12".
    • "2026-01" spells no fiscal year, so a fiscal-year selector refuses to move it.
    • A target period has to give a four-digit year, so "FY25" refuses rather than moving "2026-01" to "26-01".
    • Tests cover each case.
  • 2. The floor guard:

    • A ten is now 10, 10. at the end of a sentence, or 10.0. The guard still skips 10.5, 10% and 10_000.
    • Prose matches across a line break, a comment marker or a string's quotes. That found three wrapped tens:
      • the rent receipt's "fewer than ten". It was stale: the tool already withholds below the floor of three (MIN_SOURCE_HOUSEHOLDS), and its docstring now says so;
      • the pay ratio's "one to nine households", which is a statistical floor;
      • the SPI tape's "at least ten working days" for HMRC's review, which is a licence term.
    • The last two are kept by their exact phrases.
    • {count:>9} and minutes=10 no longer match: the pattern for constants is case-sensitive now and skips MINUTE.
  • 3. The base: UK_OBSERVATION_PERIOD_BASE (2025) now holds the contract's base, separate from UK_TARGET_REFERENCE_PERIOD, the generators' default --period. Two tests replace the old one:

    • the contract's observation periods are each the January after the base;
    • for the national and the local references, each reference whose contract target declares an observation period reads the January after its own period, and no other reference declares one.

    So authoring at 2026 gives "2027-01", which passes once the default moves with it, and moving the default leaves the base alone. The changelog now says the period is read under the selector's type.

Round-3 gaps

  • Local parity receipt: a new test replays the tool's reason notes over both committed parity receipts. It fails on any reason the tool would now write differently, and changing one of the tool's texts marks its rows stale. The national replay needs the feed, so CI didn't check either receipt's reasons before.
  • Contract errors: the local compile's message now says "UK local reference". A test covers that branch with a real fiscal-year reference that can't move.
  • Count bounds: a test checks every count up to 20,000. Each printed count's bounds equal the lowest and highest counts that print as it, and putting the 1,000 edge back to v - 50 fails it.
  • Graph boundary: UK_SPINE_ASSEMBLED_FINAL_STAGE now lives in uk_runtime/graph.py, which builds the checkpoint node from it. graph_evidence.py, spine_build.py and the stage-gate test import it from there. Each touched module imports on its own in a fresh interpreter.
  • Pay ratio: a receipt from before 821e342d1 lacks the flag, so the gate doesn't report its ratio as held. A thin ratio still reads as unmeasured. No change.

The touched test files pass locally, and so do the H2 fixture check, the H2 parity test, the coverage manifest check and the gate digests. CI runs the full tiers.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Fifth review of #1422 at 0da33d0cd

This re-reviews 0da33d0cd against my fourth review at 821e342d1. As before:

  • one reviewer re-ran each earlier finding;
  • a second read the commit for new defects.

Paths are relative to packages/microcosm-build/src/microcosm/build/ unless they start with packages/ or tools/.

Verdict: the round-4 items are fixed, CI is green, and nothing blocks a build. Two of the fixes regress latent cases:

  1. under the default match policy, typed selectors now refuse to restamp. My round-4 suggestion pointed there, again;
  2. the floor guard's case-sensitive constant pattern drops lowercase floor names.

Round-4 items

# Status
1. Typed periods Fixed for exact. A fiscal "2011-12" now moves to "2012-13", and "FY25" refuses. But see 1 below.
2. Floor guard Fixed. Sentence-final tens, 10.0 and wrapped prose are caught, and the three wrapped tens are real and counted. But see 2 below.
3. Observation base Fixed. Every mutation I tried fails the new tests, including the old trap with both constants at 2026.
Parity-reason replay Partly. It runs feed-free in CI and catches changed tool texts and stale ruled reasons. Not caught: rows that keep the classifier's generic text (318 national, 288 local), and a removed national rule. So "fails on any reason the tool would now write differently" overstates it.
Local message Fixed, but its test pins 1's regression (below).
Count bounds Fixed. Every edge mutation fails the test.
Graph boundary Fixed. There is one definition, in uk_runtime/graph.py, with no import cycle, and #1430's head merges with it cleanly.

New in 0da33d0cd

1. Typed restamps now refuse what the default policy reads (ledger_targets.py:4691-4695; target_reference_authoring.py:151).

  • Mechanism:
    • The restamp reads the observation period under the selector's type for every policy. Only exact reads it that way.
    • The default latest_not_after, which every committed DfE row uses, reads it untyped as a cutoff (:4766, :4853-4854, :4890).
  • Example: an academic_year or fiscal_year selector with observation "2026-01" and the default policy selects AY2025-26 at 2025.
    • Round 4 moved it to "2027-01", which selects AY2026-27.
    • Now it raises "cannot move the period '2026-01' by 1 year(s)". That stops the whole compile (uk_runtime/ledger_targets.py:508-516, :1453-1461) and the generators.
    • A month selector with an annual cutoff fails the same way.
  • Cause and fix: my round-4 suggestion didn't distinguish match policies. The fix is to read the period the way the policy that consumes it does: typed for exact; for latest_not_after, typed where the type can read the spelling and untyped otherwise.
  • The new local test (test_uk_ledger_targets.py:2812-2846) builds exactly this reference and calls it unmovable, so it pins the regression.
  • Impact: latent, since no committed reference has both a typed selector and an observation period.
  • Residuals:
    • the int fast path ignores the type (:4727-4728);
    • the constructor still validates the observation period untyped (:356-360), so a valid fiscal "2099-00" is refused;
    • with an untyped exact selector the match falls back to the fact's own type (:5092), so an untyped "2011-12" still restamps to "2012-12". The changelog's "as the exact-match path does" holds only for typed selectors.

2. The floor guard's case-sensitive constant pattern drops lowercase floors (packages/microcosm-build/tests/engine_free/uk/test_uk_receipt_counts.py:225).

  • Now missed: rounds 3 and 4 caught lowercase names at the start of a line. This head misses:

    • floor = 10, support_floor=10, and small_cell_floor = 10;
    • sdc_floor = 10 and disclosure_floor: int = 10;
    • row_threshold = 10.

    No live instance is affected.

  • Fix: the two false positives the reply credits to case sensitivity go away because of the new lookbehind (:222) and the narrowed key pattern (:218). So IGNORECASE can stay, with the words matched as snake-case tokens, e.g. (?:^|_)(?:min|minimum|floor|sdc|small|threshold)(?:_|\b).

  • Docstring claims it doesn't meet:

    • #: comment continuations. One live, harmless wrap sits at uk_runtime/target_weights.py:101-102.
    • Float-, Optional- and Final-annotated defaults.
    • Single-quoted dict keys.
  • False positives: fill-character format specs such as {n:0>9} are flagged, though they fail loudly.

  • Brittleness: the kept phrases embed indentation and line breaks, so reflowing those comments fails the guard, again loudly.

3. Small: the observation-period tests require each period to be the January after the reference period, and call int() on that period. A fiscal or prefixed observation period, or a string target period like "2025-26", would fail them. That's fine for D5, but it deserves a comment in the test.

Checked and found true

  • Typed shifts: every spelling each type can read moves correctly, with its start and range end both shifted. Untyped results are identical to round 4.
  • Generators: both use the default --period of 2025 and the base of 2025. The DfE rows at 2024, 2025, 2026 and 2030 give "2025-01", "2026-01", "2027-01" and "2031-01".
  • Graph: imports are clean, and the graph's loud failures on a missing boundary are intact.
  • Guard: it scans the tree in 1.3 s with no catastrophic backtracking.
  • Cross-PR: neither SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428 nor UK: company cars from the FRS job table (dashboard #316) #1430 declares observation periods, and the guard flags none of their added lines.
  • CI on 0da33d0cd: every job passes.

Disclosure-control items went to the author separately.

🤖 Generated with Claude Code

… parity reasons

- Read an observation period as its match policy does: under the
  selector's period_type for an exact match, untyped for the
  latest_not_after cutoff. A spelling read two ways refuses to move:
  an untyped exact "2011-12", and a cutoff the selector's type reads
  otherwise. The constructor checks an exact period under its type,
  and a bare year is checked under the type before it moves.
- The floor guard matches the floor words as snake-case words of a
  name in any case, defaults under any annotation, single-quoted keys
  and #: continuations, skips format specs with a fill, and matches
  kept phrases across whitespace. target_weights.py's count of homes
  is kept.
- The parity receipt replay starts each row from the classifier's
  reason for its kind (PARITY_DIFFERENCE_REASONS, now one constant in
  gates.py), so a changed classifier text or a removed rule fails.
- Note the observation-period test's assumption about D5's spellings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the fifth review, of 0da33d0cd. Everything below is at 4af602d81.

New in 0da33d0cd

  • 1. Restamps under the default policy: fixed. An observation period is now read as its match policy reads it: under the selector's period_type for exact, and untyped for the latest_not_after cutoff.
    • A fiscal- or academic-year selector's "2026-01" moves to "2027-01" again, and a month selector's "2026" to "2027".
    • The suggested rule for the cutoff, typed wherever the type can read the spelling, still picks wrong for one spelling. Take "2011-12" under the default policy:
      • the cutoff reads it as December 2011, and it reads the facts' own spellings untyped too;
      • under a fiscal-year selector it has to become "2012-13" to keep selecting the year's own fact;
      • under a calendar-year selector it has to become "2012-12".
    • So where the selector's type reads a spelling differently from the cutoff, the restamp now refuses. An untyped exact match refuses "2011-12" the same way, since it reads the period under each fact's own type. A prefixed or four-digit spelling reads one way.
    • The local compile test now uses "FY26". The parser reads it as the year 26, so it can't move under any policy.
    • Residuals:
      • the int path checks the year under the type, so a month type refuses a bare year;
      • the constructor checks an exact period under the selector's type, so a fiscal "2099-00" stands, and an exact fiscal "2026-01" is refused when the reference is built;
      • the changelog and docstrings no longer say "as the exact-match path does". They describe each policy's reading and the two refusals.
  • 2. Floor guard: IGNORECASE is back, and the floor words (min, minimum, floor, sdc, small, threshold) are matched as snake-case words of a name.
  • 3. Observation-period tests: a comment now says D5's periods are untyped Januaries after an integer year, and that a fiscal, prefixed or range-spelt period needs its own expectation.

Parity-reason replay: my round-4 reply overstated it.

  • The classifier's four reasons now live in one constant, PARITY_DIFFERENCE_REASONS in gates.py, with unchanged texts.
  • The test starts each row from its kind's reason, then replays the tool's rules, so a changed classifier text and a removed rule fail too.
    • Removing the PIP rule fails the national receipt.
    • Changing the fixture-only text fails the national receipt too. The local receipt's generic rows are all calibration drift, so a change to that text fails both.

The touched test files pass locally, and each fix fails its tests under mutation. The gate digests, the coverage manifest check and the H2 fixture check pass too. CI runs the full tiers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants