Repository navigation
Release the third uk-data batch - #608
Merged
Merged
Conversation
The cash-receipts parser targeted Table 3.8 "National insurance contributions" (total NICs, £200.08bn in 2025-26) on the ni_employee variable, which _parse_nics also targets at the Class 1 employee figure (£49.54bn). The two targets on one matrix column contradicted each other; the seeded constituency calibration on main b45c373 ended with obr/ni at 0.242 of target. Drop the row rather than retarget it: the Table 3.4 class targets (employee, employer, self-employed) already cover every NIC class the data populates, and a total target on total_national_insurance would conflict with them through the statutory payment recoveries and "Other NIC" lines PE-UK does not model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- test_obr_target_mapping: pin the variable of every target parsed from the committed receipts workbook, the Table 3.4 rows behind the NIC class targets, and the source identity that rules out a total-NICs target. - test_obr_target_conflicts: no two calibration targets whose matrix column is the same plain variable sum or count may disagree on a year's value, checked over the OBR targets offline and over every target the national matrix uses; it flags the removed obr/ni mapping, and a seeded random check holds the grouped check to its pairwise definition. - build_loss_matrix.calibration_targets: the target list create_target_matrix already built inline, extracted so the test checks the same set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PolicyEngine UK's year N is tax year N to N+1: convert_to_fiscal_year_parameters gives every parameter, the uprating indices included, its value at 30 April N. hmrc_spi mapped the 2023-24 SPI to 2024 and hmrc_cgt followed, so both treated 2023-24 outturns as 2024-25 values and every projected year carried one year too little uprating. The FRS release (2024-25 -> 2024), OBR targets, the SPI microdata (2022-23 -> 2022) and #533's salary sacrifice tables already use the start year; the end-year mapping arrived without stated reasoning in the February 2026 registry refactor (de061fc). - _SPI_YEAR and _CGT_BASE_YEAR move to 2023; incomes_projection.csv is regenerated from the same ODS (its 2023 rows equal main's 2024 rows). - The CGT base year is now gated at the £6,000 AEA the outturn was realised under. - Tests tie _SPI_YEAR to the tax year in the configured ODS name, check the CGT base-year AEA against policyengine-uk, and (as in #541) check that the committed projection is the SPI table uprated with the committed factors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_loss_matrix fetches targets without a year filter, and the registry keeps only the first target of each name. hmrc_spi emitted each band twice: the ODS outturn (base year only), then the projection (later years). The projected target was dropped, so _resolve_value carried the base-year outturn forward unuprated: the 2025 calibration has targeted raw 2023-24 SPI values since the February 2026 registry refactor. get_targets now merges the two into one target per band, and a test checks every projected band, variable and year through the calibration's own resolution path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derive_is_in_approved_training_from_frs_person read a `train` column. No FRS release has one (2020-21 to 2024-25 carry TRAIN2 and TRAINEE), and the function returned False when the column was missing, so every record was coded as not in approved training. DWP's FRS counts a 16- to 19-year-old as a dependent child only when in full-time non-advanced education or unwaged government training. In FRS 2024-25 all 20 child-table records aged 16 to 19 outside full-time education have TRAINEE = 1 (33.0k grossed), and uk-data coded every one as failing the qualifying-young-person education-or-training condition. Read TRAINEE = 1 (yes; the same yes/no format as EDUCFT), and fail the build when the column is missing instead of defaulting to False. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #561 (Subfleet 20261010-193520-ud561-review-r1) found that an adult-table trainee aged 16 to 19 would become a qualifying young person of their own benefit unit. FRS 2024-25 has none, but 2020-21 has one and 2023-24 three. The FRS has already decided those records are not dependants, and the adult TRAINEE question does not say the training is unwaged, so it may be training under a contract of employment (UC reg. 5(3)). - Flag TRAINEE = 1 on child-table records only. On FRS 2020-21, 2022-23, 2023-24 and 2024-25 the result equals the child-table records aged 16 to 19 outside full-time education exactly (8, 26, 16 and 20 records). - Say why TRAINEE and not TRAIN2: `train` was TRAIN2's question name, only the adult table has TRAIN2, and its value labels are not in the repo. - Property test over numeric, float and string codes and adult/child status; a create_frs test with an adult-table trainee in the same household. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delta review r2 of #561 (Subfleet 20261010-201405-ud561-review-r2, APPROVE WITH NITS at c952694): - The docstring said a dependant outside full-time education "carries TRAINEE = 1" as if it were guaranteed. It is what the four FRS releases checked show, so say that, and call the adult exclusion a proxy. - The adult-trainee fixture set age80 = 18 but kept age = 30, and create_frs adds the two, so it built a 48-year-old. Set age = 0 and assert age 18. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 (APPROVE_WITH_NITS): with the live ODS download failing, the merge test passed even without the merge, because only the projection targets were emitted. The test now serves get_income_band_table from incomes.csv, checks the outturn year as well as the projected years through the calibration resolver, and requires every SPI target to carry all of them. The comment now names 30 April N, the date convert_to_fiscal_year_parameters samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1.58.0 enhanced FRS carries £53bn of gains above the annual exempt amount, and -£15bn of losses on 1.4m people, against the £65.9bn the build targeted and the £119.3bn HMRC now reports for individuals in 2024-25 (uk-data#570). The calibration started 17 times over on CGT taxpayers and 5 times over on gains: the has-gains copy of each zero-weight SPI household kept a zero weight, so it received a full zero-weight prior, the same as its no-gains original. After 512 epochs the taxpayer count was still 56% over while the £12,300 to £250,000 bands had overshot to 23-64% under. - Split every household's mass between itself and a has-gains twin by the Advani-Summers share for its income band, in closed form, and split a zero-weight household the same way. - Give no twin to draws at or under the lowest HMRC band limit, losses included: no target describes them. - Stack donors only for the bands from £500,000 up. - Scale the calibration's starting weights so each HMRC size band's taxpayer count begins on target (rake_priors_to_targets). - Target Table 2.1a for 2024-25 (2026 release), individuals only, from the £3,000 exempt amount up, with the aggregates as the band sums. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The has-gains copies of zero-weight SPI households now carry a weight, so that the calibration starts from the right share of each. Salary sacrifice, student loan plans and Pension Credit take-up count people on survey weights and draw per record. Run before capital gains, they see neither those weights nor the copies, and a copy matches its original in everything but the gain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- test_obr_efo_fallback: the receipts and NICs set from #536 listed obr/ni, which this branch no longer emits. - test_obr_target_conflicts: #490/#530 added Target.countries, and create_target_matrix zeroes a column outside them, so the column key now includes countries. The random differential check uses Hypothesis, now a dev dependency, and draws country restrictions too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- obr.py: name what PE-UK lacks (statutory payment recoveries, Class 1A, 1B, settlements, unallocated) and say ni_class_3 exists but is unfilled; the _parse_nics docstring no longer claims it covers Class 3. - test_obr_target_conflicts: compare every pair in a shared column (isclose is not transitive), key countries as a set (restrict_to_countries uses np.isin), guard the registry-wide test against a silently empty registry, draw carry_forward and both orders of a two-country restriction, and pin the tolerance edge with an example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sol review r2: the pairwise reference checked a pair only in its own listed years, so two carried-forward targets that disagree in a year only a third target lists were reported by the grouped check alone. The reference now uses the column's full year set, and a regression pins the reviewer's three-target case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OBR targets came from whatever obr.uk served, falling back to committed workbooks matched by URL substring, with the vintage label taken from sources.yaml rather than the bytes, year columns fixed by letter, and missing rows logged and dropped. A manifest per EFO vintage (targets/source_manifests/) now names the two workbooks with their SHA-256, size and own title, the unit and financial years every parsed table must head, and the targets they must yield. Builds read only the verified committed bytes; each table's years come from its header row; a missing target or year, or an undeclared one, fails the build. `python -m policyengine_uk_data.targets.obr_sources ingest` adopts a new vintage from browser-downloaded workbooks and prints each target's change. On the March 2026 bytes all 29 OBR targets are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calibrate_local_areas saved and returned the weights only when epoch % 10 == 0, so a 512-epoch build shipped the state after step 511 and a 32-epoch build the state after step 31, dropping the last update (#456). Save at the last epoch too, in both the verbose and the non-verbose loop, so the saved file, the returned dataset and the logged fit are the final optimiser state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
create_target_matrix skipped a target whose year did not resolve with no log line (#456), skipped one whose column raised with a warning, and accepted columns no household contributes to (#593). It now records each offered target's status and fails unless every target is in the matrix with a finite value and signal, or is listed in targets/known_gaps.yaml under that status with a reason and an issue. Errors and non-finite values can never be listed; a listed gap that no longer applies is logged as stale. known_gaps.yaml lists the 34 gaps measured on uk-data 1.58.1 at the calibration year 2025: 15 two-child-limit targets dated 2026 only, and 19 targets with no signal. The matrix built is unchanged. The diagnostic get_loss_results and the CPS transfer reweighting opt out. `python -m policyengine_uk_data.targets.coverage` reports every target's status on a dataset (names, statuses and counts only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pull request CI now also builds the PR's base, in the same way as the head (TESTING, one clone) and with the head's lockfile, so both run one model and environment. A new job compares the two final-epoch fits from their calibration logs. It fails when the share of national or local targets within 10% falls by more than 2pp, flags national targets whose error rises by more than 0.2, and posts the comparison on the PR. The thresholds (targets/fit_policy.yaml, adopted under d1344) come from five builds of the same data code: their shares moved by at most 0.93pp (national) and 0.69pp (local), and they flagged 0 to 6 targets per pair. Per-target moves only flag while #598 (two weight sets from one code) is open. Only calibration logs are uploaded, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n unit Sol review of #592: - The manifest's vintage, its label and both workbook titles must agree (march_2026, "March 2026", "March 2026 Economic and fiscal outlook ... receipts"), at load and at ingest, so verified March bytes cannot be labelled October. - A row whose label states its own unit ("Standard colour licence fee (£)") is refused under a "£ billion unless otherwise stated" table, and the TV licence target reads the "BBC licence fee receipts" row only. - Ingest checks both workbooks and parses them in a staging folder before copying anything, and never replaces an ingested vintage. - Coverage checks every workbook target, including one without the vintage. - The NICs docstring no longer claims Class 3 or that partial results pass. All 29 OBR targets are still identical to main's on the March bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fits grouped targets; these checks measure what the groups hide, on the built dataset. Each PR build and its base build report person-weighted equivalised income medians, means and Gini, and relative low income for everyone, children, working-age adults and pensioners, against DWP's HBAI FYE 2025 estimates and 95% intervals (tables 8a and 8b, read from the published ODS files), at period 2024. They also report people by sex and single year of age against the ONS 2024-based projection at the calibration year. The comparison job adds them to the PR comment and flags a measure that leaves its interval; it fails only on a broken or mismatched report until the same-code spread is measured (#598). On uk-data 1.58.1, 5 of 14 HBAI measures are outside their intervals (median AHC, both Gini coefficients, child low income BHC and AHC), and single ages 0 to 79 are off by 19.5% on average. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first CI build of this branch failed the gate on three single-claimant UC payment bands (24,000 to 28,800 a year). Pull request (TESTING) builds hold fewer records than the release: across the final calibration logs of 12 recent ones, these three have no signal in 11 or all 12, while the released 1.58.1 build has signal in all three. A listed gap that applies to one kind of build only warns as stale in the other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…book The Scottish band targets came from a live gov.scot download. When gov.scot's CloudFront challenge blocked it, as in two of five recent pull request builds of the same code, the loader substituted a fixed 2025 total spread by fixed shares, so identical code calibrated to different targets (#596). Commit the September 2025 chargeable-dwellings workbook, verify its SHA-256, and fail if it is missing or changed. Its Scotland row equals the targets of the builds whose download succeeded. Fixes #596. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both builds are measured with policyengine-uk's aggregate-impact tool at a pinned commit (policyengine-uk#2425, stacked on #2335), under the head's locked model: receipts by tax head, spending and caseloads, poverty, inequality, deciles and the ten standard reform costings, 2025-26 to 2031-32. The comparison job runs `compare --axis data`, which fails unless both sides ran one model, runtime and set of definitions, and adds the result to the PR comment. It fails on a check the head breaks that the base passes. Only each side's result.json (aggregates) is uploaded; the tool's private household detail stays on the runner. Under the locked 2.122.2 the tool falls back to policyengine-uk main's partition, whose lists differ from that model's, so closure and two identities fail on both sides (pre-existing) until uk-data locks a model that ships its own partition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
obr.uk refuses scripted downloads, so on Budget day a person downloads the two EFO workbooks in a browser. The manifest now records each workbook's download time with its SHA-256 and size: the file's modification time, which a browser sets on saving, unless --downloaded-at is given. `ingest --dry-run` checks everything and prints titles, hashes, sizes, download times and every target's change, writing nothing; on the active vintage it says whether the bytes match the committed workbooks, so a March 2026 download rehearses the Budget-day path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sol review of #600: - Negative incomes before housing costs are reset to zero, and income after housing costs is derived from the adjusted figure (DWP HBAI methodology FYE 2025, "Negative incomes"). Weekly amounts use 365.25 / 7, as the FRS loader annualises them. - A report missing any reference measure, or with a missing or non-finite single-age summary, fails the comparison. - A missing distribution report fails the job. On uk-data 1.58.1, four of 14 measures are now outside their intervals (both Gini coefficients, child low income BHC and AHC); median AHC moves inside. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rint commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Raw calibration logs stay on the runner. Each build uploads a summary: whether every national and local target is within 10%, and the error of national targets supported by at least 10 households only (the constituency log gains a records column). An estimate from a few records, divided by a published count, could give one household's value back. The test job's raw-log uploads are replaced by the summary too. - The gate compares shares over the targets both builds have; added and removed targets are notes. - An empty national or local group, or a last epoch with fewer rows than the one before, fails. - The comment step writes a failure comment when the comparison never ran. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
UK hub: the fit-gate failure, broken down by target family (from the CI fit summaries, run 39040600634). All 11,228 common local targets are constituency targets. Shares within 10%, base against head:
The drop is concentrated in constituency self-employment amounts and working ages 30–70, while constituency employment amounts improve. It is about six times the same-code spread W3 measured, so it is not #598 noise. The national aggregate moves look consistent with the members' intent, though that isn't verified:
Next (hub): seeded leave-one-out builds in the Monday hold (main, the release, the release without #557, the release without #604) to attribute the local drop. Then a merge decision. This release stays a draft until then. |
Contributor
Author
|
Hub: release-only review at
Nits:
This stays a draft until the attribution builds report. |
This was referenced Oct 18, 2026
Closed
Contributor
Author
|
Hub, 10/19 00:40Z: attribution status, and a release blocker found before merge. The local fit drop. Seeded leave-one-out builds (#610–#617), all 32-epoch TESTING builds as in this PR's CI:
An independent investigation found no implementation defect in #571. Its prior raking of gain-bearing twins against the 12 CGT band targets shifts their pairs' mass at the start of calibration. Whether 512 epochs undo that is being measured in #618 (release) and #619 (−#571). The release blocker. At production settings (TESTING unset, as in the publish workflow), #595's coverage gate stops the release build. Three UC payment bands have no signal and aren't excused for release builds:
That's runs 39180561028 and 39180562951, with and without #571. Zero signal is decided before calibration, so the seed and the epoch count don't affect it. This PR needs a release-only fix before merge. It stays a draft. |
Contributor
Author
|
Hub, 01:05Z, update on the release blocker: the zero-signal set varies between release builds. A second production-setting run passed the three bands above, then lost |
Contributor
Author
|
Release-only fix pushed at The production workbook loader derives 24 UC bands starting at £24,000: six bands for each of SINGLE, LONE_PARENT, COUPLE_NO_CHILDREN and COUPLE_WITH_CHILDREN, through £30,000–infinity. The new #598 entry names 21 release-only zero-signal allowances, retaining the three existing COUPLE_NO_CHILDREN entries without duplication. Total release named zero-signal allowances: 38; testing retains its 17 named targets and existing pattern. The £24,000 cutoff uses SINGLE £24,000–£25,200 being zero in 20/21 testing builds and the varying release failures in runs 39180561028, 39180562951, 39181798638 and 39181880705. Bands retaining signal only warn as stale. This declares missing support; it does not restore it or resolve #598. Validation under the frozen lock (Python 3.13, policyengine-uk 2.122.2):
No dataset build, workflow dispatch, self-review, data-file push or merge ran. The body now has the release-only bullet and zero-signal limitation. The verified incremental recovery bundle and evidence are in Hub handoff: run independent final-head review and verify the gate in a production-setting CI run (TESTING unset). The existing #571 fit investigation and #598 nondeterminism remain open. #608 remains a draft. |
This was referenced Oct 19, 2026
Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
Author
|
Head 6ae9639: reworded the release-only zero_signal reason in known_gaps.yaml (review r2 P3) so it no longer says the cited runs share one commit; only #618's two runs (39180561028, 39181798638) are a controlled pair, and the PR body sentence matches. Wording only: test_target_coverage.py 30 passed, 1 skipped (dataset-dependent); ruff format clean. Still a draft, not merged. |
Contributor
Author
|
Merged by the UK hub (session local_8e19f9d5, Opus 5.5) on Max's d1621 (batch go, 2026-10-17) and d1678 = A (2026-10-19). It merged with a merge commit,
🤖 Generated with Claude Code |
Contributor
Author
|
UK hub, 22:50Z: the 1.59.0 publish failed; nothing was uploaded or tagged. Push run 39320323369 built at 512 epochs, then
Note: 🤖 Generated with Claude Code |
59 of 80 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR releases the third uk-data batch from
release/uk-data-2026-10-19, based on maina43e8c519(1.58.2): the 14 independently approved heads below, plus three release-only commits (21d5dbf9,cb92e401,6ae9639b).Max's go: d1621 (2026-10-17) for the batch, and d1678 = A (2026-10-19): publish with #571, merging past the failing 32-epoch CI fit gate.
Release decision and measured impact (UK hub, 2026-10-19)
The failing check. "Calibration fit against the base" fails at
6ae9639b(run 39194494219; comparison comment).CI builds with
TESTING=1, which trains for 32 epochs. The publish build (push.yaml,make data) sets noTESTING, so it trains for 512 (create_datasets.py:80).Cause. Seeded leave-one-out CI builds at 32 epochs (#610–#617) trace the drop to #571, mainly constituency self-employment amounts and ages 30–70.
Fit at production settings. These are seeded CI builds at production settings (
TESTING=0, 512 epochs), on diagnostic branches. Published 1.58.2's figures come from its own unseeded publish build (run 38741509997).cb92e401, #621)6ae9639bdiffers fromcb92e401only in one known-gaps entry'sreasontext; no target, status or build setting changes.cb92e401([DO NOT MERGE] Batch-3 gate check: fixed release cb92e401 at production settings #621). Its 17 stale named gaps only warn.Aggregates at production settings (policyengine-uk 2.122.2, £bn, 2025 and 2030). "Without #571" (#619) is the same seeded build with #571's merge reverted, on the release before
cb92e401.cb92e401leaves the fit and these aggregates unchanged: #618 and #621 agree. "#571's effect" is the difference between the two builds, and it also carries the build-to-build spread that #598 describes.Not measured here. No matched production-setting comparison of the whole batch against 1.58.2's aggregates was run.
Publication.
push.yamlbuilds the published files at production settings without a seed (#507 isn't in this batch). Published totals can therefore differ from the seeded figures above by the build-to-build spread (#598).Follow-ups:
Assembly record
The assembly job ran no dataset build and no production calibration. The hub's builds and measurements are in the decision section above.
Assembly-time CI at
21d5dbf9(superseded by the decision section above): Run 39040600634 completed: six checks passed; the final comparison check failed because the share of 11,228 common local targets within 10% fell from 79.4% to 75.1% (−4.3 percentage points), exceeding the 2-point allowance. National fit moved from 64.9% to 64.6% over 627 common targets. The comparison ran successfully and deliberately failed its regression gate; no missing artifact or test-integration error was reported. Distribution comparison passed with no flags; aggregate checks passed with zero newly broken checks, four failures present on both sides and 446 moves flagged for review. The cause of the local-fit change is unresolved, particularly given #598. Approved member computations and gate thresholds are unchanged. See the CI comparison comment.Members and merge order
Each member is merged with
git merge --no-ff, preserving its approved commits. Order: #604, #557, #595, #571, #589, #537, #561, #592, #594, #599, #600, #601, #602, #603. #601 is included as specified in~/reviews/uk-data-release-batch-2026-10/batch3-decision.mdand the 14-member d1621 approval. #599, #600 and #602 each receive a merge commit; #602 contains the first two.29ca7542~/reviews/uk-hub/reviews/ud604/review-r2-final.md473adafc~/reviews/uk-hub/reviews/ud557/review-r2-final.md0785612c~/reviews/uk-hub/reviews/ud595/review-delta-full.md77b6527b~/reviews/uk-hub/reviews/ud571/review-r2-full.mda2eedce4~/reviews/uk-hub/reviews/ud589/review-r2-final.mdf5de403e~/reviews/uk-hub/batch-reviews/ud537-r2/review-r2-full.md98b2583c~/reviews/uk-hub/batch-reviews/ud561-r2/review-r2-full.md133632af~/reviews/w3-uk-data-gates-2026-10-11/sol-delta-592b.mdcac704bb~/reviews/w3-uk-data-gates-2026-10-11/sol-delta-p3s.mddc81b55e~/reviews/w3-uk-data-gates-2026-10-11/sol-delta-p3s.md45d47428~/reviews/w3-uk-data-gates-2026-10-11/sol-delta-p3s.md0cf62aaa~/reviews/w3-uk-data-gates-2026-10-11/sol-review-600-601.md08226999~/reviews/w3-uk-data-gates-2026-10-11/sol-delta-602b-603c.md5881ce3e~/reviews/w3-uk-data-gates-2026-10-11/sol-delta-603d.mdAll member heads were checked against GitHub before merging. Main is unchanged from the approved base. #595’s approval at
6e03d4c1carries to0785612c: the hub verified that its additional commit changes only a YAML header comment and leaves parsed YAML identical (BATCH.md, 2026-10-16 10:30Z entry).Merge resolutions
All 14 members have been merged in the stated order with merge commits, and every approved head is verified as its merge’s second parent. Resolutions retain the approved members’ computations:
policyengine_uk_data/targets/build_loss_matrix.py:169retains Fail the calibration build on targets it drops or cannot fit #595’sselect_matrix_targets(emitted_targets())initialization for coverage statuses and duplicate tracking. It also retains Stop calibrating OBR total NICs on ni_employee #537’s extractedcalibration_targets()helper and restores itsget_all_targetsimport at line 23. Evidence:.job/remerge-537.diff.policyengine_uk_data/targets/sources/obr.py:214retains Stop calibrating OBR total NICs on ni_employee #537’s aggregate NICs exclusion with Build OBR targets only from hash-verified EFO workbooks #592’s manifest parser and NIC-class documentation.policyengine_uk_data/tests/test_obr_receipts_sheet.py:123retains Stop calibrating OBR total NICs on ni_employee #537’s absence/count assertions and removes Build OBR targets only from hash-verified EFO workbooks #592’s obsoleteload_configmonkeypatch. Build OBR targets only from hash-verified EFO workbooks #592’s deletion ofpolicyengine_uk_data/tests/test_obr_efo_fallback.pyis retained because verified local-workbook coverage replaces the obsolete network-fallback tests intest_obr_sources.py. Evidence:.job/remerge-592.diff.policyengine_uk_data/targets/source_manifests/obr_march_2026.yaml:33was regenerated with production_manifest_yaml, removing onlyobr/nifrom required targets as Build OBR targets only from hash-verified EFO workbooks #592’s body requires for Stop calibrating OBR total NICs on ni_employee #537 integration. Parsed YAML equals Build OBR targets only from hash-verified EFO workbooks #592’s manifest except that removal; every hash, provenance field and other value is identical. Evidence:.job/manifest-regeneration.json..github/workflows/pull_request.yaml:93retains Stop uploading raw calibration logs as public Actions artifacts #603’s runner-only raw-log explanation and the approved controlled fit, distribution and aggregate reports.policyengine_uk_data/tests/test_workflow_artifacts.py:88explicitly allows the six approved report files/producers and three existing comparison-commentgh apicommands. Full literal commands are matched; guard logic and wildcard permissions are unchanged. This is the integration step required by Stop uploading raw calibration logs as public Actions artifacts #603’s body. Evidence:.job/remerge-603.diff.Each conflict resolution was recorded with
git show --remerge-diff; details are in.job/RESOLUTIONS.md.Integration tests found one additional interaction, corrected in release-only commit
21d5dbf9:policyengine_uk_data/targets/build_loss_matrix.py:175now iterates over #595’s selectedall_targets, retaining #537’s helper for its invariant tests. The automatic merge had selected the helper’s list instead and bypassed the coverage gate’s selected/emitted list. The correction preserves #595’s approved first-name/level selection without changing target formulas or values. #537’s offline OBR mapping/conflict fixtures atpolicyengine_uk_data/tests/test_obr_target_conflicts.py:85andtest_obr_target_mapping.py:42now use #592’s verified committed-workbook API, retaining their assertions and blocking HTTP. The workbook-target expectations intest_obr_sources.py:68and:367change from 25 to 24 because #537 removesobr/ni.Generated-file checks reproduce approved bytes: the production SPI generator reproduces
incomes_projection.csvfrom committed public inputs, SHA25639445e49c9cb101fffd99185cf41d94a78d6c5a9fd93c99d8ea027593a7ea475, identical to #557. The production ONS generator reproduces the 80+ CSV from four existing pinned public ZIPs, identical to #604’s committed bytes. The OBR manifest has the semantic parity described above. No generated-file diff remains.Release-only changes
cb92e401extends Fail the calibration build on targets it drops or cannot fit #595's release-buildzero_signalknown gaps for Builds of the same code calibrate to two different weight sets #598 to all 24 UC payment-distribution bands starting at £24,000 across the workbook's four family types: 21 newly named targets and the three existing COUPLE_NO_CHILDREN allowances. SINGLE £24,000–£25,200 was zero in 20 of 21 testing builds. Production runs 39180561028, 39180562951, 39181798638 and 39181880705 lost different high-payment bands between release builds. Runs 39180561028 and 39181798638 ([DO NOT MERGE] Batch-3 fit attribution: seeded release at production settings (512 epochs) #618) differ only in known-gap allowances; the other two used different code. The workbook-derived regression checks every family/band, keeps release coverage strict below £24,000, and preserves testing's existing pattern. This fix names 38 release zero-signal allowances in total; the 17-target counts recorded below describe the prior assembly head.Release notes and limitations
..for each family), which the loader skips because it overlaps the finer bands. The release coverage gate passed at production settings ([DO NOT MERGE] Batch-3 gate check: fixed release cb92e401 at production settings #621).The limitation wording and release requirements come from
~/reviews/uk-data-release-batch-2026-10/BATCH.mdandbatch3-decision.md. The annual allowance limitation above is reproduced verbatim from BATCH.md.Validation
Release-only UC known-gap fix
cb92e401: locked Python 3.13 suite (test_target_coverage.py, including all coverage-gate tests;test_target_registry.py;test_uc_payment_distribution_targets.py) with four workers: 57 passed, 1 dataset-dependent skip.uvx ruff format --check .(295 files),uv lock --check, typed changelog validation andgit diff --checkpass. Release named allowances are now 38 zero_signal / 3 duplicate / 15 no_value_for_year; testing keeps 17 named zero_signal plus its UC pattern. No dataset build ran. The hub's production-setting gate check passed ([DO NOT MERGE] Batch-3 gate check: fixed release cb92e401 at production settings #621), and the independent review ofcb92e401approved it with one wording nit, folded in at6ae9639b.All 14 approved heads checked before assembly. Main refreshed before publication and remains
a43e8c519(1.58.2).uv lock --checkpasses with Python 3.13; locked and installedpolicyengine-ukis 2.122.2. The only lock refresh updates this project's stale version from 1.58.0 to 1.58.2; all dependency versions are unchanged.Every member's own tests ran in one foreground suite with four workers. The integration corrections pass a targeted rerun: 48 passed, 1 dataset-dependent skip.
Standard
make test, four workers with one numerical thread each and frozen public-input replay: 2,532 passed, 134 skipped, 1 expected failure. Missing datasets account for 132 skips; the other skips are the known NISRA endpoint and optional bundle tooling. No dataset was downloaded or built locally, and no production calibration ran.Fit/distribution tests with
-W error::RuntimeWarning: 41 passed. Artifact guard: 93 passed. Coverage gate: 29 passed, 1 dataset-dependent skip.Combined-tree named known gaps reproduce zero_signal 17, duplicates 3, and no_value_for_year 15. A fresh source-registry replay independently confirms exactly 3 duplicate names, with live HTTP disabled. Actual combined-matrix signal measurement remains pending the hub's seeded build.
SPI projection regenerated with the production generator from committed public aggregates: SHA-256
39445e49c9cb101fffd99185cf41d94a78d6c5a9fd93c99d8ea027593a7ea475, exactly the approved member output. ONS 80+ CSV regenerated from existing pinned public sources and is byte-identical. OBR manifest regenerated with every field unchanged except removing Stop calibrating OBR total NICs on ni_employee #537'sobr/nirequirement.uvx ruff format --check .: 294 files already formatted. Changelog-fragment check againstorigin/main, andgit diff --check: pass.Draft release PR CI completed: six checks pass; calibration-fit comparison fails on the local-fit regression described above. Both CI builds, repository tests (2,662 passed, 4 skipped, 1 expected failure) and all report producers passed. The built-data coverage test passed, though CI logs do not print actual zero-signal/duplicate counts. The completed logs and generated comparison comment were read; no integration-only fix is indicated. The hub must investigate the measured regression in the pending matched-seed build.
Reproducer: policyengine_uk_data/tests/test_obr_universal_credit_target.py::test_target_matrix_counts_gb_households_only
axiom: n/a: data release integration with no policy rule changes
Reviews.
21d5dbf9was APPROVE WITH NITS (comment).cb92e401was APPROVE WITH NITS (~/reviews/uk-hub/reviews/ud608/review-r2-full.md). Its one wording nit is folded in at6ae9639b.reasontext inknown_gaps.yaml.🤖 Generated with Claude Code