Repository navigation
Split total income into programme-specific concepts - #2620
Merged
Merged
Conversation
Retain total_income as the tax-law total; count statutory pay and maintenance in non-dependant gross income while preserving household and UBI input bases. Replace HB tax-attribution metadata coupling with explicit components and add legal YAML and random-input regressions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Household income totals
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileMean change in household net income per household, and as a share of the decile's base income; households stay in the base run's deciles.
No change in any year: receipts by tax head; population; poverty rates; people in poverty. Checks failing on both base and head (1)
Counted-once checkEach of 88 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
Exercise all three statutory payments in the deprecated WFP eligibility reform; clarify legal input limits and preserve the measured seven-year aggregate report. All formulas are unchanged from the fully validated implementation head. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Hand-off at Validated source 7bdd785: 8,681 Python passed / 61 allowed skipped / 2 xfailed; 4,488 YAML passed; lint, docs, Python 3.11–3.14 smoke imports, Test/Result and Aggregate impact passed. Local 3 Python properties and 9 WFP YAML passed; no local microsimulation/data build. 2025-26 combined with #2066: Income Tax +£293.298m; WFP spending +£175.420m and +783,692 weighted recipients; HB spending -£0.269m; CTR spending/caseload unchanged; government balance +£118.146m excluding suppressed DSA. Seven-year table is in RESULT.md/PR body. Aggregate impact run 40014615941 passed with no newly broken checks; one pre-existing negative-VAT check persists. These figures include the required #2066 merge prerequisite. The final commit adds locally passing tests and documentation only; fresh CI is queued/running.
|
The report is a job artifact, not part of the change; the UK hub keeps a copy outside the repository. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Apply the 16-hour threshold and the IS/JSA(IB)/ESA(IR) exclusions, using the existing personal receipt flags and #2007 parameter files. Retain existing rates, eligibility, couple rules and band edges. Add 2026 regressions and update fixtures for the flat non-worker amount. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Give the vector policy tests an age array first so the default builder creates the intended number of people. Update the two existing CTR severe-premium snapshots for the inherited flat non-worker deductions. No policy formula or parameter changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Expand the regression matrix into single-person YAML cases because Core infers population size from the first input and does not broadcast scalars. Keep the same inputs and legal expected amounts in each scenario. No policy formula or parameter changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
UK hub pushed
Measured impact (Aggregate impact run 40029283812, the full PR including #2066, certified enhanced FRS 2024-25): HB spending falls about £6m a year (−£6.0m in 2025-26 to −£7.1m in 2030-31), and CTR spending falls about £1.7m a year. The report shows 0 newly broken checks. A delta review of the gate and fixtures at this head is under way. This PR lands after #2066, which is in the merge queue. |
The independent delta review of 404daf7 found a P1. The work gate charged the flat £20.40 a week to every non-worker. Before the gate, a non-worker with no banding income got the scale's zero, because calc(0, right=True) hits the zero sentinel. Many such non-dependants are exempt by law, and the model can identify them: - the non-dependants of an Attendance Allowance claimant (HB reg 74(6), reg 55(6), CTR Sch 1 para 8(6) and equivalents); - for CTR, a non-dependant on IS, JSA(IB), ESA(IR) or Pension Credit, or on UC without earnings (para 8(8)(a), (c)); - for HB, a non-dependant under 25 on IS (reg 74(8)). Measured alone, the gate cut HB about £6m a year and CTR about £1.7m. A non-worker's deduction is now the lower of the banded amount and the flat amount. At positive banding income, including pension income, that is the flat amount, as reg 74(1)(b) and its equivalents require. At nil income it is still zero, until #2007 models the exemptions together with the flat charge. The six snapshots the gate moved for nil-income non-workers are back to their pre-gate values. These are the two pension-age AA claimant cases, the two CTR claimant cases, and the two investment-income non-worker cases. The two new files' nil-income cases now assert zero. Two guards are added, an HB non-dependant under 25 on IS and a CTR non-dependant on UC without earnings, both at nil income. The CTR IS, JSA(IB) and ESA(IR) cases note that para 8(8)(a) exempts them; that exemption stays for #2007. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Pushed
The fix takes review option (a): the gate only ever lowers a deduction. Non-workers get Fixtures:
PR body: updated for the P2. It reports the gate-only decomposition from runs 40014615941 and 40029283812 (HB −£5.7m to −£6.8m a year, CTR −£1.6m to −£1.8m; almost all of it was the nil-income charge this commit removes). It marks those figures as superseded until Aggregate impact run 40040246899 at this head finishes, and lists the round 2 P3s (band edges, couples, the NI reference, merge order with #2007) as #2007 follow-ups. Static checks: |
Main (e6f8bb1) released the winter fuel payment charge, which this branch had merged as a prerequisite. The four conflicts: - changelog.d/winter-fuel-payment-charge.added.md: main released and removed it; removed here too. This branch's own fragment already describes its total_income change. - _net_earnings.py, housing_benefit_net_earnings.py and total_income_charged_to_tax.py: documentation only. This branch's text, written on top of the charge's, is kept; the code merged cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
Main's new 2018 case (6b355c7) set total_income, which the split HB and CTR non-dependant bases no longer read, so both deductions came out at £0 instead of £793. It now supplies £5,200 of employment income (£100 a week) and 16 hours a week. The expected £15.25 x 52 = £793 is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Merged by the UK hub (Claude session 8e19f9d5) at head
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2619
total_incomehad a tax-law label but summed raw income inputs and served several different programmes. This omitted statutory pay from WFP's old income test and from HB/CTR non-dependant bands. HB and CTR now apply those bands only to non-dependants in remunerative work; other eligible non-dependants with banding income pay the flat amount instead of a band. The gate never raises a deduction. Retain the exported name with the Income Tax definition, and give each other consumer an explicit income base.Invariants
total_incomedelegates through nonemptyaddsmetadata tototal_income_charged_to_tax: ITA 2007 s.23 Step 1, interpreted by s.989 and s.31. The tax component sum is before general-income loss relief; carried-forward trading loss relief already deducted inside a component is restored, and net-pay pension relief held separately by the model is deducted. RAS pension relief and Gift Aid do not lower total income. Existing component limitations, including netted miscellaneous losses and deficiency relief deducted from dividends, remain documented.employment_benefits. HB earnings remains algebraically identical to its old calculation, with an explicit positive-component tax-apportionment denominator.gross_taxable_inputs, including its exclusion of statutory pay. This is an explicit modelling/design choice, not a legal definition. Main already contains Charge the winter fuel payment back through income tax, person by person #2066's WFP charge. This PR changes its input fromtotal_income_charged_to_taxto the equivalenttotal_incomealias, preserving the current-law charge calculation.Consumers: old → new
total_incomeadjusted_net_income_componentsbefore Step 2 reliefs as represented by the model. ITA 2007 s.23, ss.31,989.total_income; its released charge already readstotal_income_charged_to_taxtotal_incomeand switches the charge to that equivalent alias, preserving its individual £35,000 threshold and separate entitlement. ITEPA 2003 s.681I.total_income − other_investment_income + max(spousal maintenance,0)housing_benefit_non_dep_gross_income: same represented income plus statutory pay, positive child maintenance, MA, IIDB, SDA and war widow's periodic pension; disability/compensation disregards documented. HB reg74, pension-age reg55.council_tax_reduction_non_dep_gross_income; same currently representable sources, with separate national disregard references. England Sch1 para8, Scotland working-age reg90, Scotland pension-age reg48, Wales pension-age Sch1 para3, Wales working-age Sch6 para5.total_incomeand itsadds, plus statutory paypost_tax_incometotal_income − income_tax − national_insurancegross_taxable_inputs − income_tax − national_insurance, preserving the old input base. Household measure, not law.total_incomegross_taxable_inputs, preserving the design choice.The non-dependant gross concepts retain their existing other-investment-income exclusion; extending this income base remains for #2007. The deduction consumers now apply the remunerative-work gate below. Annual statutory pay may coexist with work in another part of the year; leave itself does not establish remunerative work (HB reg6(7)). Signed trading/property inputs are retained; the band provisions do not directly settle cross-source offsets, and this refactor does not introduce a separate loss-offset rule. Existing national CTR consumers still use the HB deduction scale. Benefit-unit awards cannot currently be attributed safely to a person here (adding computed UC creates an HB/benefit-cap cycle); mixed periodic/capital BSP and AFCS inputs and unrepresented compensation payments remain explicit modelling limits. This PR does not claim complete regulatory coverage of those inputs.
Compatibility
Read-only grep of sibling repositories:
policyengine.py/src/policyengine/tax_benefit_models/uk/model.py:72exports the name;tests/test_models.py:261–269requires nonemptyadds(preserved); four UK household-calculator snapshots contain output keys.policyengine-api: no tracked exact-word references.policyengine-app-v2/app/src/mocks/UK_Metadata.json: variable/module metadata at 9717–9743, generated abolition metadata at 36677–36681, namespace at 44307–44310. Module namegov.hmrc.income_tax.total_incomeremains valid.Wales Childcare Offer #2535 is now merged into main. Its gross-income test uses explicit person-level
income.gross_income_sourcesplus the parent's share ofincome.benunit_gross_income_sources; it does not usetotal_incomeand counts statutory pay once throughemployment_benefits. This PR leaves those formulas, parameters and tests unchanged.Callers supplying an income override for household post-tax income or UBI phase-outs should now supply
gross_taxable_inputs; an explicittotal_incomeoverride is a tax-total override.Remunerative-work gate and review response
Answers the P2 in the hub's independent review
uk2620/REVIEW_2620.md, at reviewed head20d873b6e8c4f7ae5273281a64280dcb3da6385b: newly counted IIDB and maintenance must not put non-workers into higher income bands.Also answers the P1 in round 2 (
uk2620-r2/REVIEW.md, at404daf7d1): the first version of the gate (6df0e6f65) charged the flat amount to non-workers with nil banding income, where the old lookup gave zero. Much of that group is exempt in law (a claimant on Attendance Allowance, PIP daily living or similar; CTR non-dependants on IS, JSA(IB), ESA(IR), SPC or UC without earnings; HB non-dependants on SPC, or under 25 on IS, JSA(IB) or ESA(IR) without the support component, or on UC without earnings; full-time students), and none of those exemptions is modelled yet.cf5b3b530makes the gate only ever lower a deduction.weekly_hours(hours_worked / 52) against the programme/nation'sremunerative_work_hoursparameter, currently 16. Exclude people on IS, income-based JSA or income-related ESA using existing person-levelis_on_*receipt flags. HB reg 6(1), (6), pension-age reg 6 and the corresponding CTR work definitions support this gate. Annual status/hours inputs cannot represent the regulations' within-week receipt or fluctuating-hours detail.min_(p.amount.calc(weekly_income, right=True), p.amount.amounts[0]). A non-worker with banding income pays the flat amount; one with nil banding income keeps the scale's zero until Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007 adds the exemptions and the flat charge together. Keepright=Truefor workers' existing band edges.4a0935b737cd71231e39340689b2148125f585c9: HBgov/dwp/housing_benefit/non_dep_deduction/remunerative_work_hours.yaml; CTRgov/local_authorities/{england/council_tax_reduction/pensioners,scotland/council_tax_reduction,wales/council_tax_reduction}/non_dep_deduction/remunerative_work_hours.yaml. This is only the work gate. Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007 retains the full implementation: national CTR scales, couples, exemptions and inclusive band edges. This branch's existing receipt flags replace the older award-payee helper used there.Existing non-workers with pension or other banding income also move from higher bands to the flat amount. This lowers their deductions as the law requires; it is wider than protecting newly counted IIDB/maintenance. The gate never raises a deduction: a non-worker with nil banding income still pays nothing, as before this PR. (The wider income base can still move a non-worker from nil banding income to the flat amount, for example one whose only income is IIDB; reg 74(1)(b) charges that, subject to the unmodelled exemptions.) Charging the flat amount to the non-exempt part of that group needs the exemptions first, which is #2007's scope. Inherited gaps in claimant, student and couple rules remain for #2007; this patch does not port them.
The 2026-27 HB table gives a £20.40 weekly flat amount. A non-worker with £12,000 IIDB pays £1,060.80 annually instead of £2,436.20. At 16 hours, £9,000 wages plus £12,000 IIDB or child maintenance is £403.85 weekly and pays £105.20 × 52 = £5,470.40; a 15-hour worker pays £1,060.80. National CTR tables were checked: flat amounts are England £5.20, Scotland £5.55 and Wales £6.70. This gate-only patch retains the inherited HB scale in the national CTR consumer, so its regression amounts intentionally match HB; adopting the correct national scales belongs to #2007.
Validation
*_non_dep_remunerative_work.yamlfiles: IIDB and child maintenance at 0/15/16/37.5 hours, each IS/JSA(IB)/ESA(IR) exclusion, non-working pension/nil income, and programme/national parameter overrides. CTR covers England, Scotland and Wales. The matrix has 54 cases (17 HB and 37 CTR). The nil-income cases assert £0, and two guards cover exempt groups at nil banding income: an HB non-dependant aged 22 on IS, and a CTR non-dependant whose benefit unit gets UC without earnings. The CTR IS/JSA(IB)/ESA(IR) cases note that SI 2012/2885 Sch 1 para 8(8)(a) gives £0; with banding income the model still charges them the flat amount, an inherited gap left for Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007.69f815b87). The investment-income, claimant-routing and severe-premium snapshots, whose non-dependants have nil banding income, keep their pre-gate values (restored atcf5b3b530; parsed and compared with20d873b6e: no output differs).uvx ruff format --check .: passed (1,998 files). Ruff lint on changed formula directories/files: passed.git diff --check: passed. Static duplicate-key YAML parsing, parameter equality to Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007 and Python AST parsing: passed. No model code was executed locally.69f815b87: all 18 checks pass. Code run 40155178579: YAML 4,809 passed, 0 failed; Python 10,037 passed, 66 skipped, 2 xfailed, 0 failed. The first run at the merge5877066a8(run 40151657406) failed one YAML case: main's 2018 uplift case still settotal_income, which the split bases no longer read;69f815b87fixes that fixture. The earlier green CI atcf5b3b530(run 40040246952) predates the main merge.57d2f5f63originally merged Charge the winter fuel payment back through income tax, person by person #2066 at76108fafe. Charge the winter fuel payment back through income tax, person by person #2066 has since landed in main. Merge5877066a8brings maine6f8bb100into approvedcf5b3b530. Its four conflicts were resolved by deleting the released WFP changelog fragment and retaining this branch's three docstrings. No formula conflict required resolution.Reproducer: policyengine_uk/tests/test_income_concepts_properties.py::test_statutory_pay_is_counted_once_in_each_programme_concept, policyengine_uk/tests/policy/baseline/finance/benefit/family/housing_benefit/non_dep_deduction/housing_benefit_non_dep_remunerative_work.yaml, policyengine_uk/tests/policy/baseline/gov/local_authorities/council_tax_reduction/council_tax_reduction_non_dep_remunerative_work.yaml
Aggregate impact figures
Current head. Aggregate impact run 40155178547 compares main
7e24c18626with this PR merged into it (e4fac15a4d), both on certifiedenhanced_frs_2024_25.h5(policyengine-uk-data 1.58.1, HF revision2526edbbb687a3b54dbb11272e5389e4b62d2ac2, SHA-256056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784) and uk 2.157.1. #2066 is in main on both sides, so these are this PR's own changes.HB caseload does not change. WFP spending and Income Tax are exactly unchanged, as are every other receipt head, population, poverty rates and people in poverty, in every year: pensioner relative poverty after housing costs in 2031–32, for example, is 12.7926999105% on both sides. 0 newly broken checks: 351 pass, 19 warnings, one fails on both sides (
sign:vat, VAT is never negative) and the DSA check is skipped. No move is flagged for review. The counted-once check (88 components, each raised by £1) has 0 failures.Historical figures, before #2066 landed in main. The figures below are from run 40040246899, which measured
cf5b3b530against pre-#2066 main. Its WFP and Income Tax movements belong to #2066 and are not this PR's effect. Aggregate impact run 40040246899 measuredcf5b3b530, merged into mainf66f0a205f4c694c1d69b2199409f571f58a314eas GitHub merge heade231b111d528a05a2d07a96df7f6c8a76e0886aa. Both sides used certifiedenhanced_frs_2024_25.h5, policyengine-uk-data 1.58.1, HF revision2526edbbb687a3b54dbb11272e5389e4b62d2ac2, SHA-256056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784. These are full-PR changes against that main. They include the #2066 prerequisite (the Winter Fuel Payment charge, which accounts for the WFP and Income Tax rows) and the other income-definition changes.Historical run 40040246899 only: 0 newly broken checks; 1 failing on both sides (
sign:vat, VAT is never negative); 0 fixed. The report records 344 passing checks, 19 warnings and one skipped check. Runtime differences and structural failures are empty. It flags 27 moves for independent review: WFP spending and caseload, Income Tax, decile mean incomes, and pensioner relative poverty after housing costs in 2031–32. None is an HB or CTR flag.Historical #2066 attribution. In run 40040246899, against pre-#2066 main, pensioner relative poverty after housing costs in 2031–32 rose by 0.37pp. #2066's own Aggregate impact run at
3a3800e24(run 40046674016), without this PR, gave the same base and head rate and count to every digit, so that move came from #2066, which is now in main. These historical runs do not measure the current head. In current run 40155178547, every poverty rate and every count of people in poverty is unchanged in every year, including 2031–32 pensioner relative poverty after housing costs at 12.7926999105% on both sides.Gate-only effect (historical). Runs 40014615941 (
7bdd78509) and 40040246899 (cf5b3b530) had matching HB/CTR baselines in 2025–26 and 2026–27 only, so their delta difference isolates the gate for those two historical years; this is not a new comparison at the current head. Run 40014615941 measured the pre-gate source7bdd78509, which is formula-identical to20d873b6e(only three docstrings differ), against mainc417f283a. Its baseline HB and CTR spending and caseloads equal run 40040246899's in 2025–26 and 2026–27. From 2027–28 they differ, because main changed between the two runs: baseline HB moves by £29m to £44m a year. So the difference of the two PR deltas isolates the gate in the first two years only:The pre-gate PR delta was −£0.27m to −£0.32m a year for HB and 0 for CTR: the wider income base raised some non-workers' bands (round 1's P2). The gate only ever lowers a non-dependant deduction. In those two historical years it raised HB by about £1.4m a year with no change in HB caseload, and gave about 925 more benefit units some CTR. The superseded gate at
6df0e6f65also charged nil-income non-workers the flat amount (HB −£6.0m to −£7.1m a year in run 40029283812);cf5b3b530removed that charge.† Household net income excludes the suppressed Disabled Students’ Allowance figures on both sides. The linked reports include tax receipts, WFP, poverty, deciles and reform costings. No local microsimulation was run; every figure here comes from the CI Aggregate impact runs linked above.
Known follow-ups (not in this PR)
From round 2, all inherited and in #2007's scope:
right=True, a worker at exactly £192.00 a week gets £20.40, where reg 74(2)(b) gives £46.85 (every edge is the same), and a remunerative worker with nil or negative banding income gets £0, where reg 74(2)(a) gives the flat amount. Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007 usescalc(weekly_income)andcalc(0).remunerative_work_hours.yamlcites the GB regulations only; NISR 2006/405 reg 6 has the same 16 hours and the same reg 6(6) exclusion. The file is byte-identical to Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007's, so the reference goes in there.git merge-treegives six content conflicts: the two gate formula files and four YAML files. The four parameter files merge cleanly. Whichever PR lands second reconciles them; Apply Housing Benefit and national Council Tax Reduction non-dependant deductions as legislated #2007 builds its income-related-benefit test from its ownis_award_payeehelper.axiom: TheAxiomFoundation/rulespec-uk#407 queued
(The UK hub extended #407 with the non-dependant gross-income basket, the remunerative-work test and the 54 work-gate cases, and #420 with s.681I's use of the ITA 2007 s.23 Step 1 total, statutory pay counted once. Each lists this PR's known deviations from the law so they are not encoded. Every quoted block was checked against a fresh fetch.)
🤖 Generated with Claude Code