Repository navigation
Add household income deciles after housing costs - #2629
Conversation
vahid-ahmadi
left a comment
There was a problem hiding this comment.
@juaristi22 thanks, first-round review at f315e2585. This matches HBAI's decile definition and the existing BHC variable, and the tests pin it. No must or should; two nits, neither needs to hold this up.
What I checked
- HBAI definition, against the HBAI background information and methodology report FYE 2025. Decile groups are ten equal-sized groups of the whole population "when ranked by household income", and "the unit of analysis is the individual", with every member of a household at the same point in the distribution. So the right construction is equivalised household income, AHC basis, with each household counted once per person.
household_income_decile_ahc.py:17-22does exactly that: it ranksequiv_hbai_household_net_income_ahc(which divides byhousehold_equivalisation_ahc, the AHC scale) with weights ofhousehold_weight * household_count_people. Thereferenceanchor (#deciles-and-quintiles) resolves. - Consistency with the BHC variable:
household_income_decile.pyuses the same weights and the sameMicroSeries.decile_rank(), so ties share the cumulative rank at the top of their group in both. The only difference is the explicitmax_(..., 0), which BHC doesn't need because its input is already reset. Consistent. - Hand check of the YAML case. Person masses are 2, 1 and 2 (total 5). BHC order 1000, 2000, 3000 gives cumulative shares 0.4, 0.6, 1.0, so deciles 4, 6, 10. AHC order 1000, 2000, 3000 sits on households 2, 3, 1 with masses 1, 2, 2, so 0.2, 0.6, 1.0, deciles 2, 6, 10. Both match the expected outputs, and the case does separate AHC from BHC and person from household weighting.
- Tests in a fresh venv: the 5 new Python tests and the 39 YAML cases under
tests/policy/baseline/household/income/pass, and the 4 BHC decile tests still pass. Withhousehold_income_decile_ahc.pyremoved, all 5 new Python tests and the new YAML case fail and the 4 BHC tests pass, so the tests pin the change.ruff checkandruff format --checkare clean on the new Python. - PR body:
Fixes #2600,Reproducer:andaxiom: n/a: ...are present; n/a fits a ranking statistic.changelog.d/2600.added.mdis present. - CI at head: Lint, Fix evidence, Test / YAML, the docs builds and the four smoke imports pass. The five Python test shards and Aggregate impact were still running after more than half an hour when I posted, so this approval assumes they finish green; if one fails I will look again.
- Not run: anything on survey data, so I haven't measured how many people have AHC income at or below zero.
Findings
-
nit: the reset to zero for ranking is not something HBAI does on the AHC basis. The methodology report says negative AHC incomes "are possible" and keeps them. In practice the two give the same groups as long as people at or below zero AHC make up at most a tenth of the population: with the reset they all share decile 1, and without it they'd all be in decile 1 anyway. Only if that group grew past 10% (say, under a large reform to housing support) would the reset push them all up together. To be fair to you, the reset is what #2600 asked for, so this one is on the issue, not the PR. The docs paragraph already says it is a model convention and not a claim to match DWP's groups, which is enough. No change needed unless you'd rather drop the
max_and match HBAI exactly. -
nit: there is a person-level
income_decilethat reads the BHC household decile (income_decile.py:14), but no person-level AHC counterpart. Not needed for #2600, which asks for the household variable; only mention it in case person-level tables are on your list.
Note: the PR is still a draft. Happy for it to go ready as it is.
🤖 Generated with Claude Code
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles. Checks failing on both base and head (1)
Counted-once checkEach of 86 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
Fixes #2600
Add
household_income_decile_ahc, ranked by equivalised household net income after housing costs. It uses household survey weight multiplied by household size, following the existing before-housing-costs decile conventions. Negative values are reset to zero for ranking, and tied values share their group’s upper cumulative rank.AHC income and poverty measures retain their negative values. The HBAI documentation explains this ranking convention and the population on which a decile is calculated.
axiom: n/a: adds a distributional statistic; no statutory rule, rate or entitlement changes.
Reproducer: policyengine_uk/tests/test_household_income_decile_ahc.py::test_household_income_decile_ahc_weights_people_and_survey_weights
Validation: 1 YAML integration case and 9 focused Python tests passed, including 4 unchanged BHC controls. The integration case distinguishes AHC from BHC ranks and person from household weighting. The original missing-feature failures are retained as TDD evidence.
make formatand its Ruff check passed before the commit.Independent local code/test and policy/source review: COMPLETE/APPROVE at
f315e258, with no findings or material gaps. The code reviewer rebound the retained passing tests to the committed file hashes without rerunning them. No private dataset calculation, full-suite run or national distributional estimate is claimed. The small Markdown documentation update has not had a documentation build.