Repository navigation
Count Missouri's self-employment tax deduction net of the federal AGI deduction - #10635
Conversation
… deduction RSMo 143.141(4)(c) allows the self-employment tax as a Missouri itemized deduction only to the extent it was not deducted in computing federal AGI, and the 2017 MO-A Part 2 line 7 instructions take Form 1040 line 57 minus line 27. The itemized source list added the full tax, so filers with self-employment income got the deductible half twice. Add mo_self_employment_tax_addback (tax less self_employment_tax_ald, floored at zero) and use it in the list for every year it covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HbXwfKTi295BXtxZU
|
Ready for hub review Head: What the item asked and how it's resolved
Hand computation14,129.55 × 0.5 = 7,064.77 deducted in federal AGI (the model's ALD), so the addback is 14,129.55 − 7,064.77 = 7,064.77 (rounding). Missouri itemized = 20,000 + 7,064.77 + state-income-tax items that net to zero = 27,064.78 (the model's float sum). The old 34,129.55 minus the new 27,064.78 equals the ALD, 7,064.77. Tests (one file per process)
Mutants (each in its own process, code restored after)
CIThis is a draft, so CI has not run. Judgment calls and limits
Proposed follow-up issueLeave dependents' FICA and self-employment tax out of Missouri itemized deductions Generated by Claude Code |
|
Hub review in progress Head under review: So far:
Generated by Claude Code |
|
Ready for hub review The head hasn't moved past FindingsP3-1. The witness comment's cents don't add up
Replacement for lines 116-119: # Single filer, $100,000 of self-employment income and $20,000 of cash
# charity. Self-employment tax 14,129.55, of which half (7,064.775) is
# deducted in federal AGI, so RSMo 143.141(4)(c) adds back the other
# 7,064.775: 27,064.775 in all. Before the fix the full tax was added: 34,129.55.P3-2. No test pins dependents' self-employment tax
Witness, 2024. Head: $100,000 of self-employment income and $20,000 of charity. Dependent, age 17: $20,000 of self-employment income.
The two columns differ by exactly the head's ALD. On both, the dependent's 2,825.91 is added in full. P3-3. "Addback" names a deduction itemOn Form MO-A, "additions" are Part 1 income additions, but MutantsEach mutant ran in its own process on the PR tree, and I restored the tree after each one.
Checked and sound
What I couldn't check
Generated by Claude Code |
The tax is 14,129.55, so the deductible half and the remaining half are 7,064.775 each, and itemized deductions come to 27,064.775 with the $20,000 of charity. The test's 0.01 margin accepts either rounding. Also name the actor in the other comments and the changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Hub impact pair, progress The base side of the 2026 impact pair finished. The head side runs next on the same eight household chunks.
Generated by Claude Code |
|
Ready for hub review The PR's head hasn't moved from Dataset. Both sides used Year and chunks. 2026, with 8 household chunks through #10561's head moved to
The table rounds What moves. Five of the seven variables move. Every row that moves belongs to a Missouri household (state FIPS 29 in the dataset).
One record carries the tax change. Tax unit 1020906 (household 1015769) holds 9,248.99 of the 9,249.67 weight across the 23 tax units whose tax moves. Its Missouri itemized deductions fall by $348.56, and its Why, from the code I read.
Neither side came near 14 GB, the peak above which the hub asks for a 16-chunk rerun, so I didn't rerun. Dropped variables. None. All seven variables exist on both sides on the same entity, and the helper accepted each one. Commands. I ran these in order. I replaced absolute paths with relative ones or a variable, and left out curl -sI https://huggingface.co/api/datasets/policyengine/populace-us | head -1
git clone https://github.com/PolicyEngine/policyengine-us.git pe && cd pe
git fetch origin pull/10635/head:pr-10635 pull/10561/head:pr-10561
HEAD_SHA=$(git rev-parse pr-10635); BASE_SHA=$(git merge-base origin/main $HEAD_SHA); echo $HEAD_SHA $BASE_SHA
git worktree add --detach ../head $HEAD_SHA
git worktree add --detach ../base $BASE_SHA
git show pr-10561:policyengine_us/tools/calculate_chunked.py > ../calculate_chunked.py
# pr-10561 was c545c478 then. To get the same helper now, use
# git show c545c47877c70b367a6fab2322037e453364997e:policyengine_us/tools/calculate_chunked.py
git diff $BASE_SHA $HEAD_SHA -- pyproject.toml uv.lock # empty
grep -n "DEFAULT_DATASET" ../base/policyengine_us/system.py ../head/policyengine_us/system.py
cmp ../base/policyengine_us/system.py ../head/policyengine_us/system.py # identical
cd ../head && uv sync
# Fetch the default dataset into the Hugging Face cache with the call the model
# makes, so that neither side's wall time includes the download.
mkdir -p ../scripts_tmp
# Save pair_side.py and compare.py next to pe/, and smoke.py and checks.py in scripts_tmp/.
cat > ../scripts_tmp/fetch.py <<'EOF'
from huggingface_hub import hf_hub_download
p = hf_hub_download(repo_id="policyengine/populace-us", filename="populace_us_2024.h5", repo_type="dataset", revision="populace-us-2024-spm-20260909")
print(p)
EOF
F=$(.venv/bin/python ../scripts_tmp/fetch.py) # downloads, then prints the cached file's path
sha256sum "$F"
# The VM had no /usr/bin/time, and the first install needed an apt-get update.
apt-get install -y -q time || (apt-get update -q && apt-get install -y -q time)
# Smoke run: the first 1,500 households in 2 chunks, with the head's code.
cd ../pe
export OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 MKL_NUM_THREADS=4
PYTHONPATH=$PWD/../head /usr/bin/time -v $PWD/../head/.venv/bin/python ../scripts_tmp/smoke.py $PWD/../calculate_chunked.py 1500 > ../smoke_head.log 2> ../smoke_head.err
mkdir -p ../out
H=$PWD/../calculate_chunked.py; PY=$PWD/../head/.venv/bin/python
PYTHONPATH=$PWD/../base /usr/bin/time -v $PY ../pair_side.py $H 8 2026 ../out/base > ../out/base.log 2> ../out/base.err
PYTHONPATH=$PWD/../head /usr/bin/time -v $PY ../pair_side.py $H 8 2026 ../out/head > ../out/head.log 2> ../out/head.err
$PY ../compare.py ../out/base ../out/head ../out/compare.json
# The per-row facts in the prose.
$PY -I ../scripts_tmp/checks.py ../out "$($PY ../scripts_tmp/fetch.py 2>/dev/null)"
# #10561's newer head, for the note above the table.
git fetch origin pull/10561/head:pr-10561-now
git log --format='%h %cI %s' c545c47877c70b367a6fab2322037e453364997e..pr-10561-now
git diff c545c47877c70b367a6fab2322037e453364997e pr-10561-now -- policyengine_us/tools/calculate_chunked.pyNo number in the tables comes from the smoke run. During each side, nothing else heavy ran on the VM, only file reads and a progress monitor. pair_side.py, compare.py, smoke.py and checks.py"""One side of the pair: python pair_side.py <helper.py> <chunks> <year> <out_dir>."""
import importlib.util
import json
import sys
import time
from pathlib import Path
helper, chunks, year, out = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]), Path(sys.argv[4])
VARIABLES = (
"mo_itemized_deductions",
"mo_taxable_income",
"mo_income_tax",
"income_tax",
"state_income_tax",
"household_net_income",
"household_benefits",
)
spec = importlib.util.spec_from_file_location("hub_calculate_chunked", helper)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
import policyengine_us # noqa: E402
from policyengine_us import Microsimulation # noqa: E402
start = time.monotonic()
result = module.calculate_chunked(
lambda dataset: Microsimulation(dataset=dataset, dataset_end_year=year),
VARIABLES,
year,
chunks,
dataset=Microsimulation.default_dataset,
)
out.mkdir(parents=True, exist_ok=True)
for entity, table in result.entities.items():
table.to_pickle(out / f"{entity}.pkl")
summary = {
"package": str(Path(policyengine_us.__file__).parent),
"chunks": chunks,
"year": year,
"weighted_sums": result.weighted_sums,
"wall_seconds": time.monotonic() - start,
}
(out / "summary.json").write_text(json.dumps(summary, indent=1, sort_keys=True))
print(json.dumps(summary, sort_keys=True), flush=True)"""Compare the two sides: python compare.py <base_dir> <head_dir> <out.json>."""
import json
import sys
from pathlib import Path
import numpy as np
import pandas as pd
base, head, out = Path(sys.argv[1]), Path(sys.argv[2]), Path(sys.argv[3])
compare = {}
for base_path in sorted(base.glob("*.pkl")):
entity = base_path.stem
left = pd.read_pickle(base_path)
right = pd.read_pickle(head / base_path.name)
key = f"{entity}_id"
if not np.array_equal(left[key].to_numpy(), right[key].to_numpy()):
sys.exit(f"{entity}: the two sides hold different IDs or a different order")
if not np.array_equal(left["weight"].to_numpy(), right["weight"].to_numpy()):
sys.exit(f"{entity}: the two sides hold different weights")
if set(left.columns) != set(right.columns):
sys.exit(f"{entity}: the two sides hold different columns")
ids = left[key].to_numpy()
weight = left["weight"].to_numpy(dtype=float)
for name in left.columns.drop([key, "weight"]):
a = left[name].to_numpy(dtype=float)
b = right[name].to_numpy(dtype=float)
if np.isnan(a).any() or np.isnan(b).any():
sys.exit(f"{name}: a side holds NaN")
difference = b - a
moved = np.abs(difference) > 1e-6
largest = np.argsort(-np.abs(difference))[: min(5, int(moved.sum()))]
compare[name] = {
"entity": entity,
"base_sum": float((a * weight).sum()),
"head_sum": float((b * weight).sum()),
"change": float((b * weight).sum() - (a * weight).sum()),
"records_changed": int(moved.sum()),
"weighted_changed": float(weight[moved].sum()),
"weighted_up": float(weight[difference > 1e-6].sum()),
"weighted_down": float(weight[difference < -1e-6].sum()),
"largest_abs": float(np.abs(difference).max()),
"rows": [
{"id": int(ids[i]), "base": float(a[i]), "head": float(b[i])}
for i in largest
],
}
out.write_text(json.dumps(compare, indent=1))
print(json.dumps({k: v["records_changed"] for k, v in compare.items()}, indent=1))"""Smoke test: run the helper on the first N households in 2 chunks."""
import importlib.util, sys, time, json
import numpy as np
helper, n = sys.argv[1], int(sys.argv[2])
spec = importlib.util.spec_from_file_location("hub_calculate_chunked", helper)
module = importlib.util.module_from_spec(spec); spec.loader.exec_module(module)
import policyengine_us
from policyengine_us import Microsimulation
VARIABLES = ("mo_itemized_deductions","mo_taxable_income","mo_income_tax","income_tax","state_income_tax","household_net_income","household_benefits")
t = time.monotonic()
ds = module._load_dataset(Microsimulation.default_dataset)
src = module._EntityTables(ds)
print("households:", src.household_count, "load s:", round(time.monotonic()-t,1), flush=True)
masks = {e: owners < n for e, owners in src.positions.items()}
small = src.chunk(masks)
del ds, src
t = time.monotonic()
r = module.calculate_chunked(lambda d: Microsimulation(dataset=d, dataset_end_year=2026), VARIABLES, 2026, 2, dataset=small)
print(json.dumps({"package": policyengine_us.__file__, "sums": r.weighted_sums, "entities": {k: list(v.columns) for k, v in r.entities.items()}, "calc_s": round(time.monotonic()-t,1)}, indent=1), flush=True)"""Checks behind the prose: python checks.py <out_dir> <dataset.h5>."""
import json
import sys
from pathlib import Path
import numpy as np
import pandas as pd
out, dataset = Path(sys.argv[1]), sys.argv[2]
compare = json.loads((out / "compare.json").read_text())
with pd.HDFStore(dataset, mode="r") as store:
household = store["/household"][["household_id", "state_fips"]]
person = store["/person"][["person_id", "person_tax_unit_id", "person_household_id"]]
missouri = set(household.household_id[household.state_fips == 29])
unit_home = person.groupby("person_tax_unit_id").person_household_id.first()
person_home = person.set_index("person_id").person_household_id
person_unit = person.set_index("person_id").person_tax_unit_id
def load(entity):
key = f"{entity}_id"
base = pd.read_pickle(out / "base" / f"{entity}.pkl").set_index(key)
head = pd.read_pickle(out / "head" / f"{entity}.pkl").set_index(key)
assert base.index.equals(head.index) and base.weight.equals(head.weight)
return base, head
tables = {entity: load(entity) for entity in ("tax_unit", "person", "household")}
def delta(entity, name):
base, head = tables[entity]
return head[name].astype(float) - base[name].astype(float)
homes = {"tax_unit": unit_home, "person": person_home, "household": None}
moved = {}
for name, c in compare.items():
d = delta(c["entity"], name)
ids = d.index[d.abs() > 1e-6]
moved[name] = ids
lookup = homes[c["entity"]]
where = set(ids) if lookup is None else set(lookup.loc[ids])
print(f"{name}: {len(ids)} rows move, up {int((d > 1e-6).sum())}, "
f"down {int((d < -1e-6).sum())}, outside Missouri {len(where - missouri)}")
units = set(moved["mo_income_tax"])
print("state_income_tax moves on the same tax units as mo_income_tax:",
set(moved["state_income_tax"]) == units)
people_units = set(person_unit.loc[moved["mo_taxable_income"]])
print("tax units of people whose mo_taxable_income moves:", len(people_units))
print("households holding the moved tax units == households whose net income moves:",
set(unit_home.loc[list(units)]) == set(moved["household_net_income"]))
# Where the weighted change sits.
weight = tables["tax_unit"][0].weight.astype(float)
for name in ("mo_income_tax", "mo_itemized_deductions"):
d = delta("tax_unit", name)
rows = d.index[d.abs() > 1e-6]
contribution = (d * weight).loc[rows].sort_values(key=abs, ascending=False)
top = contribution.index[0]
print(f"{name}: largest weighted contributor tax unit {top}, weight {weight[top]:,.4f}, "
f"change {d[top]:,.5f}, contribution {contribution.iloc[0]:,.2f} "
f"of {contribution.sum():,.2f} ({contribution.iloc[0] / contribution.sum():.2%}); "
f"other rows weigh {weight.loc[rows].sum() - weight[top]:,.4f}")
heavy = rows[weight.loc[rows] > 1]
print(f" rows with weight above 1: {len(heavy)}, weight {weight.loc[heavy].sum():,.2f} "
f"of {weight.loc[rows].sum():,.2f}, share of change {contribution.loc[heavy].sum() / contribution.sum():.4%}")
for unit in (1020906, 1175091, 1175222):
print(f"tax unit {unit}: weight {weight[unit]:.6f}, "
f"d mo_itemized_deductions {delta('tax_unit', 'mo_itemized_deductions')[unit]:,.5f}, "
f"d mo_income_tax {delta('tax_unit', 'mo_income_tax')[unit]:,.5f}")
base_person, head_person = tables["person"]
for p in person.person_id[person.person_tax_unit_id == 1175222]:
print(f" person {p}: mo_taxable_income {float(base_person.mo_taxable_income[p]):,.2f} -> "
f"{float(head_person.mo_taxable_income[p]):,.2f}")
# Household net income against the state tax of the household's tax units.
d_tax = delta("tax_unit", "state_income_tax")
d_tax_home = d_tax.groupby(unit_home.loc[d_tax.index].to_numpy()).sum()
d_net = delta("household", "household_net_income")
gap = d_net + d_tax_home.reindex(d_net.index, fill_value=0.0)
hh_weight = tables["household"][0].weight.astype(float)
rows = moved["household_net_income"]
print("dtypes:", tables["household"][0].household_net_income.dtype,
tables["tax_unit"][0].state_income_tax.dtype)
print(f"per-household gap: max {gap.loc[rows].abs().max():.5f} in moved households, "
f"max {gap.drop(rows).abs().max():.5f} elsewhere; weighted {(gap * hh_weight).sum():,.4f}")
top = (gap * hh_weight).loc[rows].abs().idxmax()
print(f" largest weighted gap: household {top}, gap {gap[top]:.5f}, weight {hh_weight[top]:,.4f}, "
f"weighted {gap[top] * hh_weight[top]:,.4f}")compare.json{
"household_net_income": {
"entity": "household",
"base_sum": 17161428039502.375,
"head_sum": 17161427887974.604,
"change": -151527.771484375,
"records_changed": 23,
"weighted_changed": 9249.667588802986,
"weighted_up": 0.0,
"weighted_down": 9249.667588802986,
"largest_abs": 849.375,
"rows": [
{
"id": 1128492,
"base": 367653.9375,
"head": 366804.5625
},
{
"id": 1072275,
"base": 442840.5625,
"head": 442055.375
},
{
"id": 1128446,
"base": 181727.359375,
"head": 181217.265625
},
{
"id": 1072304,
"base": 394948.8125,
"head": 394457.90625
},
{
"id": 1072213,
"base": 291276.40625,
"head": 290814.0
}
]
},
"household_benefits": {
"entity": "household",
"base_sum": 2059793926050.1262,
"head_sum": 2059793926050.1262,
"change": 0.0,
"records_changed": 0,
"weighted_changed": 0.0,
"weighted_up": 0.0,
"weighted_down": 0.0,
"largest_abs": 0.0,
"rows": []
},
"mo_taxable_income": {
"entity": "person",
"base_sum": 197047733172.8578,
"head_sum": 197050958639.661,
"change": 3225466.8032226562,
"records_changed": 45,
"weighted_changed": 18499.329391600564,
"weighted_up": 18499.329391600564,
"weighted_down": 0.0,
"largest_abs": 18072.125,
"rows": [
{
"id": 1328245,
"base": 413239.90625,
"head": 431312.03125
},
{
"id": 1183783,
"base": 332891.0,
"head": 346073.09375
},
{
"id": 1328138,
"base": 96983.59375,
"head": 106735.9140625
},
{
"id": 1328078,
"base": 118565.9921875,
"head": 126957.265625
},
{
"id": 1182842,
"base": 932945.875,
"head": 941158.9375
}
]
},
"mo_itemized_deductions": {
"entity": "tax_unit",
"base_sum": 33959651949.086754,
"head_sum": 32993725109.992935,
"change": -965926839.0938187,
"records_changed": 176,
"weighted_changed": 405076.13222510496,
"weighted_up": 0.0,
"weighted_down": 405076.13222510496,
"largest_abs": 18072.140625,
"rows": [
{
"id": 1175091,
"base": 46235.1953125,
"head": 28163.0546875
},
{
"id": 1096899,
"base": 52878.53125,
"head": 35305.4921875
},
{
"id": 1097245,
"base": 79457.5078125,
"head": 62750.8828125
},
{
"id": 175319,
"base": 48850.9140625,
"head": 32791.80859375
},
{
"id": 174985,
"base": 30642.91015625,
"head": 15321.4560546875
}
]
},
"mo_income_tax": {
"entity": "tax_unit",
"base_sum": 8643954003.345985,
"head_sum": 8644105598.858131,
"change": 151595.5121459961,
"records_changed": 23,
"weighted_changed": 9249.667588802986,
"weighted_up": 9249.667588802986,
"weighted_down": 0.0,
"largest_abs": 849.388671875,
"rows": [
{
"id": 1175091,
"base": 19241.64453125,
"head": 20091.033203125
},
{
"id": 1097245,
"base": 19467.9296875,
"head": 20253.14453125
},
{
"id": 1175032,
"base": 4711.44921875,
"head": 5221.54345703125
},
{
"id": 1097285,
"base": 15586.958984375,
"head": 16077.876953125
},
{
"id": 1097155,
"base": 11317.326171875,
"head": 11779.751953125
}
]
},
"income_tax": {
"entity": "tax_unit",
"base_sum": 2595105975379.617,
"head_sum": 2595105975379.617,
"change": 0.0,
"records_changed": 0,
"weighted_changed": 0.0,
"weighted_up": 0.0,
"weighted_down": 0.0,
"largest_abs": 0.0,
"rows": []
},
"state_income_tax": {
"entity": "tax_unit",
"base_sum": 578044599361.5671,
"head_sum": 578044750957.0793,
"change": 151595.51220703125,
"records_changed": 23,
"weighted_changed": 9249.667588802986,
"weighted_up": 9249.667588802986,
"weighted_down": 0.0,
"largest_abs": 849.388671875,
"rows": [
{
"id": 1175091,
"base": 19241.64453125,
"head": 20091.033203125
},
{
"id": 1097245,
"base": 19467.9296875,
"head": 20253.14453125
},
{
"id": 1175032,
"base": 4711.44921875,
"head": 5221.54345703125
},
{
"id": 1097285,
"base": 15586.958984375,
"head": 16077.876953125
},
{
"id": 1097155,
"base": 11317.326171875,
"head": 11779.751953125
}
]
}
}Generated by Claude Code |
|
US + core hub merge audit, us#10635 at
🤖 Generated with Claude Code |
Summary
Missouri allows the self-employment tax as an itemized deduction only "to the extent that such taxes were not deducted in the computation of the taxpayer's federal adjusted gross income" (RSMo 143.141(4)(c)). For 2017, filers include federal Form 1040 line 57 less line 27 on Form MO-A Part 2 line 7 (MO-1040 instructions, page 42). The model listed
self_employment_taxin full ingov.states.mo.tax.income.deductions.itemized, so it counted twice the half that filers deduct in computing federal AGI.Change
mo_self_employment_tax_addback(TaxUnit):max_(0, add(self_employment_tax) - self_employment_tax_ald).itemized.yamlin place ofself_employment_tax, from 1993 on (the list's only period).Tests
mo_itemized_deductionsfalls from 34,129.55 to 27,064.775 in both 2017 and 2024. The drop equalsself_employment_tax_ald, half the tax (7,064.775).mo_self_employment_tax_addback.yamlchecks that the amount:impact: we estimate that Missouri itemized deductions fall $966 million in 2026, for about 405,000 tax units with self-employment tax. Only federal itemizers can itemize in Missouri, and they take the larger of the itemized and standard deductions. Missouri income tax rises $152,000, and 99.95% of that comes from one record with a weight of 9,249. Household net income falls by the same $152,000; federal income tax and benefits don't change (pair).
axiom: TheAxiomFoundation/rulespec-us#2002 queued
🤖 Generated with Claude Code