Repository navigation
Conversation
This was referenced Sep 9, 2026
Draft
16 tasks
hua7450
pushed a commit
to hua7450/microcosm
that referenced
this pull request
Sep 20, 2026
…rces The rule was decided on 15 September and written only on the PolicyEngine#893 integration branch, so main carried no authority for its own standing check. This states the rule, and the four things a from-scratch attempt on main measured between main and that build: a US Chronicle feed with no dimension labels, no build stage that emits the SPM independence role, a July selection that does not map onto a rebuilt base, and three raw inputs that exist in one untracked directory. It records the 17 September decisions on SPM composition and the stage times that were measured. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 21, 2026
`_SPM_AGE_BANDS` was declared and never read — the vocabulary it documented belongs in `_age_band`'s docstring, where it now lives with the reason exact ages are not emitted. The gate-site comment asserted "~0.05 s on a 900k-person pool" as if measured here. It was measured by the #893 lane on the 907k-person phase-2 base pool and recorded in the journal; the comment now says so instead of claiming it fresh. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
4 of 7 tasks
MaxGhenis
added a commit
that referenced
this pull request
Sep 23, 2026
Conflicts (release input coverage) resolved as a union of both sides: is_spm_independent_minor_role (this branch, #893) and the three ASEC reported-receipt inputs receives_wic/snap/tanf (#980, #978 option 1) are all required with no reviewed exclusion; manifest counts 167 required / 174 total; derivation text names both additions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Sep 23, 2026
Closed
juaristi22
added a commit
that referenced
this pull request
Sep 25, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py is registered in the explicit shared-spec lane. Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 25, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 26, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py is registered in the explicit shared-spec lane. Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 26, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 26, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py is registered in the explicit shared-spec lane. Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 26, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 28, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py lives in tests/engine_free/shared/ (the directory is the lane; re-stacked on #901 b2538d1). Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 28, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the invented three-system support helpers (parts, payloads) that the lineage test imported from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 28, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py lives in tests/engine_free/shared/ (the directory is the lane; re-stacked on #901 b2538d1). Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 28, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the invented three-system support helpers (parts, payloads) that the lineage test imported from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 28, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py lives in tests/engine_free/shared/ (the directory is the lane; re-stacked on #901 b2538d1). Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 28, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the invented three-system support helpers (parts, payloads) that the lineage test imported from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 29, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py lives in tests/engine_free/shared/ (the directory is the lane; re-stacked on #901 b2538d1). Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 29, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the invented three-system support helpers (parts, payloads) that the lineage test imported from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 29, 2026
atomic_geography.py, graph_atomic_geography.py and tests/test_atomic_geography.py are the bytes at #893's 2300e56 (ebb8db8 introduced them, 1cebe35 added the gate's validation artifact and emit_validation_artifact, which the UK node swap consumes); they are blob-identical to #893's current head a402dfc, so #893 rebases over this commit without conflict. Country-agnostic: their imports (keyed_uniform, load_source_bytes, KernelRole, canonical) are on main. test_atomic_geography.py lives in tests/engine_free/shared/ (the directory is the lane; re-stacked on #901 b2538d1). Verified: 37 tests passed; ci_test_groups --verify ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Sep 29, 2026
Port uk_runtime/atomic_area_support.py, atomic_household_identity.py and atomic_household_lineage.py, their tests test_uk_atomic_area_support.py and test_uk_atomic_household_lineage.py, and docs/geography-assignment.md and docs/uk-atomic-household-lineage.md byte-for-byte from #893 at 2300e56 (Max's b1626e1 and aaf3174, as amended on that line through 1cebe35). The modules stay module-local imports: nothing on #932 reaches them through uk_runtime/__init__.py, so main's eager package init is untouched. Re-stacked on #901 b2538d1: the two tests live in tests/engine_free/uk/, and the invented three-system support helpers (parts, payloads) that the lineage test imported from its sibling now live in test_support/microcosm_build/uk_atomic_area_support.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
Author
juaristi22
added a commit
that referenced
this pull request
Nov 6, 2026
…ext (#931) Three review notes, none of which changes code behaviour: - **UC capital (Vahid, item 1).** The note says `UC_CAPITAL_MINIMUM_CELL_DONORS` counts a cell's donor benefit-unit rows after the geographic fold, so a high-value copy counts as a donor of its own. It is a row threshold, not a disclosure floor. Whether it should count true sources is for the move's audit of K-sensitive constants, since switching it can move outputs. - **Scotland (Vahid, item 3).** The conditioned supports' provenance now notes that Scotland's band mix is its council's, so within a council a household is placed in proportion to households. It names a mix by 2022 data zone as the way to lift that, once one is registered in Chronicle. The support bytes are unchanged. - **The US doc passages (planning review, item 3).** `docs/geography-assignment.md` notes, at the head of each, that the US passages describe microcosm#893, which closed unmerged on 9 October 2026. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
added a commit
that referenced
this pull request
Nov 7, 2026
A structural node stores every column of its version, then the whole frame. _write_node first built a copy of every column, each with its own copy of the entity's ids, and only then wrote the first one. On a late 15-clone spine frame that is a second frame plus one id array per column, about 10 GiB held at every structural write on top of the version itself. A cache hit on a structural node did the same the other way round: it loaded every stored column, each with its own ids, and held them all while it loaded the frame, though it returns only the frame. The spine runs on a persistent node store with resume="auto", so a spine restarted after a failure meets that at every structural node it already stored. - The structural write now streams. Every dtype token is still taken first, so an unsupported dtype is refused before anything is written. Then each entity's ids are copied once, each column is copied only when it is written, and each copy goes before the next is made. The order is the same sorted (entity, column) order, so the record lists the same entries. - put_column only reads the series and its ids. It wraps the ids in a new index without copying them, writes them and keeps nothing once it returns, so one id array shared by an entity's columns writes the same bytes as one copy per column. - A structural hit still loads every column, in the record's order, and still refuses a damaged or repeated one, but it keeps none of them: its version comes from the frame. An ordinary hit still returns the columns it loads. - The ordinary-node write and the frame, weight, opaque and record writes are unchanged. Integration layer 02 (69b94b4) once wrote this per column too, taking each dtype token only when it reached the column, so an unsupported dtype would have been refused after earlier columns were stored. The #893 reconciliation (80fef22) dropped it for want of a consumer. This version takes every token first and shares one id index per entity, and the 15-clone UK spine is its consumer. No output moves. The new graph tests keep the old structural column copies verbatim but for their names, and only those stand in for the new ones, so the rest of the write is shared. A toy graph with CREATE, FILTER, REWEIGHT and EXPAND nodes stores byte-identical objects under the same keys with either, cold, warm and rewritten under resume="forbid", every node's artifacts match in order, and the warm run, where every node hits, reports what the cold run did. A direct write of a frame with every graph dtype, nulls, and entities and columns out of sorted order returns the same artifacts and record, and an unsupported dtype still leaves the store empty. Weak references show each column copy gone before the next is made, and each entity's columns share one id index. They show a structural hit loading each column in the record's order and holding none once it loads the next one or the frame, and a damaged or repeated column is still refused. On a synthetic frame with 10 MB of columns, tracemalloc puts the write's peak at 2.1 MB, against 22.3 MB before, and a structural hit's at 34.9 MB, against 55.1 MB before. What remains of the hit is the frame load itself. No kernel source hash or committed pin binds the executor, so no node key moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Construct one US population through ACS+ASEC harmonization, complete initial support clones, household Census-block assignment, derived geographies, financial enrichment and a PUF alternative on the combined survey frame. National, state and congressional-district analyses use this population with explicit calibration and pruning. The release path and review guide describe the architecture and remaining release work.
The graph retains source observations, reporting universes, unresolved values, design weights, relationships and geography. Prior wages are excluded. Financial and property-income stages precede the PUF alternative; subsequent amount, health and housing stages retain checked parent populations. Country models own benefit awards and caps. Unknown inputs remain unknown, and incomplete tax inputs refuse PUF qualification.
This integration includes optional child property-income completion, person-status and household-role stages. The completion host now distinguishes private donor/recipient support frames from the full US population while retaining frame, manifest and mutation checks. It also incorporates current main, including María's UK target changes in #904. The UK specification fingerprint was computed from the actual merged source/resource set before resolving the conflict. Shared calibration snapshots, best-iterate selection, fiscal measurement, geography primitives and UK lineage remain included.
Validation is scoped to the source and run recorded in the dated evidence:
81099c1986b54ec92a3efceb72867938887cd4e7e3c9a81bb21630154824e1d9. The maintained case completed in 11.33 seconds wall time and 9.78 seconds CPU; its receipt is2d81ad311965fb795038c1cbf9d7656b9f3e0566c44b3cc228e209ac0409b796. This validates compilation and identity, not a UK population release.The latest native property/tax attempt also stopped at its 21,600-second CPU limit without a final acceptance receipt. Neither that attempt nor the 45-node test certifies a native PUF result. Repeated verification is under investigation; no larger native budget or automatic retry is implied.
Release work still includes measuring applicable missing-input routes, choosing any necessary completion models, the retirement measurement bridge, a complete small native build, held-out evaluation, national/state/CD calibration, progressive scale, full/compact exports and a dashboard tied to the accepted bytes. Historical native financial replay is useful evidence for the earlier prefix only. This draft PR publishes source and review evidence; it does not release a dataset, promote a default or deploy the site.
Reconcile with main (17 September)
Two merges bring the branch level with
mainatd1196af10(74 commits, including the US engine lock at policyengine-us 2.2.1 / spm-calculator 1.0.0 / policyengine-core 3.32.5 from #936 and UK #927, #934, #937):8ce9224e7ande47adc6db. Six files conflicted, all identity pins or tests; pins were regenerated from the merged tree rather than taken from either side. Before the push: ruff clean;tools/ci_test_groups.py --verifyok;tools/spec_engine_coverage.py --check42156/42156 fields and 41/41 inventory checks; the identity and pin test set 354 passed; the three lock-binding tests 13 passed (sha256(uv.lock)equals the approved lock digest inworker_identity.pyandtools/spec_seed_identity_diagnostics.py).An independent read-only review of the two merge commits found nothing lost from main (its non-text
county_fipsgate and tests are present beside this branch's own nine ladder tests; the branch carried no edits of its own underus/spec/, so taking main's values for those four inventory hashes dropped nothing), no1.819residue outside three historical comments, the eight engine parameter-file pins equal to the installed 2.2.1RECORD, and the 15 September decisions text intact. The review's open notes are closed on the branch.graph_fiscal_measurement.pynow binds the release SPM selection ({"geography_kind": "county"}, asserted equal toreform_validation.US_RELEASE_SPM_SELECTION) into the model contract and passes it to the engine adapter. The static dependency closure is blind to what the calculator-generated measurement variables read (spm_unit_spm_thresholdhas no input leaves in the index, although the provider reads the household county, the unit's tenure and each member's age), so a root that reaches one of those variables now declaresage,county_fipsandspm_unit_tenure_typeas leaves: an undeclared one refuses the node declaration (UNDECLARED_MODEL_LEAF) and a missing value refuses the kernel, both before the engine runs. A test pins the measurement roster against the installed engine and calculator, so an upgrade that adds a consumer moves it.DISTRIBUTIONSintools/spec_seed_identity_diagnostics.pynow lists spm-calculator. The loader golden vector is a value neither side had because it binds the seed protocol implementation hash, which both sides moved; the test now asserts that tie.