Repository navigation
docker: carry the engine's selected-admit helper and manifest in the release image - #182
luisleo526 wants to merge 2 commits into
Conversation
…release image The release image now installs two more files from the same sha256-verified engine release tarball as the library and headers: - /opt/pineforge/bin/pineforge-selected-admit (mode 0755) - /opt/pineforge/share/pineforge/pineforge-selected-admit.manifest.json (mode 0644) The engine stage records the sha256 of both extracted files. A read-only build mount checks the installed bytes against that record, and the record is never copied into the final image. A separate exact-inventory check refuses extra entries in bin/, share/ and share/pineforge/, requires regular non-symlink files owned by root:root with the exact modes, and ties the manifest's helper name, path, sha256 and byte count to the installed helper. The sixteen-name harness check stays and excludes only the one literal helper name. The HPO target inherits the same files and checks. Image smoke repeats the layout, metadata and manifest checks for each architecture digest, then requires `pineforge-selected-admit --describe` to exit zero and print the canonical manifest with only helper.bytes and helper.sha256 removed. Tests replay the real Dockerfile RUN and the real in-container Python source on isolated fixtures: third files, paths, modes, symlinks, missing or altered assets, source-record names, manifest fields, a one-byte description difference and a nonzero helper exit. The historical runtime-stage digest test keeps its original hash and removes only the exact approved helper additions, and refuses a left-over helper line. The engine-stage download test replays the real instruction on an archive that now carries the two files, checks their installed bytes and modes and the two-name record, and a new test refuses an archive that lacks either file. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 1 changed function(s) (1 not verified).
Formal verification. PR-changed functions: 0/1 verified (0 proven, 0 may-equivalent, 0 distinguished) · 1 not verified (1 vacuous).
Not verified on this run: verify\_runtime (vacuous: never exercised).
Graphify review — findings
Adds the engine's selected-admit helper (/opt/pineforge/bin/pineforge-selected-admit) and its manifest to the image, taken from the same sha256-verified engine release tarball as the library and headers. The build records the two files' sums from the extracted tarball and runs a separate exact helper check (extra entries in bin/, share/ or share/pineforge/, non-regular or non-root-owned files, wrong modes, sha256 mismatches, manifest name/path/hash/size) while the harness check stays a sixteen-name check; the HPO target inherits all of it. Image smoke now rechecks the installed layout, metadata and manifest on each architecture, and requires --describe to exit zero and print the canonical manifest minus helper.bytes and helper.sha256. The release-workflow, HPO-image and asset-fetch tests are extended to pin the new Dockerfile steps and smoke behaviour, and the README, AGENTS.md and CLAUDE.md document the new files and checks.
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 535 functions depend on the 527 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
verify_runtime()— 1 callers, 9 callees
Verification — 535 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 535 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
34 of 34 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_facts_cards.py— full-run-safetytests/test_facts_descriptions.py— full-run-safetytests/test_facts_fanout.py— impact, full-run-safetytests/test_facts_validate.py— full-run-safetytests/test_fingerprint_canonical.py— full-run-safetytests/test_parity_bundle.py— full-run-safetytests/test_parity_source_proof.py— full-run-safetytests/test_release_action_pins.py— full-run-safetytests/test_release_asset_fetch.py— impact, changed-test, full-run-safetytests/test_release_binding.py— full-run-safetytests/test_release_binding_workflows.py— full-run-safetytests/test_release_coordinator_receipts.py— full-run-safetytests/test_release_docs_runtime.py— full-run-safetytests/test_release_fanout_check.py— full-run-safetytests/test_release_guard_policy.py— full-run-safetytests/test_release_hosted_postmerge_receipts.py— full-run-safetytests/test_release_hpo_image.py— impact, changed-test, full-run-safetytests/test_release_main_push_inventory.py— full-run-safetytests/test_release_pair.py— full-run-safetytests/test_release_parity.py— full-run-safetytests/test_release_parity_image.py— full-run-safetytests/test_release_parity_runtime.py— full-run-safetytests/test_release_parity_wire.py— full-run-safetytests/test_release_receipt_collect_sweep.py— full-run-safetytests/test_release_receipt_legacy_previous.py— full-run-safetytests/test_release_receipt_shards.py— full-run-safetytests/test_release_receipts.py— full-run-safetytests/test_release_redeliver.py— full-run-safetytests/test_release_runtime_smoke.py— impact, changed-test, full-run-safetytests/test_release_sink_image.py— full-run-safetytests/test_release_sources.py— impact, full-run-safetytests/test_release_workflows.py— impact, changed-test, full-run-safetytests/test_repo_descriptions.py— full-run-safetytools/release-dry-run/test_wfrun.py— full-run-safety
non-code file(s) changed (
AGENTS.md,CLAUDE.md,README.md,docker/Dockerfile) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
AGENTS.md,CLAUDE.md,README.md,scripts/release_smoke.py) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Docs that may be stale (advisory)
docs/release-chain.md§ Consumer receipt contract v1 (lines 476-1252): references changed symbolsinventoryfacts/README.md§ Trades compared with TradingView (lines 33-46): references changed symbolsinventoryfacts/README.md§ Inventory source and refill rule (lines 47-86): references changed symbolsinventoryfacts/README.md§ Public repository descriptions (lines 87-127): references changed symbolsinventory
Formal verification
Could not verify: Could not verify verify\_runtime.
The verifier did not have enough to check verify\_runtime, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 264 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)
· 1 more finding(s) on lines outside this diff (see the check run).
The test for an engine archive that lacks one of the two helper assets built the incomplete archive and asserted a non-zero exit plus the files installed before the failing step. It never showed that the complete archive, with the same version, architecture and seed bytes, passes the same real instruction through the same harness, so a failure for any other reason right after the helper install would have satisfied it. Before each omission the test now runs that complete archive and asserts the full positive result (downloaded bytes, checksum, both sha256sum runs before the extraction, installed bytes and modes, the two-name record). The omission case also requires the verified checksum line and the extraction before the failure, the exact "cannot stat" error of the install of the omitted member, and that install as the last command the shell traced, so that only the missing member can satisfy it. Every earlier assertion stays. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Test evidence for the four conditions, by row. All rows ran on one aarch64 Linux host, from a clean checkout of a test-only commit above the first commit Condition 1: an exact allowlist, 16 to 18 named paths, every other check named as unchanged. The PR body lists the unchanged checks with their Condition 2: red first. The changed test files were copied onto the parent commit and run against the parent's Dockerfile and smoke script; each row must fail, and does, with its message:
The tests behind them: a third file is still refused ( Condition 3: the installed bytes against the verified tarball, on fixtures; actual asset provenance is not covered. The engine stage computes the sha256 of both files from the already verified, extracted tarball into a two-name record; the helper RUN, with that record mounted read-only, checks the installed bytes with Real root and the image's own shell. Row
The ordinary-user tests normalize only the fixture's owner; these rows use real root ownership. Row Condition 4 is not covered by these rows. The build of the release image target and the publication-time runtime verification need the engine release asset that carries the two files; they run on the PR head after the engine release and are added here, together with the pushed-index parity-child proof and the consumer replay. Until then neither the provenance of the real helper bytes (actual engine release asset to installed image) nor the behaviour of a built and pushed image is established by this PR. |
|
Closing: no update for more than 24 hours. The branch is kept; the change returns inside one combined pull request after it is verified on current main. |
The release image now carries the engine's
pineforge-selected-admithelper and its manifest from the same sha256-verified engine release tarball as the library and headers:/opt/pineforge/bin/pineforge-selected-admit, mode 0755./opt/pineforge/share/pineforge/pineforge-selected-admit.manifest.json, mode 0644.The payload allowlist remains explicit: sixteen vendored harness files plus these two named assets. The two COPY instructions name individual files and preserve their installed modes. The HPO target inherits the same assets and checks.
New checks:
bin/,share/andshare/pineforge/reject every additional entry, including hidden files and helper backups. The harness's separate sixteen-name check excludes only the one literal helper name.--describewhose bytes equal the canonical manifest with onlyhelper.bytesandhelper.sha256removed. The build-time comparison ties the installed bytes to the files of the sha256-verified engine tarball; the release bindings do not contain these per-file hashes. No actual engine release asset that carries the two files has been run through this build yet.Existing checks and runtime settings are UNCHANGED as listed below. References are to
docker/Dockerfilein this change.HARNESS-SHA256SUMS, its exact sixteen-namewantandlistedchecks, strict installed-byte verification, existing failure and success messagesEXPECTED_PF_ABIversus bundled headerPF_ABI_VERSION, including refusal of missing valuesLICENSE-SHA256SUMScompleteness, placeholder refusal and strict verification of every copied licenceThe only executable change inside the existing harness RUN is the exact helper-name exception on line 184. The new adjacent RUN enforces the complete directory layout. The historical runtime digest test keeps its original hash and removes only the exact approved helper additions before comparison. All existing harness negative tests remain.
Added tests replay the real Dockerfile RUN and the actual in-container Python source. They cover third files, exact paths and modes, missing or altered assets, source-record names, symlinks, manifest fields, a one-byte description mismatch and a nonzero helper exit, with passing controls and asserted failure messages. Ordinary-user fixtures normalize only ownership metadata after real stat; actual root ownership is enforced by the image build and runtime checks. The helper in every fixture is a small shell script written by the test builder, not the engine's helper binary, so these tests exercise the Dockerfile's checks and the smoke script, not the real helper's bytes. The engine-stage download tests replay the real instruction on an archive that now carries the two files and check their installed bytes, their modes (the archive entries deliberately differ from the installed modes) and the two-name sha256 record. The test that refuses an archive lacking either file first runs the complete archive (same version, architecture and seed bytes) through the same instruction and asserts the whole positive result; each omission then must show the verified checksum line and the extraction before the failure, the exact install error naming the omitted file, and that install as the last command the shell traced, so only the missing file can satisfy it.
Checks run
Run on one aarch64 Linux host with Python 3.12, from a clean checkout of this branch's head, with the hub CI's own command for the release rules,
python3 -m unittest discover -s tests -p 'test_release_*.py', and the CI's two HPO wheels (downloaded with the commands of that CI step and checked against their pinned sha256 values):tests/test_release_workflows.py,tests/test_release_hpo_image.py,tests/test_release_runtime_smoke.py,tests/test_release_asset_fetch.py) were copied onto the parent commit and each was run against the parent's Dockerfile and smoke script. All four fail, for the stated reasons: the Dockerfile holds no helper RUN, there is no approved helper fragment to remove from the release stage, the smoke script has no helper runtime check, and the engine-stage replay installs no helper files and runssha256sumonce instead of twice.false;line), makes the omission test fail at its control in both of its cases, and its omission case is never reached; on the unmodified Dockerfile the same test passes.sha256sum -c --strictlines and its success line, and it is refused, with the exact messages, for a third file inbin/, a helper owned by another user, a manifest owned by another user, mode 775 on the helper, and one appended byte. The in-container Python check that image smoke runs passes as root and refuses a helper owned by another user. The ordinary-user tests normalize only the fixture's owner; this run uses real root ownership. The helper in that layout is the tests' shell fixture helper, not the engine's helper binary, and the container is the published 1.11.0 image, not a build of this change.Not run yet: the build of the release image target (its engine stage needs an engine release asset that carries the two files), so the provenance of the real helper bytes, from an actual engine release asset to the installed image, is not established by the tests above; the runtime verification of a built image on each architecture digest; the pushed-index parity-child proof; the consumer replay; x86_64 and the hosted CI legs. Required before the v1.11.1 tag: build the release image target on the PR head, run the same runtime verification commands that publication uses without pushing an image, record the results here, then run the consumer replay and the pushed-index parity-child proof.
🤖 Generated with Claude Code