test(capsule): prove every remote-triggered cache-fill records Relayed provenance - #442
Conversation
3d1f819 to
6980fde
Compare
loop-security — IN PROGRESS, not the verdictAudited head: Tooling: grep + direct read of git objects at the head SHA, no worktree, no checkout, no mutation of the shared submodule (only Confirmed so far: 1. Exactly one production caller — the signature change is total. 2. 3. Ordering: no window where a capsule is discoverable with provenance unwritten. 4. Failure direction is fail-closed. Still open, being worked now: the |
loop-security — IN PROGRESS (2/3), not the verdictHead still 5. 6. The inventory scan really is the only production producer. 7. The money chain is exactly as the brief describes, traced end to end. 8. Peer transport threads 9. The control test is real and would fail on the over-correction. 10. The out-of-directory edit widens NOTHING.
One defense-in-depth item and the scope question remain; verdict next. |
loop-security: PASSAudited head: Nothing is CRITICAL or HIGH in this diff. Under the default posture a remote peer can NO LONGER cause a The reachability answer, stated preciselyThree routes a stranger could use to put bytes in this node's cache, and what each records now:
I closed the one call site the lane flagged as unverified. Verified, in the brief's order1. The wrong thing is now inexpressible on this path, not merely checked. 2. No window, and the red-run fix did not create one. The sequence at 3. Failure direction is closed. I checked that cleanup specifically, because 4. The control test is real and the pair is non-vacuous. 5. The origin input cannot be spoofed. 6. The out-of-directory edit widens nothing. 7. The money chain, traced end to end. Concurrency note (benign, checked rather than assumed). One defense-in-depth item — NOT gating
It is not exploitable at this head, which is why it is not a gating finding: Not weighed against this PRDefect A (#446) and the already-held early return are the deliberate scope decisions stated in the brief, and I agree with both — in particular the early return errs toward forgoing a bond rather than staking one on a stranger's content, which is the correct direction. dig-download's own sink remains un-audited by me as well. I traced by grep and direct read of git objects at the head SHA and used no call-graph tool (the registered gitnexus No shared checkout was mutated: everything was read via FYI for the merge gate rather than a security finding: |
6980fde to
e857d31
Compare
Provenance decides whether this node announces a capsule and stakes the operator's $DIG on it. It is not carried in the capsule's bytes -- those are content-addressed and identical whether the node pulled them for itself or for a stranger -- so it lives in a `<root>.relay` sidecar whose ABSENCE means `Held`, the bondable state. Until now that sidecar's only writer was `persist_holder_claim`, reached solely from the reshare path's staging promotion, so every land route that did not stage inherited the bondable default. A missed write failed OPEN. `land_capsule_bytes` now takes a required `HolderClaim` and records it BEFORE the bytes become visible, so there is no window in which a capsule is discoverable with its provenance unwritten, and a marker that cannot be written fails the land rather than landing unmarked. A future land route cannot inherit `Held` by omission -- there is no signature left to call incorrectly. This is the reasoning that already made `CapsuleProvenance` an enum with no `Default`, extended to the filesystem that was quietly supplying one. `cache.pushCapsule` derives its claim from the request origin: a push over the opened peer surface is `Suppress`. The authorized-writer signature does not change that -- authority answers "is this content legitimate", provenance answers "should THIS operator stake THEIR money on it", and a third party who owns the store's key is not thereby entitled to spend the node operator's $DIG. Tests, both over the real push->land path, varying only ReadOrigin: - a_local_push_lands_held_and_therefore_bondable -- the control, which catches an over-correction marking every land Relayed and breaking operator bonding. - a_peer_originated_push_must_land_relayed -- was RED (left: Some(Held)), now green. Closes #436
e857d31 to
e0371cd
Compare
Rebase Complete ✓
Branch rebased onto |
|
IN PROGRESS — not the verdict. Independent correctness gate, head Confirmed so far, with evidence: Q2 — the Q1 — the other failure directions, each stated:
**Q5 — Still open: the third cache-write path ( |
loop-security re-gate — IN PROGRESS, NOT THE VERDICTHead audited: Posting findings as they are established so they survive an interruption. This is not the verdict. Note on tooling: the CLEARED — Q3, attacker-controlled path (
|
| # | writer | holds cache_lock? |
writes a marker? |
|---|---|---|---|
| A | land_capsule_bytes (capsule_store.rs:402) — the push path |
yes (caller) | yes — this PR |
| B | sync_module_from (lib.rs:2531) via cache_fetch_and_cache (capsule_store.rs:272) |
yes | NO |
| C | sync_module_from via sync_module_and_bound → dispatch.rs:899 (read path) |
no (lib.rs:2425 says so explicitly) |
NO |
| D | promote_into_cache (module_reshare.rs:287-333) — reshare warm |
no (cache_lock appears nowhere in module_reshare.rs) |
yes |
Two consequences, both being verified before I call them:
- The doc's lock claim is narrower than stated.
capsule_store.rs:387says "The caller MUST holdcache_lockso a concurrent pull-land of the same capsule cannot race the write." That serializes A against B only. C and D write the same path without ever takingcache_lock, so the write A is protecting against is not actually excluded. sync_module_fromlands with no marker at all, so every capsule that arrives through B or C reads asHeldand is bondable regardless of who asked for it.
Reachability of C from a remote peer is what I am verifying next: dig.getContent is on the peer allowlist (peer.rs, peer_allowlist_is_byte_identical_to_the_pre_adoption_set), and I have not yet found an origin gate on the sync_module_and_bound call at dispatch.rs:899 — the land_origin fold appears only at dispatch.rs:942, after it. If that holds, it is a live money finding of exactly this PR's class, independent of the push path.
To be explicit about attribution: B/C/D are pre-existing, not introduced by this diff. The PR is a strict improvement on the push route. What is at issue is whether its stated invariant — and the PR title, "prove every remote-triggered cache-fill records Relayed provenance" — is true as written.
Owner for anything that comes out of this: the loop. Nothing here is routed to Copilot.
CHANGES-REQUIREDSHA audited: The fix itself is well built and its two tests are genuinely non-vacuous. The blocker is not the fix; it is that the enumeration this PR closes dig-node#436 on is incomplete in the one place that matters, and two safety claims on the PR are false as a result. 1. GATING — an eleventh cache-write path, remote-triggerable, with NO flag gateFull evidence in the inline thread. Summary:
Consequently these two statements are false as written and must not be merged standing:
Owner: the loop. Acceptable remedies: fix the read path in this PR (thread a required 2. Failure-path bondability — every path checked, all correct
The one I most doubted going in — the 3. Test vacuity — REVERT-PROVEN, both directionsRun in my own worktree at Baseline: Mutation A — guard removed ( Mutation B — over-correction ( Each test fails on exactly the mutation it claims to catch and survives the other. Neither is vacuous, and the pair discriminates in both directions — this is the both-inequalities property done right, and the control is what makes it so. Reading the answer through 4. Public surface / version — correct
5. dig-constantsNeither question is triggered. Nothing here is a shared/cross-repo constant: Non-gating notes (resolve without blocking): the I did not re-run the full 1014-test suite or clippy; the PR reports both green and CI covers them. |
… point `dig.getContent` for a capsule this node does not hold funnels through `sync_module_and_bound` -> `sync_module` -> `sync_module_from` -> `write_atomic`, reaching the cache WITHOUT passing `cache_fetch_and_cache` or `land_capsule_bytes`. It carried no `HolderClaim`, so the capsule landed `Held` -- the bondable state a mirror coin is minted against. It is remote-triggerable and behind no feature flag, so a stranger requesting content this node does not hold made it fetch a whole capsule and land it bondable in a default configuration. `lib.rs` already described it as "a remotely-triggered read-path backfill" in unrelated size-cap work. `sync_module_from` now takes a required `HolderClaim`, written before `write_atomic` publishes the bytes, so no window exists where a capsule is discoverable with its provenance unwritten. The claim is threaded PER CALL SITE rather than defaulted at the choke point: `dig.getContent` derives it from the read's `origin` (`Local` -> `Announce`, otherwise `Suppress`), while `cache.fetchAndCache` -- operator-initiated and not peer-reachable -- keeps `Announce`. Suppressing every land through this shared choke point would have disabled the reshare flywheel for the operator's own content. Refs #436 Co-Authored-By: Claude <noreply@anthropic.com>
The first regression tests called `sync_module_from` directly with each `HolderClaim` and asserted the `.relay` sidecar's presence and absence. They passed, and they were not enough: they prove the PLUMBING carries a claim, and would have passed just as happily with `dispatch.rs` mapping every origin to `Announce`. The defect would have survived its own regression test. The origin -> claim decision is now a named `holder_claim_for_read`, asserted directly in both directions: a peer read backfills `Suppress`, a local read backfills `Announce`. An end-to-end assertion on a landed marker passes for many reasons, only one of which is this mapping being right. Extracting it also reads better than a match buried mid-arm -- the rule now has a name, a doc comment stating why the asymmetry is deliberate, and a home where a reader looking for "what decides bondability" will find it. Refs #436 Co-Authored-By: Claude <noreply@anthropic.com>
Pure reflow of lines my own edits made over-width. No behaviour change. Refs #436 Co-Authored-By: Claude <noreply@anthropic.com>
loop-security — IN PROGRESS, not the verdictHead re-anchored: I checked the fmt-only claim rather than taking it:
Conclusion: the head move is semantically inert. Findings derived at Audit method: sink-first. I am deriving the caller set by starting at every filesystem write into the capsule cache directory and walking up, deliberately without reading the PR body's enumeration first. Findings will be posted as they resolve. |
loop-security — FINDING 1, HIGH. IN PROGRESS, not the verdictAudited at The claim is derived from the UNFOLDED origin, so the cross-site axis is ignored
let claim = holder_claim_for_read(origin);
And the trait's own doc,
Concrete exploit — reachable, no feature flag, default installState: operator browses any store through the node's own local surface. The page is attacker-supplied content.
fetch('http://localhost:<port>/', {method:'POST',
headers:{'Content-Type':'application/json'},
body: JSON.stringify({jsonrpc:"2.0",id:1,method:"dig.getContent",
params:{store_id:"<ATTACKER_STORE>", root:"<ATTACKER_ROOT>", retrieval_key:"<rk>"}})})
Result: the capsule lands with no Forty-seven lines later the same request computes Why this is a gate finding and not a nitThis is the same defect class the PR was opened to fix — a land route that reaches the cache and reads as The fix is one argumentlet claim = holder_claim_for_read(crate::download::landing_origin(origin, provenance));
Continuing: I am still working the de-suppression escalation (an |
loop-security — FINDING 2, MEDIUM (defense-in-depth). IN PROGRESS, not the verdictAt
let claim = match origin {
ReadOrigin::Local => crate::seams::dig_peer::HolderClaim::Announce,
_ => crate::seams::dig_peer::HolderClaim::Suppress,
};Raw
And the function's own doc-comment,
So within one function, Why MEDIUM and not HIGH — I checked the reachability rather than assuming it
So a malicious page cannot reach it today. This is defense-in-depth, and I am not gating on it by itself — but it should be fixed in the same commit as Finding 1, because it is literally the same expression and the two will not stay in sync otherwise. Fixlet claim = match landing_origin(origin, provenance) {
Cleared while checking this —
|
loop-security — FINDINGS 3-5 + answers to Q3/Q4/Q5. IN PROGRESS, not the verdictAt FINDING 3, MEDIUM alone / escalates Finding 1 to HIGH — an
|
loop-security VERDICT: CHANGES-REQUIREDAudited head: Method: sink-first derivation, per the brief. I enumerated filesystem write primitives across all five crates, identified the sinks that publish bytes into Ranked findings
Findings 1-3 and 5-6 are detailed in my three prior comments. Finding 4 is new and stated in full below. FINDING 4 -- MEDIUM. The
|
…ort alone `holder_claim_for_read` read the raw `origin`. Every other landing decision in this file folds `origin` with `provenance` through `landing_origin` first (`dispatch.rs:381`, `:953`, `content_serve.rs:366`, `push_capsule.rs:272`), and the trait's own doc states that contract -- so this one call was the outlier. The transport axis is un-spoofable and was read correctly; it is simply not sufficient. A browser is a confused deputy holding a loopback socket on someone else's behalf: a `chrome-extension://` page, or any cross-site page, can POST `dig.getContent` to the loopback port -- CORS admits both (`is_local_origin`, server.rs:433) -- so the request arrives with `origin = Local` while being made for a stranger. That landed an attacker-chosen capsule ANNOUNCED, i.e. bondable, in a default install with no flag in front of it. Because `Announce` also REMOVES an existing marker, it could additionally un-suppress a capsule already correctly relayed. The fold is only ever restrictive (`CrossSite` -> `Peer`, `FirstParty` unchanged), so it cannot cost the operator their own flywheel. Also corrects a comment at the `cache.fetchAndCache` call site that claimed the method is "not peer reachable". That is false -- the inbound-demand backfill reaches it from a remote request -- and only "behind a default-OFF flag" is true. Tracked as #446. Both pre-existing `holder_claim_tests` pass with the single-axis bug present, which is why `a_cross_site_read_over_a_local_socket_backfills_suppressed` exists. Refs #436 Co-Authored-By: Claude <noreply@anthropic.com>
…e shared rule `cache.pushCapsule` decided provenance from the raw `origin` -- the identical defect just fixed on the read path, in a function that ALREADY folds both axes 150 lines earlier for its authority check (`push_capsule.rs:272`). So for a cross-site push over the loopback port the authority half correctly demanded an authorized-writer signature, and the provenance half handed that same push `Announce`. Creating a store is permissionless, so holding a store key is not a trust boundary: a third party entitled to push is not thereby entitled to spend this operator's $DIG. Both sites now call ONE `holder_claim_for_landing`, re-exported from `dig_rpc`. Two copies of this rule would be a rival implementation that can drift apart -- which is exactly how the push site kept the bug after the read site was fixed. Refs #436 Co-Authored-By: Claude <noreply@anthropic.com>
`sync_module_from` wrote the provenance marker and then, on a failed `write_atomic`, left it behind -- so a `Suppress` marker could outlive its land and mis-describe a later capsule arriving at the same path by a different route. `capsule_store.rs` already rolls back this way; this path did not. `Announce` is the REMOVING claim, and removal is only safe because `write_atomic` is temp-in-the-same-directory then rename: on failure it unlinks the temp and leaves NO file at the capsule path, so there is no unmarked capsule for the rollback to expose. Recorded at the call site, because if that ever stops being true this rollback becomes a hole rather than a cleanup. Completes the five items the pre-merge gate required. Refs #436 Co-Authored-By: Claude <noreply@anthropic.com>
All five required items closed — head
|
| # | Item | Commit |
|---|---|---|
| 1 | dispatch.rs — fold both axes via landing_origin |
a1894e78 |
| 2 | push_capsule.rs:422 — same fold |
065772c1 |
| 3 | sync_module_from — restore the marker on a failed land |
82cc673f |
| 4 | capsule_store.rs — correct the false "not peer reachable" justification |
a1894e78 |
| 5 | A test asserting (Local, CrossSite) -> Suppress |
a1894e78 |
Item 2 became a sweep, not a second patch
Both landing sites now call ONE holder_claim_for_landing, re-exported from dig_rpc. Two copies of this rule would be a rival implementation that can drift — which is exactly how the push site kept the bug after the read site was fixed. Worth stating plainly: push_capsule.rs:272 had folded both axes for its AUTHORITY check all along, 150 lines above the provenance decision that read one axis. So for a cross-site push the same function correctly demanded an authorized-writer signature and then handed that push Announce.
Item 3 is the one that would have shipped
Four of five were done when the code looked finished. sync_module_from wrote the marker and, on a failed write_atomic, left it behind, where capsule_store.rs already rolled back — so a Suppress marker could outlive its land and mis-describe a later capsule arriving at the same path by another route.
The rollback uses Announce, the removing claim, and that is only safe because write_atomic is temp-in-the-same-directory then rename: on failure it unlinks the temp and leaves no file at the capsule path. Recorded at the call site, because if that ever stops being true this rollback stops being a cleanup and becomes a hole.
One finding that is NOT fixed here, and must not be read as fixed
#450 — the same-origin variant survives this fold. /s/*path and POST / are the same router on the same port (server.rs:255, :281), and STORE_CSP grants script-src 'unsafe-inline' 'unsafe-eval' plus connect-src 'self' — where 'self' is the RPC endpoint. A store's own page therefore reaches dig.getContent as same-origin → FirstParty → Announce.
The two-axis model cannot express that fix: the operator's own UI and attacker content served at /s/ are both (Local, FirstParty). Same-origin stopped being a trust signal when the node began serving untrusted content on its control origin. Traced link-by-link, not fired.
What this PR now claims, stated exactly
A cross-site or extension-origin request can no longer cause a Held landing on the read or push path. It does not claim the class is closed — #450 is open, and #446's inbound-demand path remains behind its default-OFF flag.
1 participant
Enumerates every path that writes a cache entry, then closes the class the enumeration exposed. Full enumeration with a verdict per path: #436 (comment)
The defect class
Provenance decides whether this node announces a capsule and stakes the operator's $DIG on it. It is not in the capsule's bytes — they are content-addressed and identical whether this node pulled them for itself or for a stranger — so it lives in a
<root>.relaysidecar whose absence meansHeld, the bondable state.That sidecar's only writer was
persist_holder_claim(module_reshare.rs:345), reached solely frompromote_into_cache, the reshare path's staging promotion. Every land route that does not stage inherited the bondable default. A missed write failed OPEN.Two remote-reachable routes did exactly that:
cache.pushCapsulevialand_capsule_bytes. Fixed here.cache_fetch_and_cache. Not fixed here — see below.The fix
land_capsule_bytestakes a requiredHolderClaimand records it before the bytes become visible, so no window exists in which a capsule is discoverable with provenance unwritten, and a marker that cannot be written fails the land rather than landing unmarked. A future land route cannot inheritHeldby omission — there is no signature left to call incorrectly. The same reasoning that madeCapsuleProvenancean enum with noDefault, extended to the filesystem that was quietly supplying one.cache.pushCapsulederives its claim from request origin; a peer-surface push isSuppress. The authorized-writer signature does not change that: authority answers is this content legitimate, provenance answers should THIS operator stake THEIR money on it. Reasoning recorded at the call site.A note on the failure direction: the first run of the fix went red — every peer push refused to land, because the marker was written before
write_atomichad created the store directory. That is the correct direction, and it is worth recording: the guard refused to land rather than landing unmarked.Blast radius
impact-equivalent by grep + read (gitnexus not indexed in this worktree, stated rather than implied).land_capsule_byteshas exactly one caller,push_capsule.rs:413. Also touched:persist_holder_claimvisibility (fn→pub(crate), no behaviour change) and aHolderClaimre-export. No production symbol outside those changed.One edit outside my assigned directory, both mechanical and in files no sibling lane owns:
seams/dig_peer/module_reshare.rs(visibility) andseams/dig_peer/mod.rs(re-export). Flagging rather than assuming it was in scope.Tests
a_local_push_lands_held_and_therefore_bondable— GREEN. The control; catches an over-correction marking every landRelayed, which would break legitimate operator bonding.a_peer_originated_push_must_land_relayed— was RED (left: Some(Held)), now GREEN and un-ignored.dig-node-core --lib: 1014 passed, 0 failed. Clippy-D warningsclean.Unproven — stated, not assumed safe
cache_fetch_and_cache, which has 4 call sites outside my directory —control.rs:1206,control.rs:1456,dig_rpc/dispatch.rs:764,dig-wallet/src/lib.rs:408— plus ~6 inlib.rstests. All appear operator-initiated (Announce), but I did not make that change and no test exercises the inbound-demand path.dig-download's own sink for a route finalizing into<cache>/modules/outside the ten enumerated.promote_into_cacheandwrite_atomicare the only writers I traced there — by grep and read, not a call-graph tool.Relayeduntil evicted.