fix: read archiver tips and proposed checkpoint as one atomic snapshot - #25384
Merged
spalladino merged 1 commit intoSep 7, 2026
Merged
Conversation
spalladino
force-pushed
the
spl/a-1897-chain-status-snapshot
branch
from
September 1, 2026 18:25
f8fb9a5 to
20ad5d2
Compare
spalladino
force-pushed
the
spl/a-1897-chain-status-snapshot
branch
from
September 1, 2026 19:46
b09de45 to
3f7f486
Compare
spalladino
force-pushed
the
spl/a-1897-chain-status-snapshot
branch
from
September 2, 2026 19:19
3f7f486 to
142a281
Compare
This was referenced Sep 2, 2026
spalladino
force-pushed
the
spl/a-1897-chain-status-snapshot
branch
from
September 2, 2026 23:22
142a281 to
83bbc06
Compare
alexghr
approved these changes
Sep 7, 2026
Base automatically changed from
spl/a-25344-fee-slot-floor
to
merge-train/spartan-v5
September 7, 2026 17:10
spalladino
added a commit
that referenced
this pull request
Sep 7, 2026
Fixes #25344. Fixes A-1885. Part of A-1903. Design: https://claude.ai/code/artifact/e9b78bd2-501d-4c30-a81a-5a10cbeed392 ## The symptom On a fresh sandbox, a wallet asks the node what fee to pay, pads it by 50%, and sends the transaction to `simulatePublicCalls`. Sometimes the node's own simulation rejects it: ``` maxFeesPerGas.feePerL2Gas must be greater than or equal to gasFees.feePerL2Gas, but got maxFeesPerGas.feePerL2Gas=1058030306 and gasFees.feePerL2Gas=3415500000 ``` The wallet paid exactly what the node told it to pay. ## Where the fee number comes from Both the wallet quote and the simulation ask the L1 rollup contract the same question — *"what is the minimum mana fee for a block in slot X?"* (`Rollup.getManaMinFeeAt`). The answer depends on an L1 gas oracle updated when checkpoints are proposed, and the new value only kicks in a couple of slots later, so the answer is a **step function of the slot**. While the sandbox's anvil base fee is decaying (the first minutes after start) each step is a large drop — in the issue, 3,415,500,000 → 920,600,000. The two sides therefore only agree if they ask about the **same slot**. They didn't. ## The bug: the simulator could target a slot that was already taken The fee quote (`FeeProviderImpl`) picks its slot as `max(slot of the latest checkpoint on L1 + 1, next slot by the node clock)` — anchored to **L1**. The simulator (`NodePublicCallsSimulator.computeTargetSlot`) picked `max(next slot by the node clock + pipelining offset, slot of the locally proposed checkpoint + 1)` — anchored to the **node clock**, and that second term disappears once the archiver promotes the proposed checkpoint to checkpointed. Worked example (72s slots, the oracle steps at slot 15): - The sandbox builds checkpoint 14, sends it to L1, anvil mines it. L1 now says: latest checkpoint is at slot 14. The node's clock has not been bumped yet — the automine sequencer only advances it at the very end of its publish routine. - Fee quote: the latest L1 checkpoint is at slot 14, so the next block is slot 15 at the earliest → fee for slot 15 → cheap (post-step) → the wallet declares 1.5x that. - Simulator: the node clock says the next slot is 13, plus the pipelining offset → slot 14. There is no proposed checkpoint any more (already promoted) → fee for slot 14 → expensive (pre-step). - expensive > 1.5 x cheap → the assertion fires. Slot 14 is nonsense for the simulator to target: a checkpoint already exists there on L1, so the next block can only land in slot 15 or later. The simulator didn't know, because it trusted its clock over the chain. ## The fix `computeTargetSlot` gains a third term in its `max`: **slot of the latest checkpointed checkpoint + 1**. - The slot is read from the checkpointed tip's block header **by the tip's block hash**, not by its number, so a checkpoint unwind that replaces the block at that number cannot silently answer with a different block's slot. A miss means the archiver no longer holds the block its own tips name — a torn snapshot — and throws a retryable error rather than dropping the floor. - The term never lowers a correct answer: when the clock is ahead, as it normally is, the clock term is already larger. It only binds when the clock is behind the chain, which is exactly the broken case. In the example above, `max(14, 15) = 15` — the same slot the quote used. - Before the first checkpoint lands the checkpointed tip is the genesis block, which the archiver does not store; there is no slot taken yet, so the floor is skipped rather than failing the simulation. Nothing else changes: no new L1 call on any path, no fee RPC touched, and transaction admission (`isValidTx`, the p2p validators) is untouched. ## What this does not fix - **The residual L1-poller window.** The fee provider and the archiver each run their own L1 poll, so they can briefly hold different views of which checkpoints exist on L1, and the quote and the simulation can still disagree across that window. A follow-up PR drives the fee provider from the archiver's L1 sync point and pins every fee read to that L1 block, which closes it. - **The frozen mid-checkpoint fee (Bug 2).** All blocks in a checkpoint share the fee frozen into its first block. The simulator honours that (it copies the latest proposed block's header); the quote only looks at forward-looking L1 projections and never sees the frozen value, so a correctly-priced transaction can still fail simulation on a network with multi-block checkpoints when fees are falling fast. This is not reachable on the sandbox (one block per checkpoint) and was not the reported issue. A later PR in this stack makes the quote lead with the fee the next block will actually charge, which fixes it. ## Tests - `aztec-node/src/aztec-node/fee_quote_vs_simulation.integration.test.ts` (new): real anvil, real L1 contracts, `RollupContract`, `EpochCache`, `GlobalVariableBuilder`, `FeeProviderImpl`, `AztecNodeService`/`NodePublicCallsSimulator`, and a real `PublicProcessor` on a real world-state fork; only the archiver is mocked. It steps the oracle 1000 gwei → 1 gwei (~1000x fee step) and plants the L1 pending checkpoint at the slot before the step via storage cheats. Two cases: a lagging node clock (reproduces the exact assertion from the issue without the fix) and nothing lagging (quote and simulation already agree, and must keep agreeing). - Unit tests in `node_public_calls_simulator.test.ts`: the floor binds when the clock lags; it does not raise the slot when the clock is ahead; the checkpointed tip is read by hash and not by number; a missing tip block throws a retryable error; the floor is skipped at genesis. - `RollupCheatCodes.setPendingCheckpoint` extracted from `fee_predictor.test.ts` so both suites plant a pending checkpoint the same way. ## Stack #25384 (atomic archiver `L2Frontier` snapshot, A-1897) is stacked on top of this PR. It replaces the by-hash tip-header read added here with a field of the snapshot, so the slot and the overrides plan come from one atomic archiver read. This is the bottom PR of the five-PR fee-quote / public-simulation series (GitHub stack #25385), merged in order: - #25357 — slot floor (this PR, A-1885) - #25384 — atomic archiver `L2Frontier` snapshot (A-1897) - #25392 — fee provider driven by the archiver's L1 sync point (A-1904) - #25393 — next-block predictor, planning and fee cache out of the simulator (A-1905) - #25394 — quote leads with the fee the next block will charge (A-1906)
Callers that plan the next block used to read the tips, the leading proposed checkpoint and the pending-chain validation status separately, so a concurrent archiver write could tear them apart. `L2BlockSource.getL2Frontier()` now serves all of that from one store transaction, cached in memory. The snapshot also carries the data the transaction already had in hand: the L1 sync point it is anchored to, the proposed tip's block header, and the latest checkpointed checkpoint's header plus its L1 publication data. The anchor moves at the start of a sync pass, before that pass's writes, so data can never be newer than the block it is attributed to. The public calls simulator and the sequencer's sync check now derive every decision from that one read: no by-hash header lookup for the mid-checkpoint fee copy or the slot floor, and no separate validation-status call.
spalladino
force-pushed
the
spl/a-1897-chain-status-snapshot
branch
from
September 7, 2026 17:10
83bbc06 to
1502c0b
Compare
spalladino
added a commit
that referenced
this pull request
Sep 7, 2026
Fixes A-1904. Part of A-1903. Design: https://claude.ai/code/artifact/e9b78bd2-501d-4c30-a81a-5a10cbeed392 Stacked on #25384. Third of five PRs on the fee-quote / public-simulation series. ## Problem The node ran two independent L1 pollers. `FeeProviderImpl` polled `getBlockNumber()` on its own schedule and the archiver polled L1 on its own, so for up to one poll interval the two held different views of the chain. Both answer the same underlying question — the minimum mana fee for a block in slot X — and the answer is a step function of the slot, so a disagreement about which checkpoints exist shows up as a wallet quote the node's own public simulation then rejects. Worked example. L1 mines checkpoint 14 at slot 41. The archiver's next pass runs 200ms later and its frontier now reports checkpoint 14 as the checkpointed tip, so the simulator prices the next block at slot 42. The fee provider's own poll has not fired yet, so its cached "current min fees" still describes the L1 block before checkpoint 14 landed and prices slot 41. If the L1 gas oracle stepped at slot 42, the wallet quotes the slot-41 fee, pads it, and the simulation charges the slot-42 fee and rejects the transaction. The same window exists in reverse when the fee provider polls first. On top of that, the provider read the "current" fee at `latest` even though it pinned the predictor's reads to a block number, so a single quote could mix two L1 blocks. ## Design - **Sync point source.** `L2BlockSource` gains `getL1SyncPoint()`, returning the same `{ blockNumber, blockHash }` that `getL2Frontier().l1SyncPoint` carries (added in #25384). The archiver serves it from the frontier cache without awaiting the snapshot promise, so following it costs a field read, not a store or L1 call. The fee provider depends on that one method only (`Pick<L2BlockSource, 'getL1SyncPoint'>`). - **Refresh on hash change.** `FeeProviderImpl` no longer polls an L1 head of its own. Its loop reads the sync point and refreshes only when the hash differs from the newest entry's — hash, not number, so a same-height L1 reorg still invalidates. Before the archiver's first pass the sync point is undefined and the provider falls back to L1's latest block, so a starting node can answer fee queries while the archiver catches up. - **Every read pinned.** `getPendingCheckpoint` and `getTimestampForSlot` gained an optional `{ blockNumber }`, and `getManaMinFeeAt` takes `{ stateOverride?, blockNumber? }` as a single options bag (threaded through `getCheckpointNumber` / `getCheckpoint`, guarded by `checkBlockTag` like the other pinned readers); the predictor's state read was already pinned. One refresh therefore describes exactly one L1 block. - **Ring of 4.** Each refresh appends `{ blockNumber, blockHash, currentMinFees, predictorState }`, newest first. `FeePredictor` no longer caches state internally: `computeState(blockNumber)` returns it and `computePredictions(state, manaUsage)` derives fees from it, so each retained view carries its own state and a tagged answer is computed from the state read at that block. - **Single-flight.** One in-flight refresh promise, shared by the loop and every waiting request. A burst of requests during a transition costs one round of L1 reads. - **`asOf` tagging.** `getCurrentMinFees(asOf?)` and `getPredictedMinFees(manaUsage?, asOf?)` take an optional `{ blockNumber, maxWaitMs? }`. Untagged serves the newest view. A tag the ring holds is served from that view even when a newer one exists. A tag ahead of every view awaits the shared refresh and, if the pass it joined had already started from an older sync point and so did not produce its block, one more pass of its own — both within `maxWaitMs` when set, falling through to the newest view on timeout — and then serves the match, or the newest view with a debug log. A tag behind the ring serves the oldest view with a debug log. A tagged miss never throws; only the initial `start()` refresh propagates errors. `getPredictedMinFees` still returns `[currentMinFees, ...predictions]`, both halves from the same entry. - **Who tags.** In this PR the only tagged caller is the public simulator, which passes its frontier's sync point when pricing a checkpoint boundary (`buildCheckpointGlobalVariables` gained a trailing `{ blockNumber }`). PR 4 adds the next-block predictor and PR 5 the fee quote. Admission — `isValidTx`, gossip validation, pool ingress and eviction — stays untagged and unchanged: it tolerates staleness and must not gain a wait. ## Residuals - The overrides plan built for the simulation (`buildSimulationOverridesStateOverride` and the pipelined parent fee-header reads inside `buildCheckpointSimulationOverridesPlan`) is still read unpinned. Those reads describe the proposed parent, which the frontier snapshot already pins logically, but they are not yet pinned to an L1 block number. - Fee freshness is now bounded by the archiver's poll interval plus the provider's, rather than the provider's alone. That is the point of the change — both sides move together — but it does mean an L1-driven oracle step reaches the quote slightly later than before. ## Tests - `fee_provider.test.ts` rewritten against a stub sync-point source and mocked contract/predictor: pinned reads (the block number passed to each contract call is asserted — here the pin is the behaviour), refresh only on hash change with no head poll, same-height reorg refreshes, L1-head fallback before the first archiver pass, tagged hit served past a newer view, tagged miss refreshing and serving, single-flight across two concurrent misses, capped wait serving the newest view, tag behind the ring serving the oldest, ring capped at four, start failing on a failed initial refresh, and a failed background refresh leaving the last view intact. - `node_public_calls_simulator.test.ts`: the boundary fee read is pinned to the frontier's sync point, and left unpinned when the archiver has not synced. - `fee_quote_vs_simulation.integration.test.ts` (anvil): the mocked archiver now reports a real sync point and the provider is built against it. Both existing cases stay green, and a new case lands a checkpoint on L1, mines past it, and shows the quote unchanged while the archiver has not run a pass, then moving to the new fee together with the simulation once the sync point advances. This is what proves pinned `eth_call` with a `stateOverride` works against anvil at a recent block. - `fee_predictor.test.ts` updated to the new `computeState` / `computePredictions` API; the state-caching block is gone with the cache. - `archiver-sync.test.ts` asserts `getL1SyncPoint()` matches the frontier's field after a sync pass; `stdlib/src/interfaces/archiver.test.ts` round-trips the new RPC method. Next PR moves next-block planning and the boundary fee cache out of the simulator.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes A-1897. Part of A-1903.
Design: https://claude.ai/code/artifact/e9b78bd2-501d-4c30-a81a-5a10cbeed392
Planning the next block needs several facts about the same instant: the L2 tips, the leading proposed (not-yet-L1-confirmed) checkpoint, the pending-chain validation status, and the headers those tips name. They came from separate reads with different backing —
getL2Tips()served from the in-memoryL2TipsCache,getProposedCheckpointData()andgetPendingChainValidationStatus()read straight from the store, headers fetched by a further lookup — so an archiver write landing between them produced a torn set. The typical shape: a checkpoint that has just been promoted is gone from the proposed map while the cache still reports the pre-promotion tips, so the next block is classified as continuing an in-progress checkpoint and gets mis-priced.Design
BlockStore.getL2Frontier(genesisBlockHash)resolves the four tips, the leading proposed checkpoint, the proposed tip'sBlockHeader, the latest checkpointed checkpoint (CheckpointHeader+ itsL1PublishedData), and the pending-chain validation status inside a singledb.transactionAsync, which is a consistent LMDB read snapshot. It replacesgetL2TipsData, reusing its body. Every added field comes from bytes the transaction already loaded: the latest block entry carries its header, the latest checkpoint entry carries the header the call already deserialized for the tip plus its L1 buffer, and the validation status is one singleton read.L2TipsCachebecomesL2FrontierCache: it holds the whole snapshot behind one promise, refreshed at exactly the points the tips cache refreshed before (everyArchiverDataStoreUpdaterwrite, after the transaction commits).getL2Tips()stays and derives from the snapshot. Because readers get one immutable object, refresh ordering versus commit no longer affects consistency — only freshness — so a plain post-commit refresh suffices.l1SyncPoint: { blockNumber, blockHash }, the L1 block whose state its data reflects.L1Synchronizer.syncFromL1sets it at the top of a pass, right after the no-new-block early return and before any of that pass's writes, viaL2FrontierCache.setL1SyncPoint(which rewrites the cached object with no store read). The ordering is the invariant: the pass's writes reflect L1 state up to that block, so the anchor must move ahead of the data and never behind it. A reader that saw data from L1 block N under an anchor of N-1 would price a fee at N-1 while the data already includes a checkpoint that landed at N. Data behind the anchor is harmless, because the overrides plan derived from the snapshot fully describes the parent.L2BlockSource.getL2Frontier()is the new interface method, with theL2Frontiertype and zod schema inl2_block_source.tsand an entry inArchiverApiSchemaso the whole snapshot round-trips over JSON-RPC.getCheckpointedTipSlot(frontier)is a free helper over the checkpointed checkpoint's header, so the slot has one source rather than a duplicated field.Consumers migrated
NodePublicCallsSimulatormakes onegetL2Frontier()call and performs no block lookup at all. The mid-checkpoint globals copy readsfrontier.latestBlockHeaderinstead ofgetBlockData({ number }); a missing header at a non-genesis proposed tip is now an invariant violation and throws, rather than falling through to the boundary path and double-inserting the ongoing checkpoint's L1-to-L2 messages. The slot floor reads the checkpointed checkpoint's header slot from the same snapshot, deleting the by-hash header read fix: price public simulation at a slot L1 has not already taken #25357 added.buildSimulationOverridesPlanreadsfrontier.pendingChainValidationStatusinstead of callinggetPendingChainValidationStatus().checkSynctakes tips, proposed checkpoint and validation status from onegetL2Frontier()instead of three reads in the samePromise.all; the orphan-block guard now compares the block against a pair from the same snapshot, and the comment claiming there was no split read to reconcile is corrected.checkSyncalso now looks the tip block up by hash instead of by number: the tips come from the cached snapshot, so after a prune-and-replace commits (but before the cache refreshes) a by-number read could return the replacement block while every hash check in the function still described the pruned one (pre-existing, surfaced by review). The archiver'sgetBlockDataresolves a hash matching the initial header to the genesis sentinel, so the genesis case still works.automine_sequencer.tsreplaces its unguardedPromise.all([getL2Tips(), getProposedCheckpointData()])with onegetL2Frontier().TXEArchiverandMockL2BlockSourceimplement the new method directly (neither ever holds a proposed checkpoint, and both report a valid pending chain and no L1 sync point).Tests
block_store.test.ts:getL2Frontieron an empty store (no headers, valid pending chain), the tips / proposed checkpoint / block header / checkpointed header + L1 data together, the validation status read in the same snapshot, and a promotion moving the checkpointed tip and dropping the proposed entry within one snapshot.data_store_updater.test.ts: a reader that lands in the window after a promotion commits but before the cache refreshes gets a self-consistent snapshot, header included. Verified red — with the pair split back apart the frontier and the proposed tip disagree (Expected: 1, Received: 0) — and green with the atomic read.l2_frontier_cache.test.ts: warm reads do not hit the store; a refresh reloads once; a sync point set before or after the first load is attached without a store read and survives a refresh; each update yields a fresh object, so a snapshot a reader already holds cannot change under it.archiver-sync.test.ts: the anchor-before-writes invariant, asserted over a real synchronizer and store. Verified red by movingsetL1SyncPointto the end ofsyncFromL1(Expected: < 18, Received: 23) and green with the call in place.node_public_calls_simulator.test.ts: rewritten around amakeFrontierhelper. The floor tests keep their coverage but now read the slot from the snapshot; the by-hash-lookup and torn-snapshot tests are replaced by "no block read happens at all" and "a snapshot missing the proposed tip header is rejected"; genesis still lets the clock win because no checkpoint has landed yet.archiver.test.tsin stdlib covers the JSON-RPC round-trip for every new field (bigint andBuffer32included).Notes
merge-train/spartan-v5automatically once fix: price public simulation at a slot L1 has not already taken #25357 merges.FeeProviderImplfroml1SyncPoint: it stops polling L1 for its own head, refreshes when the archiver's anchor advances, and pins every fee read to that block, so the quote and the simulation can no longer disagree about which checkpoints exist on L1.l1SyncPointis set when a sync pass starts, before any write, so the L2 data is never ahead of its anchor. The other direction is possible while a pass is in flight, and during a same-height L1 reorg the anchor points at the new fork until that pass reconciles the frontier; a pass that fails part-way leaves the anchor at the new block, and the next pass retries since the "no new L1 block" early return is only armed on success. That window was the same before this series, when fees were read atlatest.L2TipsCache.refreshAfter), now closed.getL2Frontierunconditionally, so a new node against an older remote archiver gets method-not-found until the archiver is upgraded. Upgrade the archiver first (or together, as our deployments do); a client-side fallback to the old read pair would reintroduce the torn snapshot this PR removes, so none is provided.