refactor: move next-block planning and the boundary fee cache out of the public calls simulator - #25393
Open
spalladino wants to merge 1 commit into
Open
refactor: move next-block planning and the boundary fee cache out of the public calls simulator#25393spalladino wants to merge 1 commit into
spalladino wants to merge 1 commit into
Conversation
This was referenced Sep 2, 2026
spalladino
force-pushed
the
spl/next-block-predictor
branch
2 times, most recently
from
September 2, 2026 23:22
61ffbb7 to
9cd2b10
Compare
spalladino
added a commit
that referenced
this pull request
Sep 7, 2026
Fixes #25344. Fixes A-1885. Part of A-1903. Design: https://claude.ai/code/artifact/e9b78bd2-501d-4c30-a81a-5a10cbeed392 ## The symptom On a fresh sandbox, a wallet asks the node what fee to pay, pads it by 50%, and sends the transaction to `simulatePublicCalls`. Sometimes the node's own simulation rejects it: ``` maxFeesPerGas.feePerL2Gas must be greater than or equal to gasFees.feePerL2Gas, but got maxFeesPerGas.feePerL2Gas=1058030306 and gasFees.feePerL2Gas=3415500000 ``` The wallet paid exactly what the node told it to pay. ## Where the fee number comes from Both the wallet quote and the simulation ask the L1 rollup contract the same question — *"what is the minimum mana fee for a block in slot X?"* (`Rollup.getManaMinFeeAt`). The answer depends on an L1 gas oracle updated when checkpoints are proposed, and the new value only kicks in a couple of slots later, so the answer is a **step function of the slot**. While the sandbox's anvil base fee is decaying (the first minutes after start) each step is a large drop — in the issue, 3,415,500,000 → 920,600,000. The two sides therefore only agree if they ask about the **same slot**. They didn't. ## The bug: the simulator could target a slot that was already taken The fee quote (`FeeProviderImpl`) picks its slot as `max(slot of the latest checkpoint on L1 + 1, next slot by the node clock)` — anchored to **L1**. The simulator (`NodePublicCallsSimulator.computeTargetSlot`) picked `max(next slot by the node clock + pipelining offset, slot of the locally proposed checkpoint + 1)` — anchored to the **node clock**, and that second term disappears once the archiver promotes the proposed checkpoint to checkpointed. Worked example (72s slots, the oracle steps at slot 15): - The sandbox builds checkpoint 14, sends it to L1, anvil mines it. L1 now says: latest checkpoint is at slot 14. The node's clock has not been bumped yet — the automine sequencer only advances it at the very end of its publish routine. - Fee quote: the latest L1 checkpoint is at slot 14, so the next block is slot 15 at the earliest → fee for slot 15 → cheap (post-step) → the wallet declares 1.5x that. - Simulator: the node clock says the next slot is 13, plus the pipelining offset → slot 14. There is no proposed checkpoint any more (already promoted) → fee for slot 14 → expensive (pre-step). - expensive > 1.5 x cheap → the assertion fires. Slot 14 is nonsense for the simulator to target: a checkpoint already exists there on L1, so the next block can only land in slot 15 or later. The simulator didn't know, because it trusted its clock over the chain. ## The fix `computeTargetSlot` gains a third term in its `max`: **slot of the latest checkpointed checkpoint + 1**. - The slot is read from the checkpointed tip's block header **by the tip's block hash**, not by its number, so a checkpoint unwind that replaces the block at that number cannot silently answer with a different block's slot. A miss means the archiver no longer holds the block its own tips name — a torn snapshot — and throws a retryable error rather than dropping the floor. - The term never lowers a correct answer: when the clock is ahead, as it normally is, the clock term is already larger. It only binds when the clock is behind the chain, which is exactly the broken case. In the example above, `max(14, 15) = 15` — the same slot the quote used. - Before the first checkpoint lands the checkpointed tip is the genesis block, which the archiver does not store; there is no slot taken yet, so the floor is skipped rather than failing the simulation. Nothing else changes: no new L1 call on any path, no fee RPC touched, and transaction admission (`isValidTx`, the p2p validators) is untouched. ## What this does not fix - **The residual L1-poller window.** The fee provider and the archiver each run their own L1 poll, so they can briefly hold different views of which checkpoints exist on L1, and the quote and the simulation can still disagree across that window. A follow-up PR drives the fee provider from the archiver's L1 sync point and pins every fee read to that L1 block, which closes it. - **The frozen mid-checkpoint fee (Bug 2).** All blocks in a checkpoint share the fee frozen into its first block. The simulator honours that (it copies the latest proposed block's header); the quote only looks at forward-looking L1 projections and never sees the frozen value, so a correctly-priced transaction can still fail simulation on a network with multi-block checkpoints when fees are falling fast. This is not reachable on the sandbox (one block per checkpoint) and was not the reported issue. A later PR in this stack makes the quote lead with the fee the next block will actually charge, which fixes it. ## Tests - `aztec-node/src/aztec-node/fee_quote_vs_simulation.integration.test.ts` (new): real anvil, real L1 contracts, `RollupContract`, `EpochCache`, `GlobalVariableBuilder`, `FeeProviderImpl`, `AztecNodeService`/`NodePublicCallsSimulator`, and a real `PublicProcessor` on a real world-state fork; only the archiver is mocked. It steps the oracle 1000 gwei → 1 gwei (~1000x fee step) and plants the L1 pending checkpoint at the slot before the step via storage cheats. Two cases: a lagging node clock (reproduces the exact assertion from the issue without the fix) and nothing lagging (quote and simulation already agree, and must keep agreeing). - Unit tests in `node_public_calls_simulator.test.ts`: the floor binds when the clock lags; it does not raise the slot when the clock is ahead; the checkpointed tip is read by hash and not by number; a missing tip block throws a retryable error; the floor is skipped at genesis. - `RollupCheatCodes.setPendingCheckpoint` extracted from `fee_predictor.test.ts` so both suites plant a pending checkpoint the same way. ## Stack #25384 (atomic archiver `L2Frontier` snapshot, A-1897) is stacked on top of this PR. It replaces the by-hash tip-header read added here with a field of the snapshot, so the slot and the overrides plan come from one atomic archiver read. This is the bottom PR of the five-PR fee-quote / public-simulation series (GitHub stack #25385), merged in order: - #25357 — slot floor (this PR, A-1885) - #25384 — atomic archiver `L2Frontier` snapshot (A-1897) - #25392 — fee provider driven by the archiver's L1 sync point (A-1904) - #25393 — next-block predictor, planning and fee cache out of the simulator (A-1905) - #25394 — quote leads with the fee the next block will charge (A-1906)
spalladino
force-pushed
the
spl/next-block-predictor
branch
from
September 7, 2026 17:10
9cd2b10 to
4b8f05e
Compare
Base automatically changed from
spl/fee-provider-archiver-sync
to
merge-train/spartan-v5
September 7, 2026 18:24
…the public calls simulator Three components each formed their own idea of the next block, and the public calls simulator hit L1 on every request at a checkpoint boundary, so abusive L2 RPC traffic turned into pressure on the node's L1 RPC. Splits that work into aztec-node/src/aztec-node/next_block/: a pure planner (continues-vs-opens, the three-term target slot, the boundary fee key), a NextBlockFeeCache that owns the one L1-derived value with a single-flight background refresh, and a NextBlockPredictor facade the simulator consumes. The plan is re-derived from a fresh archiver snapshot per request; only the boundary fee is cached, keyed logically so a new L1 block alone is a hit and a miss means a real transition. The predictor is built and started by the node factory like every other service. The simulator now syncs the world state to the plan's block by number and hash, replanning once if a prune moved the chain underneath, then forks, inserts messages, and executes. No behaviour change for RPC callers beyond the removed per-request L1 calls; the sequencer is untouched.
spalladino
force-pushed
the
spl/next-block-predictor
branch
from
September 7, 2026 18:24
4b8f05e to
ac99757
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes A-1905. Part of A-1903.
Design: https://claude.ai/code/artifact/e9b78bd2-501d-4c30-a81a-5a10cbeed392
Stacked on #25392. Fourth PR of the fee-quote series. No
behaviour change for RPC callers except that they no longer cause an L1 call per request.
The structural problem
Three components each formed their own idea of "the next block and its fee": the sequencer, the L1-only fee
provider, and the public calls simulator. The simulator's copy lived inline in
simulate, and at a checkpointboundary it priced the block by calling L1 on every request. An abusive L2 RPC user could therefore turn
simulatePublicCallstraffic into pressure on the node's own L1 RPC — the opposite of what an RPC node shoulddo with a shared, rate-limited upstream.
The decomposition
A new
aztec-node/src/aztec-node/next_block/module, consumed only by RPC-side code:next_block_planner.ts— pure functions.planNextBlock(frontier, clockSlot)decides whether the nextblock continues the in-progress checkpoint or opens a fresh one, computes the three-term target slot (this
node's clock, the proposed parent's slot + 1, the checkpointed tip's slot + 1 as a floor), the target
checkpoint, and the block the plan builds on.
computeBoundaryFeeKeyturns a checkpoint-opening plan into atyped key. No I/O.
next_block_fee_cache.ts—NextBlockFeeCacheowns the one L1-derived value: the checkpoint globals ablock opening a fresh checkpoint would carry. It keeps the current and previous records, refreshes them on a
RunningPromiseloop, and builds the sameSimulationOverridesPlanthe sequencer applies (moved here fromthe simulator, no-rollup-contract fallback included).
next_block_predictor.ts—NextBlockPredictor, the façade.predict()returns the plan, the frontierit came from, and the globals the next block would carry.
quoteMinFees()returns the fee that block willcharge. Both re-derive the plan; only the fee comes from the cache.
The simulator shrinks to what it should be: check the gas limit, ask the predictor, sync the world state to the
block the plan builds on (by number and hash, replanning once if a prune moved the chain underneath), fork it,
insert the L1-to-L2 messages a checkpoint-opening block would see, run the processor.
The predictor is constructed and started in
createAztecNodeServicelike every other node service, and injectedinto
AztecNodeService, which only stops it.Why the plan is re-derived and only the fee is cached
The next block combines two clocks: rapidly moving L2 execution state and much slower L1 fee state. Caching a
whole
{ plan, globals }object would freeze the fork block, the target checkpoint (so the wrong checkpoint'sL1-to-L2 messages get appended), and the mid-checkpoint frozen-fee detection, for up to one refresh interval.
Worked example, 1s refresh, fee step 3,415.5M → 920.6M. At t=0.00 the loop caches "opens checkpoint 8 at slot
23, fee 920.6M". At t=0.40 block 42 arrives over gossip: the first block of checkpoint 8, built at slot 22
with 3,415.5M frozen into its header. A request at t=0.55 served from a cached plan answers 920.6M — wrong;
the wallet pads to 1,380.9M and the tx cannot enter block 43. With a fresh plan the node sees it is
mid-checkpoint, copies the header, answers 3,415.5M, and involves no L1 at all. Archiver-driven changes carry
the largest fee jumps, so those must be seen instantly; L1-driven changes lag by at most one refresh, the same
lag the fee provider already has.
Re-planning is cheap: the frontier is one in-memory read from the archiver's cache, the clock slot is
arithmetic, and the mid-checkpoint header arrives with the frontier.
The cache key
Records are looked up by a logical key: target slot, checkpointed checkpoint number, the block the plan builds
on, and either the proposed parent's fee-relevant fields (header hash, archive root, checkpoint out hash,
total mana used, fee asset price modifier) or the pending chain's validity. The L1 block a record was priced at
is stored with it for pinning, but is deliberately not part of the lookup: the min fee for a fixed slot and
parent depends only on rollup storage, and the rollup transactions that move it (checkpoints landing,
invalidations, prunes) all move the frontier and therefore the key. So a new L1 block on its own is a hit, and a
miss means a real transition — a slot rollover, a checkpoint landing or being proposed, a validity flip. The one
exception is a governance parameter update such as the mana target, which changes the fee without touching the
frontier; the background pass re-prices a matching record whenever the L1 anchor moves, so that lags by at most
one refresh interval, the same lag the fee provider has.
A pass that confirms nothing moved re-stamps the record instead of re-pricing it, which is what makes the
staleness cutoff mean "how long we have been unable to confirm" rather than expiring a value that is still
exactly right.
Single-flight and the wait policy
The rule the cache exists to enforce: the request path never originates an L1 call the background loop would
not make, and never has more than one in flight. A refresh is single-flight and shared between the loop and
every request, so a burst of requests during a transition costs one L1 round trip.
QUOTE_MAX_WAIT_MS) and never throws: on a timeout or failure it servesthe matching record if it is under the staleness cutoff, otherwise nothing. During an L1 outage the shared
call hangs until the L1 RPC timeout, and a quote that waits on it would become a multi-second RPC exactly
when users are already struggling.
always still right, but after a long outage the fee may have stepped and serving it would underquote.
The sequencer is untouched
NextBlockPredictorlives inaztec-nodeand is consumed only by RPC code. The sequencer computes its ownplan from the same frontier and the same overrides builder, with a deliberately different slot policy (it
declines to build when its clock slot is taken; the RPC side predicts inclusion and adds two floors), and must
never read this cache — a stale fee would make L1 reject its checkpoint. Extracting a shared "describe the next
checkpoint" helper is a reasonable follow-up, not part of this work.
Residuals
(inside
buildCheckpointSimulationOverridesPlan); only the final fee read is pinned to the frontier's syncpoint. Same as before this series, and shared with the sequencer. Pinning those reads means threading the block
number through the stdlib helper and is a follow-up.
provider until the archiver's next pass reconciles the frontier. The window is one archiver pass, and was the
same before this series when fees were read at
latest.(
l1HttpTimeoutMS) fires; the capped quote gives up on its own after 5s, so only simulations wait that long.What RPC callers observe
Nothing changes, except that repeated
simulatePublicCallsrequests at a checkpoint boundary no longer eachperform an L1 call.
getPredictedMinFeesis not touched here; the next PR makes the quote lead withquoteMinFees().Tests
next_block_planner.test.ts: continues vs opens; each of the three slot terms and the genesis case; the keyis undefined mid-checkpoint, stable when nothing moves, and moves on the target slot, the checkpointed
checkpoint, the latest block hash, a validity flip, a changed first-invalid checkpoint, a proposed parent
replacing the checkpointed one, and each proposed-parent fee field.
next_block_fee_cache.test.ts: skips the re-price when nothing moved; re-prices on a key move while stillserving the boundary it just left; re-prices in the background on an L1 anchor move without a reader miss;
one in-flight refresh shared by two readers and a loop pass; the capped wait serving a record under the
cutoff and nothing past it; the uncapped wait rethrowing; idempotent start; start surviving a failed priming
pass; the overrides plan for the idle, pipelined and invalid-chain shapes, with and without a rollup contract.
next_block_predictor.test.ts: the mid-checkpoint copy with no cache call, boundary globals with zero payoutaddresses, the missing-header rejection, and the three
quoteMinFeesoutcomes.node_public_calls_simulator.test.ts: rewritten against a mocked predictor — forks the block the plan buildson, hands the processor the predictor's globals, appends messages only at a boundary, replans once when the
sync reports a block-hash mismatch and fails with a retryable error when it persists, propagates other sync
failures untouched, plus the existing gas-limit and
L1ToL2MessagesNotReadyErrorcases.fee_quote_vs_simulation.integration.test.ts(anvil) keeps its three cases and gains one that fails beforethis change: two simulations with nothing moving in between price the boundary once, not twice.