Repository navigation
Conversation
stellar-rpc rebuilds its tx-hash window index while it ingests ledgers, and the build's workers share cores, caches and memory bandwidth with ingest. On a 32-vCPU box, ingest loses about 20 to 26 ms of p99 beside the build, and fewer workers do not win it back. Ingest is bursty, and a block is about half a millisecond of one worker, so the build can run in the gaps: with a gate that the ingest loop holds while it ingests a ledger, passed as WithYield(gate.Wait), ingest comes back to within 0.3 to 4 ms of running alone, while the build runs at 16 workers and 99M keys/s. WithYield(yield) makes the goroutine about to build a block call yield first, in every build mode, inline or parallel, empty blocks included. A yield that blocks holds the build back, one that returns an error fails the build with it, and WithYield(nil) is the same as not passing the option. The workers yield in the block build they share, so a yield error is the block's result and reaches the writer like a build error; the unsorted builder's parallel finish waits on the writer while it reads partitions. A failed block's result is no longer sent once the pipeline's context is done, since a yield that returns its context's error would otherwise stand in for a failure that skipped the writer: 117 of 200 four-worker builds over a sorted file with a record out of order reported context.Canceled. Without a yield the cost is a nil check per block, and the builder benchmarks do not move. The tests cover every mode, the unsorted builder's spill to disk included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DjQed1VX9uGU9PWpAjBiz7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
stellar-rpc rebuilds its tx-hash window index in the same process that ingests ledgers. The build's workers share cores, caches and memory bandwidth with ingestion, and ingestion's tail latency grows beside the build however few workers it has. But ingestion works in bursts, one per ledger, and a Bijection block of about 3,000 keys takes one worker about half a millisecond. So the build can run in the gaps, if the caller can pause it between blocks.
Before each block, empty blocks included, the goroutine about to build it calls
yieldand goes on when it returns. That holds in every build mode: sorted and unsorted, one worker or several, andBuildFromSortedFiles.yieldthat blocks holds the build back. The rest of the build stops behind the parked workers: what feeds them stops once their queue is full, and the writer once it has committed the blocks already solved. A parked build holds only its buffers and open files.yieldthat returns an error fails the build with that error, as a block that fails to build does.yieldthat waits on it returns then.WithYield(nil)is the same as not passing the option. Without a yield, the cost is a nil check per block, and the builder benchmarks don't move.A failed block's result is no longer sent to the writer once the pipeline's context is done. Otherwise a yield that returns its context's error stands in for the real failure: with one, 117 of 200 four-worker builds over a sorted file with a record out of order reported
context.Canceledinstead of the out-of-order error.The option is §6.4 of the spec. The tests cover every build mode, the unsorted builder's spill to disk included.
In stellar-rpc, the ingest loop holds a gate while it ingests a ledger, and the build gets
WithYield(gate.Wait). At sac-9000 load, with diagnostic events stripped from the ledgers, ingestion's p99 then stays within about 4 ms of its value alone while the build runs 16 workers at 99.4M keys/s. Without the gate, an 8-worker build adds about 26 ms to it.🤖 Generated with Claude Code