Repository navigation
DDIR: a demonstration of durable ledger logic - #1084
Draft
frankmcsherry wants to merge 6 commits into
Draft
frankmcsherry wants to merge 6 commits into
frankmcsherry wants to merge 6 commits into
Conversation
A ledger is a growing collection of updates, recorded as spans, that actors change only by transactions. interactive/model.md states the model: ledgers and spans, clocks, the invariants, transactions resolved in rounds, and what committing and reading must guarantee. The model runs. model/ledger.ddp is the state, the write frontier and the invariants. model/resolve.ddp is the rules for one round: Create and Append actions, sequence numbers, clocks, and rejections that say where a ledger stands. tests/ledgers.rs checks the two against each other. Both run as ordinary programs on the stock server, shown by three demos under server/demo. bench/ledger/fabric.py drives rounds from outside, makes each durable with a blob and a root swap, rebuilds from that log, and records what it measured in bench/ledger/README.md. The one change to DDIR itself is a set of frontier functions for lambdas: comparison, join, meet and canonical form over lists of lists of ints. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4EnzBnYpvwz5E2ZGmD
The recorded sweep used power-of-two round sizes, which run 20 to 30% slow. At those sizes the workers' merges fall out of step, and each round waits for one worker. Every figure is now from its own run on a fresh log, over 128 rounds, at sizes that avoid the effect. The README says what the effect is and where it comes from, and adds a table by worker count. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4EnzBnYpvwz5E2ZGmD
`frontier_eq` and `frontier_meet` go: nothing calls them. `server/demo/ledger_rounds.txt` goes: the test runs the same rounds, and the harness shows how a round is driven. The comment in `lambda.rs` keeps its wrapping. `model.md` now says plainly that this is a model and not an implementation, and ends with what comes next and what is open. `resolve.ddp` explains its two recurring idioms. The bench README records where the workers' time goes. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4EnzBnYpvwz5E2ZGmD
frankmcsherry
changed the title
Everything the ledger demonstration adds now lives in `interactive/examples/ledger/`: the model, its two programs, the harness and its two programs, the measurements, the session and the nine-session script. A README there says what each file is for, why there are four programs, how a round goes, and how to run each piece. Outside it are only the test, which cargo wants under `tests/`, the frontier functions, and one line in the DDIR README. The server README is as it was. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4EnzBnYpvwz5E2ZGmD
The demonstration is a work in progress, and not yet something to point readers at. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4EnzBnYpvwz5E2ZGmD
The rules in `resolve.ddp` are reworked to be shorter and to do less work each round. They are 231 lines, down from 290. What a request says about itself is no longer checked by the rules. Those checks move, whole, to `check.ddp`, which reads no state and is for a client to run before it submits. `tests/ledgers.rs` runs it in front of the rules. The decision judges each action against the ledger as its own transaction's earlier actions would leave it. So a transaction's verdict does not depend on whether it now counts as rejected, and a request that skips the checks cannot stop a round. A new test covers that. A new span's `since` is written as its `lower`, which marks it as not compacted. This removes a lookup of each ledger's start. The clocks a round needs are climbed once, from the clocks, and not from each touched ledger. A clock that many touched ledgers follow is met by (clock, part), so that its followers do not all land on one worker. The rules no longer judge an append's width against its ledger. A checked append of another width is told it is stale, where it was told it is invalid. That is the only change in what any request is told. What words a rejection is now the last section of the file, and changes nothing that commits. At one million ledgers under one clock, four workers, 25,600 appends a round: 522k spans committed a second becomes 667k. The tables in `MEASUREMENTS.md` predate this and are marked so. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4EnzBnYpvwz5E2ZGmD
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A demonstration of durable ledger logic in DDIR, which demonstrates what could happen using DDIR as the transaction resolution mechanism, and an external durability hook. Currently just the logic and a test harness, rather than a full integration.