Skip to content

Use filesystem JSONL for the append-only ledger - #62

Draft
sfloess wants to merge 13 commits into
mainfrom
feature/filesystem-append-only-ledger
Draft

sfloess wants to merge 13 commits into
mainfrom
feature/filesystem-append-only-ledger

Conversation

@sfloess

@sfloess sfloess commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Implements the first Ensemble filesystem-backed append-only ledger slice, using the POC JSONL adapter as the behavioral reference.

  • Adds a dedicated JSONL ledger store with cross-process file locking, fsync durability, stable ledger identity, sequence cursors, SHA-256 hash chaining, verification, idempotent import, conflict detection, and explicit legacy assurance.
  • Exposes append, pull, lookup, status, verify, and import REST endpoints under /api/v1/ledger.
  • Adds the shared contract fixture, storage tests, REST tests, and an optional bidirectional live HTTP transfer test/runner.
  • Documents the storage decision and recovery/concurrency rules.
  • Leaves Ensemble's existing SQLite-backed Memory, Vector, Index, Knowledge, Graph, GA, Thompson, Learning, and operational stores untouched.

The ledger file defaults to <ENSEMBLE_PERSISTENCE_ROOT>/ledger/events.jsonl; metadata and lock files are sidecars. No SQLite ledger table or migration is introduced.

Validation

GitHub Actions is running the full Python suite on the latest branch head. An earlier run found one test-import issue; that import was corrected in the latest commit. The live POC/Ensemble HTTP transfer test is intentionally skipped unless both service URLs are configured.

Sonar analysis is not part of this implementation gate.

@sonarqubecloud

sonarqubecloud Bot commented Nov 9, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant