diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 6e66e6fa..00e41afe 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -182,6 +182,15 @@ The facade is orchestration glue. It is not the storage engine itself. bundles build deterministic bounded-fanout trees and support targeted member traversal without hydrating the complete structure. +- **`StagingWorkspaceRegistry` and `StagingWorkspace`** — own renewable + temporary RootSet generations for multi-step application construction. + `WorkspaceCompoundAdmission` and `WorkspaceCompoundScope` serialize an + explicitly bounded sequence of provisional page and bundle batches through + one operation-owned persistence view, then install the union of prior and new + targets in one exact generation. Existing workspace methods retain each + result independently and remain the boundary when a handle leaves private + construction code before later writes begin. + - **`RetentionService` and `PublicationService`** — validate complete handle graphs, then either retain them in a RootSet with generation-scoped evidence or publish them atomically under an allowlisted application ref. diff --git a/CHANGELOG.md b/CHANGELOG.md index dad64263..38b19dbe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **Compound staging-workspace admission** - `workspace.batch()` runs an + explicitly operation-bounded sequence of dependent page and ordered-bundle + batches through one private persistence scope, then returns the callback + value only after one exact workspace generation retains every staged handle. + Scope calls serialize by invocation order, return frozen handle arrays, stop + queued work after failure, and preserve the prior generation on refusal. + +### Performance + +- **One retained generation for dependent write waves** - a clean five-sample + SHA-1/SHA-256 witness reduced a 33-operation, 81-handle graph from 200 to 23 + Git child processes and from 33 workspace commits and checked ref updates to + one, with identical application-handle digests. Median wall time fell by + 80.5% on the measured host. Compound admission changes workspace-ref update + frequency, while object bytes, handle identity, ref layout and namespaces, + readers, and existing independently retained workspace methods remain + compatible. + ## [6.5.8] — 2026-08-23 ### Added diff --git a/GUIDE.md b/GUIDE.md index 9976f6d0..8e01beb4 100644 --- a/GUIDE.md +++ b/GUIDE.md @@ -202,6 +202,31 @@ no partial result array on failure. Higher `maxBatchAssets` values can reduce protocol round trips when the caller can afford more simultaneously live source pipelines; the default remains four. +When later bundle waves depend on handles created by earlier waves, keep the +intermediate handles inside one compound staging-workspace operation: + +```js +const admitted = await workspace.batch({ + maxOperations: 3, + operation: async (scope) => { + const pages = await scope.pages.putBatch({ pages: pageRequests }); + const leaves = await scope.bundles.putOrderedBatch({ + bundles: leafRequests(pages), + }); + return (await scope.bundles.putOrderedBatch({ + bundles: [rootRequest(leaves)], + }))[0]; + }, +}); +``` + +`admitted.value` becomes caller-visible only with `admitted.retention`, after +one exact workspace generation anchors every staged handle. The default +operation ceiling is 64 and the hard ceiling is 1,024; each scope call also +preserves its ordinary page or bundle batch bounds. Use the existing +independently retained workspace methods when intermediate handles must leave +the private callback. + Repeated `pages.get()` calls reuse immutable payload reads within the store's bounded page cache. The defaults retain at most 128 payloads and 8 MiB; use `pageCacheEntries` and `pageCacheBytes` to tune that ceiling. Every result is a diff --git a/README.md b/README.md index 52fcb79e..b575d10e 100644 --- a/README.md +++ b/README.md @@ -41,8 +41,9 @@ Unlike traditional LFS which moves files to external servers, `git-cas` treats t blob reads at or below a fixed 10 MiB ceiling use one bounded session read; larger payloads retain the genuine one-shot streaming path. Explicitly bounded page, asset, and ordered-bundle batches pipeline independent Git - writes and retain each successful workspace batch under one exact generation - without changing content identity. + writes. `workspace.batch()` can compose dependent page and bundle waves in + one private persistence scope and retain their union under one exact final + generation without changing content identity. - **Key Lifecycle**: Envelope encryption separates DEKs from KEKs. Rotate passphrases across an entire vault without re-encrypting data blobs. Privacy mode HMAC-hashes slug names to prevent metadata discovery. - **Runtime-Adaptive**: A single core supports Node.js 22+, Bun, and Deno through a strict hexagonal port architecture with runtime-specific crypto adapters. @@ -157,9 +158,10 @@ Core capabilities: compare-and-swap refs, and immutable lifecycle evidence. - **Scoped staging workspaces**: `workspaces.open()` mirrors application writes behind one renewable temporary RootSet, supports one-generation bounded - asset/page/bundle batches, returns only after each handle is anchored, promotes - destination-first, and exposes bounded age, expiry, logical-content, and - direct-root diagnostics with opaque cleanup pagination. + asset/page/bundle batches plus compound dependent page/bundle admission, + returns only after each public result is anchored, promotes destination-first, + and exposes bounded age, expiry, logical-content, and direct-root diagnostics + with opaque cleanup pagination. - **Envelope recipients**: multi-recipient key wrapping and recipient rotation avoid re-encrypting data blobs. - **Operational diagnostics**: `cas.diagnostics.doctor()` streams repository diff --git a/UPGRADING.md b/UPGRADING.md index c63fb7cf..d2e4ec01 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -2,6 +2,45 @@ v6.0.0 is a major release that simplifies the encryption model, hardens security defaults, and cleans up the architecture. This guide covers every breaking change and what you need to do. +## v6.5.8 To v6.5.9 + +v6.5.9 adds `workspace.batch()` and requires no application or stored-data +migration. Existing handles, object bytes, workspace descriptors, ref +namespaces, retention witnesses, read paths, and independently retained +workspace methods remain compatible. Existing repositories open in place. + +Use the new method only when intermediate page and bundle handles remain +private to one bounded construction: + +```js +const admitted = await workspace.batch({ + maxOperations: 3, + operation: async (scope) => { + const pages = await scope.pages.putBatch({ pages: pageRequests }); + const leaves = await scope.bundles.putOrderedBatch({ + bundles: buildLeafRequests(pages), + }); + return (await scope.bundles.putOrderedBatch({ + bundles: [buildRootRequest(leaves)], + }))[0]; + }, +}); + +admitted.value; // retained root BundleHandle +admitted.retention; // exact final workspace generation and witnesses +``` + +The operation defaults to at most 64 scope calls and cannot exceed 1,024. +Every page or bundle call retains its existing count, object, member, and byte +bounds. A callback or staged-write failure publishes no compound generation; +immutable objects written before failure remain unreachable for Git's normal +reclamation. Use the existing workspace methods when a handle must become +independently retained before arbitrary caller code observes it. + +The release changes physical admission cost only. It does not introduce a new +transaction format, migration command, authority cutover, or mixed-version +rewrite. + ## v6.5.7 To v6.5.8 v6.5.8 adds bounded application-write batches and requires no application or diff --git a/docs/API.md b/docs/API.md index 189f6ef4..a13874cd 100644 --- a/docs/API.md +++ b/docs/API.md @@ -1436,6 +1436,55 @@ await workspace.bundles.putOrdered(options); await workspace.bundles.putOrderedBatch(options); ``` +Dependent page and bundle waves can instead share one bounded compound +admission: + +```javascript +const admitted = await workspace.batch({ + maxOperations: 3, + operation: async (scope) => { + const pages = await scope.pages.putBatch({ pages: pageRequests }); + const leaves = await scope.bundles.putOrderedBatch({ + bundles: buildLeafRequests(pages), + }); + return (await scope.bundles.putOrderedBatch({ + bundles: [buildRootRequest(leaves)], + }))[0]; + }, +}); + +admitted.value; // BundleHandle returned by the callback +admitted.retention; // exact WorkspaceCheckpointResult +``` + +`batch({ operation, maxOperations })` calls `operation` exactly once. Its scope +exposes only `pages.putBatch()` and `bundles.putOrderedBatch()`. Both methods +retain their existing per-call limits and validation, serialize by invocation +order, and return frozen arrays of provisional handles rather than staged +result objects. The operation defaults to at most 64 scope calls and rejects a +limit above the exported hard maximum of 1,024. + +Success installs the union of previously retained targets and every compound +target in one exact workspace generation. The returned `value` is the callback +value; it becomes caller-visible only after scoped Git resources close and the +paired `retention` result proves the final generation. The scope closes before +the outer promise settles. Calling an escaped scope later fails with +`WORKSPACE_STATE_INVALID`. + +The callback is trusted application code, not a JavaScript capability sandbox. +The API cannot prevent a callback from assigning a provisional handle into +external state as a side effect. Such a handle has no compound retention +witness and must not be used outside the callback. Only the settled outer +result carries the admission guarantee. + +An empty operation, invalid bound, callback failure, staged-write failure, +session-close failure, or final checked-ref failure returns no admitted value. +Queued work stops after the first failure, the prior workspace generation does +not move, and distinct callback/write or operation/close failures remain +available through `AggregateError`. Immutable objects written before refusal +may remain unreachable for Git's ordinary reclamation. Compound admission is +not a cross-workspace, cross-ref, or arbitrary application transaction. + Each method returns only after a direct workspace generation reaches the returned typed handle. The result is otherwise the ordinary staged result plus a workspace `RetentionWitness`. Calls on one workspace serialize their ref diff --git a/docs/design/0060-compound-workspace-admission/compound-workspace-admission.md b/docs/design/0060-compound-workspace-admission/compound-workspace-admission.md new file mode 100644 index 00000000..36c39498 --- /dev/null +++ b/docs/design/0060-compound-workspace-admission/compound-workspace-admission.md @@ -0,0 +1,607 @@ +--- +title: 'PERF-0060 - Compound Workspace Admission' +cycle: '0060' +task_id: 'compound-workspace-admission' +legend: 'PERF' +release_home: 'v6.5.9' +issue: 'https://github.com/git-stunts/git-cas/issues/123' +goalpost_issue: 'https://github.com/git-stunts/git-cas/issues/123' +tracker_source: 'github' +status: 'active' +base_commit: '33738af7ce31f9117e9ced24ea20745a8541eea8' +owners: + - '@git-stunts' +sponsors: + human: 'James' + agent: 'Codex' +blocking_issues: [] +supersedes: [] +superseded_by: null +created: '2026-08-24' +updated: '2026-08-24' +--- + +# PERF-0060 - Compound Workspace Admission + +## Linked Issue + +- [#123 - Compound staging-workspace admission into one retained generation](https://github.com/git-stunts/git-cas/issues/123) + +## Linked Tracker + +- Milestone: [`v6.5.9`](https://github.com/git-stunts/git-cas/milestone/19) +- Goalpost issue: [#123](https://github.com/git-stunts/git-cas/issues/123) +- Downstream consumer: [`git-warp#852`](https://github.com/git-stunts/git-warp/pull/852) + +## Design Type + +This design is primarily: + +- [x] Runtime/API +- [x] Storage/substrate +- [x] Migration/release +- [ ] CLI/operator +- [x] Docs/public guidance +- [ ] TUI/visual surface +- [x] Test/tooling + +## Decision Summary + +`StagingWorkspace` will add one bounded compound-admission operation for +dependency-ordered page and bundle batches. Provisional content-addressed +handles are returned through the scope for private callback composition. The +outer promise will return the callback value plus exact retention evidence only +after one checked workspace generation anchors every staged target. The +callback is trusted code: JavaScript cannot prevent it from leaking a handle by +side effect, and such leakage carries no retention witness. The operation will +reuse one git-cas-owned persistence scope, serialize sub-operations in +invocation order, enforce an explicit operation ceiling, close +deterministically, and expose no Plumbing or Git session authority. Existing +staging APIs and persisted formats remain unchanged. + +## Sponsored Human + +An application operator wants dependency-ordered materialization to pay for one +temporary-retention publication rather than one publication per construction +wave, so that Think capture and reading spend time on causal work instead of +launching dozens of redundant Git processes, without weakening pruning safety +or changing retained data. + +## Sponsored Agent + +An agent needs a bounded, typed compound surface and exact generation evidence +so it can build a content-addressed dependency graph efficiently without +receiving an unretained handle through the outer result, inferring session +lifetime, or receiving raw Git process authority. + +## Hill + +By the end of this cycle, a caller can build several dependent page and bundle +waves inside one `StagingWorkspace` compound operation and receive its result +only after one exact RootSet generation retains every staged target. Real Git +tests and a SHA-1/SHA-256 witness prove identical handles, one generation, +failure containment, immediate-prune safety, bounded operation count, closed +sessions, and a materially smaller Git child census. + +## Current Truth + +- Each workspace page or bundle batch calls its underlying service and then + immediately installs the workspace's complete growing target set. This + correctly anchors returned handles but creates one RootSet generation per + dependency wave. + [cite: `src/domain/services/StagingWorkspace.js#59-78@33738af7ce31f9117e9ced24ea20745a8541eea8`] + [cite: `src/domain/services/StagingWorkspace.js#216-283@33738af7ce31f9117e9ced24ea20745a8541eea8`] +- Installation writes a new lease page, RootSet metadata, tree, parentless + commit, and checked ref update before updating the in-memory generation and + target set. + [cite: `src/domain/services/StagingWorkspace.js#285-330@33738af7ce31f9117e9ced24ea20745a8541eea8`] +- Page batches already accept an operation-owned persistence view. Bundle + batches use a private write scope and already route inline page batches + through the same scoped persistence. + [cite: `src/domain/services/PageService.js#70-140@33738af7ce31f9117e9ced24ea20745a8541eea8`] + [cite: `src/domain/services/BundleService.js#101-125@33738af7ce31f9117e9ced24ea20745a8541eea8`] + [cite: `src/domain/services/BundleService.js#430-449@33738af7ce31f9117e9ced24ea20745a8541eea8`] +- `GitPersistenceAdapter.withWriteScope()` owns deterministic session closure. + Its scope can preserve one fast-import process across dependent blob phases + while retiring stale mktree sessions whenever a new pack becomes visible. + [cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#160-177@33738af7ce31f9117e9ced24ea20745a8541eea8`] + [cite: `src/infrastructure/adapters/GitPersistenceWriteScope.js#8-95@33738af7ce31f9117e9ced24ea20745a8541eea8`] +- The public workspace declaration exposes only independently retained + singleton and batch calls plus checkpoint, renew, promotion, and release. It + has no compound scope. + [cite: `index.d.ts#1667-1708@33738af7ce31f9117e9ced24ea20745a8541eea8`] +- git-warp's current exact-head hosted benchmark reduces cold materialization + from 781 to 139 Git children and incremental materialization from 372 to 149, + with identical semantic fingerprints. The survivor census includes 60 + `hash-object`, 21 `commit-tree`, 22 `symbolic-ref`, and 21 checked + `update-ref` operations. These are downstream observations, not universal + git-cas costs. + +## Problem + +The safe public API makes each dependency wave independently authoritative. +That is necessary when a handle escapes to arbitrary caller code, but redundant +when all intermediate handles remain private to one bounded construction. A +dependency graph cannot be supplied as one static input because parent bundle +members require child OIDs that Git has not produced yet. Without a compound +scope, the caller must choose between repeated retention publication or unsafe +unanchored handles outside a declared operation. + +## Scope + +This cycle includes: + +- one generic workspace compound operation for page and ordered-bundle batch + construction; +- handle-only provisional results inside the callback; +- an explicit default and maximum sub-operation count; +- deterministic invocation-order serialization and failure poisoning; +- one operation-owned Git persistence scope; +- one final exact workspace generation for all newly staged and previously + retained targets; +- exact retention evidence paired with the callback value; +- unit, declaration, real-Git pruning, failure, lifecycle, and benchmark proof; +- public docs, architecture, changelog, design witness, and v6.5.9 release + evidence. + +## Non-Goals + +This cycle does not include: + +- cross-workspace or cross-ref transactions; +- arbitrary singleton asset composition inside the first compound profile; +- returning provisional staged objects as if they were already retained; +- a declarative git-warp trie or graph planner in git-cas; +- manual blob, tree, or commit OID derivation; +- a new storage format, ref namespace, descriptor version, or migration; +- weakening existing independently retained workspace methods; +- holding a process beyond the bounded callback and final installation. + +## Runtime / API Contract + +The public shape is conceptually: + +```ts +const admitted = await workspace.batch({ + maxOperations: 16, + operation: async (scope) => { + const pages = await scope.pages.putBatch(pageOptions); + const leaves = await scope.bundles.putOrderedBatch({ + bundles: pages.map((handle) => ({ members: [['leaf/data', handle]] })), + }); + const roots = await scope.bundles.putOrderedBatch({ + bundles: [{ members: leaves.map((handle, index) => [`child/${index}`, handle]) }], + }); + return roots[0]; + }, +}); + +admitted.value; // callback result, visible only after retention +admitted.retention; // exact WorkspaceCheckpointResult +``` + +Contract laws: + +1. `operation` is required and called exactly once. +2. `maxOperations` is a positive safe integer no larger than the exported hard + ceiling; each scope method invocation consumes one operation. The first + invocation past that ceiling poisons the admission immediately, and later + calls reuse that refusal without extending the bounded execution queue. +3. Scope methods are serialized by invocation order even if the callback starts + them concurrently. +4. Scope page and bundle methods preserve the existing per-call batch bounds, + input order, validation, handles, and object identity, but return only frozen + handle arrays. +5. The scope becomes closed before the outer promise settles. Any escaped scope + invocation rejects without writing. +6. An empty compound operation rejects without moving the workspace ref. +7. Success installs the union of prior workspace targets and all compound + targets exactly once. `retention` names that generation and its witnesses. +8. The callback value is not exposed unless object-session closure and exact + retention both succeed. +9. Callback or sub-operation failure poisons queued later work, closes the + object scope, preserves the previous workspace generation, and returns no + callback value or retained subset. +10. Immutable objects written before failure may remain unreachable for Git's + normal reclamation. That is not a partial admission. + +## User Experience / Product Shape + +There is no rendered interface. The user-visible surface is the package API, +typed declarations, stable error codes, release notes, and machine-readable +benchmark witness. + +## Data / State Model + +| State | Source of truth | Derived state | Invalid states | Reset behavior | Serialization | Determinism assumptions | +| -------------------------- | ------------------------------------------------- | --------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------ | -------------------------------------- | -------------------------------------------- | +| Prior workspace generation | checked workspace ref | in-memory target map | ref differs from expected generation | caller releases or retries with a new workspace | existing RootSet tree and lease page | exact expected-head authority | +| Open compound scope | in-process bounded operation | ordered provisional handle ledger | escaped use, excessive operations, queued work after failure | scope closes on success or failure | none | invocation order defines execution order | +| Provisional object graph | Git immutable objects plus staged-target evidence | handle-only callback values | handle cardinality/type mismatch | unreachable objects are reclaimable | existing blobs and bundle trees | existing page/bundle codecs define OIDs | +| Admitted compound result | one checked workspace generation | retention witnesses | callback result exposed without exact retention | release, checkpoint, or promotion follows existing law | unchanged workspace descriptor version | canonical target order and Git object format | + +## Architecture / Anti-SLUDGE Posture + +| Concern | Decision | +| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| Domain changes | Add a named compound-operation coordinator; keep service composition out of `StagingWorkspace` helper corridors. | +| Port changes | None below git-cas; the public workspace capability gains one semantic method. | +| Adapter changes | Reuse `GitPersistenceAdapter.withWriteScope()`; expose no adapter or session object publicly. | +| Boundary validation | Validate callback, operation limits, method lifecycle, result cardinality, and staged target evidence at the owning boundary. | +| Runtime-backed nouns introduced | A compound workspace scope and admitted result with explicit open/closed and retained semantics. | +| Expected failure representation | Existing `CasError` codes for invalid options, workspace state, and retention; preserve original storage failures and aggregate close failures. | +| Banned shortcuts avoided | No raw Git commands in domain code, no content-hash OID guessing, no boolean mode flags, no unbounded task queue, and no git-warp vocabulary. | + +## Cost / Residency Posture + +| Surface | Current cost | Target cost | Limit/budget | Failure mode | +| ------------------------------ | --------------------------------------- | ----------------------------------------- | --------------------------------------------------- | ------------------------------------------------ | +| Workspace generations | one per dependent batch | one per successful compound operation | one exact checked update | no generation movement on failure | +| Fast-import sessions | one per bounded service batch | one per compound operation when supported | scope lifetime plus existing 64 MiB per-blob cutoff | deterministic close; fallback remains executable | +| Mktree sessions | reopened after newly checkpointed packs | unchanged dependency-wave floor | one active session at a time | deterministic retire/reopen | +| Provisional target ledger | not applicable across public calls | linear in staged targets | existing 100,000 workspace-target cap | reject before ref movement | +| Compound sub-operations | not available | linear, serialized | conservative default; hard exported maximum | typed invalid-options failure | +| Per-call page/bundle residency | explicitly bounded | unchanged | existing page/bundle item and byte limits | existing typed batch-limit failures | + +No design claim treats wall time as deterministic. Object identity, generation +count, command/session topology, and closure state are the hard gates. + +## Determinism / Replay / Causality + +- Invocation order is recorded when a scope method is called, not when its + promise happens to settle. +- The same ordered inputs must yield the same page and bundle handles under + sequential retained calls and compound admission in SHA-1 and SHA-256 repos. +- The final target set uses the existing canonical handle ordering. +- The final generation is new causal retention evidence; it does not alter the + identity or history of any immutable payload object. +- Failed compound work emits no admitted result and does not advance the + workspace generation. + +## Git Substrate Impact + +| Substrate area | Impact | +| ----------------------- | ---------------------------------------------------------------------------------------------------------- | +| refs | One existing workspace ref moves once per successful compound operation instead of once per sub-operation. | +| commits | One existing parentless RootSet generation is authored; no commit format change. | +| trees/blobs | Existing page, bundle, lease, metadata, and RootSet encodings remain byte-identical. | +| object ids | Determined by existing Git and codec behavior; grouping must not change handles. | +| tag/release behavior | Publish as git-cas v6.5.9 before downstream adoption. | +| migration compatibility | No migration; existing repositories and workspaces remain readable. | + +## Compatibility / Migration Posture + +| Concern | Decision | +| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| Public API compatibility | Additive method; all existing methods retain their contracts. | +| Package export changes | Add declarations for compound scope/result and limits. | +| Storage/read compatibility | No serialized bytes, ref layout or namespaces, or readers change; successful compound admission reduces workspace-ref update frequency. | +| Legacy behavior retained | Singleton and independently retained batch calls remain available and tested. | +| Deprecation behavior | None. | +| Migration path | None required. | +| Release note impact | State explicitly that v6.5.9 is migration-free and changes physical admission cost only. | + +## Error Contract + +| Failure | Error/result | Caller recovery | Test | +| ------------------------------------------- | ----------------------------------------------------------------- | ----------------------------------------------- | ----------------------- | +| Missing callback or invalid operation bound | `INVALID_OPTIONS` | correct options | boundary table test | +| Empty compound work | `INVALID_OPTIONS` | stage at least one bounded batch | no-ref-movement test | +| Operation ceiling exceeded | `INVALID_OPTIONS` with observed and maximum counts | split into multiple compound operations | hostile call-count test | +| Escaped scope used after closure | `WORKSPACE_STATE_INVALID` | do not retain or reuse scope | lifecycle test | +| Page/bundle validation failure | existing typed service error | correct that sub-operation | earliest-failure test | +| Callback and close both fail | `AggregateError` preserving both failures | inspect both causes; retry with a new operation | injected close test | +| Final exact retention fails | `WORKSPACE_RETENTION_FAILED` with staged count and original error | inspect ref posture; retry safely | no-partial-result test | + +## Security / Trust / Redaction Posture + +- trust boundary: caller code may use only the bounded semantic scope; +- authority or capability checked: exact workspace expected-head mutation; +- secret-bearing values: unchanged; payload bytes are never added to evidence; +- redaction behavior: benchmark artifacts contain counts, versions, OIDs, and + digests but no payloads or machine-local paths; +- log/report behavior: failures report counts and operation names, not content; +- abuse or replay concern: scope calls are structurally bounded, and an escaped + scope is closed before caller-visible settlement; callback time, CPU, memory, + and side effects remain caller-controlled. + +## Lower Modes + +No visual mode exists. Public declarations, plain-text docs, structured errors, +and JSON witness data expose the same operation, generation, identity, bound, +failure, and lifecycle facts. + +## Accessibility Posture + +Docs and witnesses follow linear reading order, use descriptive table headers, +and repeat status in text instead of relying on color or layout. The API does +not require a visual interface. + +## User-Facing Text / Directionality + +English API docs, error messages, changelog entries, and witness labels change. +They use logical sequence words such as prior, next, and final; no directional +screen placement or visual-only distinction is introduced. + +## Agent Inspectability / Explainability Posture + +An agent can inspect configured and observed sub-operation counts, staged and +retained handle counts, exact generation count, command/session census, +semantic digests, object formats, prior and resulting ref OIDs, and active +session count after closure. It need not scrape `ps`, parse timing prose, or +infer whether intermediate results escaped. + +## Linked Invariants + +- every handle visible after a successful workspace method is already anchored; +- provisional handles exist only inside a bounded callback; +- one failed operation cannot produce a partial caller-visible success; +- input order equals handle order; +- persisted identity does not depend on sequential versus compound grouping; +- the prior workspace generation remains authoritative until one exact final + checked update succeeds; +- no-dereference and symbolic-ref containment remain unchanged; +- Git-authored object and commit semantics remain authoritative; +- all child processes and queued operations settle before the outer promise; +- release order remains Plumbing, git-cas, git-warp, then Think. + +## Design Alternatives Considered + +### Continue publishing every dependency wave + +Pros: + +- already shipped and safe; +- every intermediate handle can escape immediately. + +Cons: + +- repeated lease, metadata, tree, commit, and ref work dominates the remaining + downstream process census; +- physical publication cost scales with dependency depth even when nothing + escapes. + +### Let git-warp use unretained global page and bundle APIs + +Pros: + +- removes intermediate workspace generations with no git-cas API change. + +Cons: + +- violates the materialization workspace's retention contract; +- concurrent pruning can delete objects between dependency waves; +- moves a substrate safety decision into one consumer adapter. + +### Accept a static declarative dependency DAG + +Pros: + +- one bounded input and no callback lifecycle. + +Cons: + +- parent members require child OIDs not known before Git writes them; +- token references would create a second graph language and leak git-warp-like + topology into git-cas. + +### Open a public Git/Plumbing session + +Pros: + +- maximum caller control. + +Cons: + +- breaks the semantic storage boundary, leaks process lifecycle, and makes + correctness depend on caller-specific session choreography. + +### Add a bounded compound callback + +Pros: + +- models the actual point at which provisional handles may safely exist; +- keeps Git sessions private; +- permits dependent writes and one final authority transition; +- is generic across content-addressed applications. + +Cons: + +- callback lifecycle and queued failure behavior require explicit tests; +- the operation must be structurally bounded to avoid turning a session into an + arbitrary long-lived resource. + +## Decision + +Add the bounded compound callback. It is the smallest generic semantic boundary +that preserves the existing external retention law while removing redundant +physical publication. Begin with page and ordered-bundle batches because they +cover the proven downstream dependency graph and already have explicit item and +byte limits. Do not generalize further until another measured workload requires +it. + +## Proof Surface + +The implementation must be proven through: + +- actual surface under test: public `StagingWorkspace.batch()` against memory + adapters and real SHA-1/SHA-256 Git repositories; +- first RED test: two dependent page/bundle waves yield one checked workspace + update rather than one update per wave; +- required witness command: a counterbalanced sequential-versus-compound + diagnostic that records semantic digests, Git child/interactions, generation + count, operation limits, memory high water, and closure state; +- non-acceptable proof: documentation-only tests, elapsed time without object + identity, mocks without real-Git pruning, or lower process counts obtained by + bypassing retention. + +Named mutation calibration: + +1. Install after every sub-operation: generation-count test fails. +2. Expose the callback value before installation: injected retention failure + observes an illegal success. +3. Execute concurrent calls by settlement order: deterministic order test + fails. +4. Permit one operation beyond the configured ceiling: hostile bound test + fails. +5. Reuse an escaped scope: lifecycle test fails. +6. Drop one staged target from final installation: prune/readback test fails. +7. Open a new fast-import session per sub-operation: process-topology witness + fails. + +## Implementation Slices + +1. Add RED domain and declaration tests for the bounded callback, handle-only + scope, one generation, lifecycle, and failure laws. +2. Factor a named compound coordinator and scoped page/bundle service methods; + implement one exact final installation. +3. Add real-Git SHA-1/SHA-256 immediate-prune and no-generation-on-failure tests. +4. Add the counterbalanced process/identity witness and calibrate its mutation + checks. +5. Update README, API docs, architecture, changelog, design witness, and release + evidence; run all runtime and package gates. +6. Publish v6.5.9, consume the registry artifact in git-warp, and rerun its + exact reference plus migration-compatibility proof. + +## Tests To Write First + +- [x] Two dependent page/bundle calls share one resulting generation and one + checked update. +- [x] The callback receives frozen handle arrays, while the outer result pairs + its value with exact retention evidence. +- [x] Concurrent scope invocations execute in invocation order. +- [x] Invalid, empty, excessive, failed, and escaped operations never move the + prior workspace generation or expose a partial result. +- [x] A callback failure plus a session-close failure preserves both causes. +- [x] Existing independently retained APIs remain unchanged. +- [x] SHA-1 and SHA-256 compound handles equal sequential handles byte for byte. +- [x] Immediate prune after compound success preserves every retained support + object; release followed by prune reclaims them. +- [x] The process witness detects per-wave retention publication and per-wave + fast-import reopening mutations. + +## Acceptance Criteria + +The work is done when: + +- [x] Public behavior tests prove provisional scope, exact final retention, and + one-generation semantics. +- [x] Operation count, per-call bytes/items, targets, queues, and session + lifetime are explicitly bounded. +- [x] Existing page/bundle bytes and handles are identical under sequential and + compound modes in SHA-1 and SHA-256 repositories. +- [x] Failure tests prove no ref movement and no caller-visible partial result. +- [x] Real-Git pruning proves retained success and releasable cleanup. +- [x] Machine evidence reports a material process reduction and zero active + sessions after close. +- [x] Existing public APIs, storage readers, v6 workspaces, and release surfaces + remain compatible with no migration. +- [x] Public docs, architecture, changelog, and release notes are accurate. +- [ ] Issue and PR are linked; CI and complete local validation are green. +- [ ] Released v6.5.9 is consumed from the registry by git-warp before any + downstream performance claim. + +## Validation Plan + +```sh +npx vitest run test/unit/domain/services/StagingWorkspace.compound.test.js +npm test +npx eslint . +npm run test:integration:node +npm run test:integration:bun +npm run test:integration:deno +npm run release:verify +``` + +The witness will run isolated workers against temporary SHA-1 and SHA-256 bare +repositories. It will compare equivalent sequential-retained and compound +operations over repeated samples and reject semantic-digest, cardinality, +generation, process-topology, bound, or closure disagreement. + +## Playback / Witness + +Human playback questions: + +1. Did grouping change any handle, byte, or retained support graph? +2. Did all dependency waves become reachable through one exact generation? +3. How many Git children and ref publications disappeared? +4. What commands remain, and why are they required? + +Agent playback questions: + +1. Were configured operation and batch bounds observed? +2. Did invocation order equal execution and result order? +3. Did failure preserve the prior ref and withhold the callback value? +4. Were all sessions, queued calls, and workspace resources closed? + +Required artifacts: + +- machine-readable sequential/compound SHA-1/SHA-256 witness; +- readable verification summary with residual process floor; +- real-Git prune test output; +- exact-head hosted CI URLs; +- v6.5.9 candidate and publication identity evidence; +- downstream git-warp exact-head benchmark and migrated-v18 read gate. + +## Risks + +Known risks: + +- caller code may stall while owning the callback; +- an escaped scope may be invoked after close; +- concurrent callback calls may reorder writes or failures; +- target accumulation may approach the existing workspace cap; +- a pack checkpoint invalidates an already-open mktree object snapshot; +- reducing generations widens the private unanchored interval. +- trusted callback code can leak provisional handles through side effects. + +Mitigations: + +- bound method invocations and every individual page/bundle input; +- close and poison the scope before outer settlement; +- serialize by invocation order and stop queued work after first failure; +- enforce the existing workspace target ceiling before ref movement; +- keep the existing mktree retire/reopen rule after new packed objects; +- document callback-side-effect leakage as outside the contract and perform one + exact final retention before the outer operation returns any value. + +## Follow-On Debt + +The clean witness leaves 18 `mktree` children in the 33-operation compound +profile because descriptor packs must become visible before dependent tree +waves. Measure the released API in git-warp before deciding whether a typed +tree-writing protocol is justified. If it is, open a separate Plumbing/git-cas +issue with SHA-1/SHA-256 identity and validation evidence rather than widening +this compound API. Singleton assets or a wider operation profile require the +same evidence and separate scope. + +## Tracker Disposition + +| Issue | Role | Expected disposition | +| ----------------------------------------------------------------- | ------------------- | -------------------------------- | +| [git-cas#123](https://github.com/git-stunts/git-cas/issues/123) | primary goalpost | close after publication evidence | +| [git-warp#851](https://github.com/git-stunts/git-warp/issues/851) | downstream consumer | update after released adoption | + +## Done Does Not Mean + +When this lands, it does not prove: + +- one Git child for an arbitrary dependency graph; +- a cross-ref, cross-workspace, or application transaction; +- that wall time is identical on every host; +- that mktree can safely observe packs created after its ODB snapshot; +- that manual object encoding or a native Git implementation is justified; +- any storage migration or change to domain atomicity. +- a capability sandbox that can prevent trusted callback side effects. + +## Retrospective + +The implementation and clean witness are complete. A 33-operation, +81-handle graph fell from 200 to 23 Git children and from 33 retained +generations to one in both SHA-1 and SHA-256 repositories. Median wall time +fell by 80.5% with identical handle digests. The remaining work is hosted +multi-runtime review, v6.5.9 publication, and released downstream adoption. + +PR: + +- [#124](https://github.com/git-stunts/git-cas/pull/124) diff --git a/docs/design/0060-compound-workspace-admission/witness/compound-workspace-admission.json b/docs/design/0060-compound-workspace-admission/witness/compound-workspace-admission.json new file mode 100644 index 00000000..259d954a --- /dev/null +++ b/docs/design/0060-compound-workspace-admission/witness/compound-workspace-admission.json @@ -0,0 +1,406 @@ +{ + "schema": "git-cas.bounded-write-waves/v2", + "generatedAt": "2026-08-24T17:45:37.106Z", + "environment": { + "node": "v26.0.0", + "git": "git version 2.50.1 (Apple Git-155)", + "platform": "darwin", + "architecture": "arm64", + "gitCasCommit": "2a1b1eb177ba5dddd6c7c9e29044317a7384cd75", + "gitCasDirty": false, + "plumbingSource": "installed:@git-stunts/plumbing", + "plumbingCommit": null, + "plumbingDirty": null + }, + "metricScope": { + "processCount": "Git child processes opened by the instrumented git-cas operation", + "protocolOperationCount": "typed operations invoked on persistent Git sessions", + "gitInteractionCount": "one-shot Git commands plus typed persistent-session calls", + "wallMs": "isolated worker elapsed time including Git children and session close", + "workerCpuMs": "Node worker CPU only; excludes Git subprocess CPU", + "workerPeakRssBytes": "Node worker peak RSS only; excludes Git subprocess RSS" + }, + "parameters": { + "items": 16, + "samples": 5, + "assetConcurrency": 4, + "assetBytes": 2048, + "assetChunkBytes": 1024, + "bundleMembers": 3, + "compoundPagesPerGroup": 4 + }, + "objectFormats": { + "sha1": { + "assetWrite": { + "individual": { + "sampleCount": 5, + "semanticDigest": "c6bd1291520b73bfc1fc7729a6023d7a8f0ec4721d3bcd5d05ff768e85af358c", + "resultCount": 16, + "processCount": 64, + "protocolOperationCount": 48, + "gitInteractionCount": 80, + "counts": { + "hash-object": 32, + "session:fast-import": 16, + "session:mktree": 16 + }, + "protocolOperations": { + "fast-import:writeBlobs": 16, + "fast-import:checkpoint": 16, + "mktree:writeMany": 16 + }, + "wallMs": 1157.834, + "workerCpuMs": 125.949, + "workerUserCpuMs": 88.288, + "workerSystemCpuMs": 37.661, + "workerPeakRssBytes": 79052800 + }, + "batch": { + "sampleCount": 5, + "semanticDigest": "c6bd1291520b73bfc1fc7729a6023d7a8f0ec4721d3bcd5d05ff768e85af358c", + "resultCount": 16, + "processCount": 2, + "protocolOperationCount": 19, + "gitInteractionCount": 19, + "counts": { + "session:fast-import": 1, + "session:mktree": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 9, + "fast-import:checkpoint": 9, + "mktree:writeMany": 1 + }, + "wallMs": 117.501, + "workerCpuMs": 39.845, + "workerUserCpuMs": 36.749, + "workerSystemCpuMs": 3.096, + "workerPeakRssBytes": 76169216 + }, + "semanticDigestEqual": true, + "processReductionPercent": 96.875, + "gitInteractionReductionPercent": 76.25, + "wallReductionPercent": 89.852, + "workerCpuReductionPercent": 68.364 + }, + "workspaceBundleWrite": { + "individual": { + "sampleCount": 5, + "semanticDigest": "dc75736f497fc932d846dc368a0ee59ab9439e258e79abc3c86dcbf84dfa266a", + "resultCount": 16, + "processCount": 147, + "protocolOperationCount": 80, + "gitInteractionCount": 224, + "counts": { + "hash-object": 112, + "session:mktree": 1, + "session:cat-file": 1, + "commit-tree": 16, + "symbolic-ref": 16, + "session:update-ref": 1 + }, + "protocolOperations": { + "mktree:write": 48, + "cat-file:infoMany": 16, + "update-ref:update": 16 + }, + "wallMs": 2583.31, + "workerCpuMs": 239.777, + "workerUserCpuMs": 154.75, + "workerSystemCpuMs": 85.027, + "workerPeakRssBytes": 83181568 + }, + "batch": { + "sampleCount": 5, + "semanticDigest": "dc75736f497fc932d846dc368a0ee59ab9439e258e79abc3c86dcbf84dfa266a", + "resultCount": 16, + "processCount": 8, + "protocolOperationCount": 9, + "gitInteractionCount": 13, + "counts": { + "session:fast-import": 1, + "session:mktree": 1, + "hash-object": 2, + "session:cat-file": 1, + "commit-tree": 1, + "symbolic-ref": 1, + "session:update-ref": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 2, + "fast-import:checkpoint": 2, + "mktree:writeMany": 2, + "cat-file:infoMany": 1, + "mktree:write": 1, + "update-ref:update": 1 + }, + "wallMs": 240.325, + "workerCpuMs": 52.95, + "workerUserCpuMs": 46.576, + "workerSystemCpuMs": 6.374, + "workerPeakRssBytes": 76283904 + }, + "semanticDigestEqual": true, + "processReductionPercent": 94.558, + "gitInteractionReductionPercent": 94.196, + "wallReductionPercent": 90.697, + "workerCpuReductionPercent": 77.917 + }, + "compoundWorkspaceAdmission": { + "perWave": { + "sampleCount": 5, + "semanticDigest": "191507709b1c0ec4027e14d87d7a6096c3dc900a4d1ab7c3b9a253729124c544", + "resultCount": 81, + "processCount": 200, + "protocolOperationCount": 248, + "gitInteractionCount": 380, + "counts": { + "session:fast-import": 33, + "hash-object": 66, + "session:cat-file": 1, + "session:mktree": 33, + "commit-tree": 33, + "symbolic-ref": 33, + "session:update-ref": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 33, + "fast-import:checkpoint": 33, + "cat-file:infoMany": 33, + "mktree:write": 33, + "update-ref:update": 33, + "mktree:writeMany": 34, + "cat-file:read": 49 + }, + "wallMs": 3763.82, + "workerCpuMs": 370.336, + "workerUserCpuMs": 253.614, + "workerSystemCpuMs": 116.722, + "workerPeakRssBytes": 86163456 + }, + "compound": { + "sampleCount": 5, + "semanticDigest": "191507709b1c0ec4027e14d87d7a6096c3dc900a4d1ab7c3b9a253729124c544", + "resultCount": 81, + "processCount": 23, + "protocolOperationCount": 236, + "gitInteractionCount": 238, + "counts": { + "session:fast-import": 1, + "session:cat-file": 1, + "session:mktree": 18, + "commit-tree": 1, + "symbolic-ref": 1, + "session:update-ref": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 35, + "fast-import:checkpoint": 35, + "cat-file:info": 80, + "mktree:writeMany": 34, + "cat-file:read": 49, + "cat-file:infoMany": 1, + "mktree:write": 1, + "update-ref:update": 1 + }, + "wallMs": 733.116, + "workerCpuMs": 113.672, + "workerUserCpuMs": 95.198, + "workerSystemCpuMs": 18.66, + "workerPeakRssBytes": 83509248 + }, + "semanticDigestEqual": true, + "processReductionPercent": 88.5, + "gitInteractionReductionPercent": 37.368, + "wallReductionPercent": 80.522, + "workerCpuReductionPercent": 69.306 + } + }, + "sha256": { + "assetWrite": { + "individual": { + "sampleCount": 5, + "semanticDigest": "394db33860f1a82a6490b404d868b369addeff9de7101c53924adb76b4719a60", + "resultCount": 16, + "processCount": 64, + "protocolOperationCount": 48, + "gitInteractionCount": 80, + "counts": { + "hash-object": 32, + "session:fast-import": 16, + "session:mktree": 16 + }, + "protocolOperations": { + "fast-import:writeBlobs": 16, + "fast-import:checkpoint": 16, + "mktree:writeMany": 16 + }, + "wallMs": 1202.542, + "workerCpuMs": 128.075, + "workerUserCpuMs": 89.573, + "workerSystemCpuMs": 38.502, + "workerPeakRssBytes": 79216640 + }, + "batch": { + "sampleCount": 5, + "semanticDigest": "394db33860f1a82a6490b404d868b369addeff9de7101c53924adb76b4719a60", + "resultCount": 16, + "processCount": 2, + "protocolOperationCount": 19, + "gitInteractionCount": 19, + "counts": { + "session:fast-import": 1, + "session:mktree": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 9, + "fast-import:checkpoint": 9, + "mktree:writeMany": 1 + }, + "wallMs": 116.591, + "workerCpuMs": 39.569, + "workerUserCpuMs": 36.604, + "workerSystemCpuMs": 3.043, + "workerPeakRssBytes": 76070912 + }, + "semanticDigestEqual": true, + "processReductionPercent": 96.875, + "gitInteractionReductionPercent": 76.25, + "wallReductionPercent": 90.305, + "workerCpuReductionPercent": 69.105 + }, + "workspaceBundleWrite": { + "individual": { + "sampleCount": 5, + "semanticDigest": "5261e344aeacebd73f2b6b51abb94e4400244e49098fc58da06b433d33909d65", + "resultCount": 16, + "processCount": 147, + "protocolOperationCount": 80, + "gitInteractionCount": 224, + "counts": { + "hash-object": 112, + "session:mktree": 1, + "session:cat-file": 1, + "commit-tree": 16, + "symbolic-ref": 16, + "session:update-ref": 1 + }, + "protocolOperations": { + "mktree:write": 48, + "cat-file:infoMany": 16, + "update-ref:update": 16 + }, + "wallMs": 2524.779, + "workerCpuMs": 239.257, + "workerUserCpuMs": 154.909, + "workerSystemCpuMs": 84.348, + "workerPeakRssBytes": 83279872 + }, + "batch": { + "sampleCount": 5, + "semanticDigest": "5261e344aeacebd73f2b6b51abb94e4400244e49098fc58da06b433d33909d65", + "resultCount": 16, + "processCount": 8, + "protocolOperationCount": 9, + "gitInteractionCount": 13, + "counts": { + "session:fast-import": 1, + "session:mktree": 1, + "hash-object": 2, + "session:cat-file": 1, + "commit-tree": 1, + "symbolic-ref": 1, + "session:update-ref": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 2, + "fast-import:checkpoint": 2, + "mktree:writeMany": 2, + "cat-file:infoMany": 1, + "mktree:write": 1, + "update-ref:update": 1 + }, + "wallMs": 235.862, + "workerCpuMs": 53.075, + "workerUserCpuMs": 46.32, + "workerSystemCpuMs": 6.738, + "workerPeakRssBytes": 76349440 + }, + "semanticDigestEqual": true, + "processReductionPercent": 94.558, + "gitInteractionReductionPercent": 94.196, + "wallReductionPercent": 90.658, + "workerCpuReductionPercent": 77.817 + }, + "compoundWorkspaceAdmission": { + "perWave": { + "sampleCount": 5, + "semanticDigest": "f544138550771bb30f3340098d4913e1731807aa6a988e7906e1a1df56f5b785", + "resultCount": 81, + "processCount": 200, + "protocolOperationCount": 248, + "gitInteractionCount": 380, + "counts": { + "session:fast-import": 33, + "hash-object": 66, + "session:cat-file": 1, + "session:mktree": 33, + "commit-tree": 33, + "symbolic-ref": 33, + "session:update-ref": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 33, + "fast-import:checkpoint": 33, + "cat-file:infoMany": 33, + "mktree:write": 33, + "update-ref:update": 33, + "mktree:writeMany": 34, + "cat-file:read": 49 + }, + "wallMs": 3709.93, + "workerCpuMs": 373.898, + "workerUserCpuMs": 255.144, + "workerSystemCpuMs": 118.42, + "workerPeakRssBytes": 86458368 + }, + "compound": { + "sampleCount": 5, + "semanticDigest": "f544138550771bb30f3340098d4913e1731807aa6a988e7906e1a1df56f5b785", + "resultCount": 81, + "processCount": 23, + "protocolOperationCount": 236, + "gitInteractionCount": 238, + "counts": { + "session:fast-import": 1, + "session:cat-file": 1, + "session:mktree": 18, + "commit-tree": 1, + "symbolic-ref": 1, + "session:update-ref": 1 + }, + "protocolOperations": { + "fast-import:writeBlobs": 35, + "fast-import:checkpoint": 35, + "cat-file:info": 80, + "mktree:writeMany": 34, + "cat-file:read": 49, + "cat-file:infoMany": 1, + "mktree:write": 1, + "update-ref:update": 1 + }, + "wallMs": 726.976, + "workerCpuMs": 114.407, + "workerUserCpuMs": 95.219, + "workerSystemCpuMs": 18.082, + "workerPeakRssBytes": 83738624 + }, + "semanticDigestEqual": true, + "processReductionPercent": 88.5, + "gitInteractionReductionPercent": 37.368, + "wallReductionPercent": 80.405, + "workerCpuReductionPercent": 69.402 + } + } + } +} diff --git a/docs/design/0060-compound-workspace-admission/witness/verification.md b/docs/design/0060-compound-workspace-admission/witness/verification.md new file mode 100644 index 00000000..2b49802e --- /dev/null +++ b/docs/design/0060-compound-workspace-admission/witness/verification.md @@ -0,0 +1,118 @@ +# Compound Workspace Admission Verification + +## Exact Source and Environment + +- git-cas commit: `2a1b1eb177ba5dddd6c7c9e29044317a7384cd75` +- git-cas worktree: clean (`gitCasDirty: false`) +- installed Plumbing: `@git-stunts/plumbing@3.3.0` +- Node.js: `v26.0.0` +- Git: `2.50.1 (Apple Git-155)` +- host: macOS arm64 +- witness: [`compound-workspace-admission.json`](./compound-workspace-admission.json) + +The benchmark was run from the exact implementation and benchmark-harness +commit. The JSON witness was written outside the repository, so its clean-tree +claim does not exclude an in-progress output file. + +## Command + +```bash +GIT_CAS_BENCHMARK_OUTPUT=/tmp/git-cas-compound-workspace-admission-clean.json \ + node scripts/diagnostics/measure-bounded-write-waves.js 16 5 4 +``` + +Each mode ran five isolated worker samples. Mode order alternated by sample. +Every worker created a fresh bare repository, performed only the measured +operation, closed git-cas and its Git sessions, and was then discarded. The +reported wall and Node-worker CPU values are medians. Worker CPU excludes Git +subprocess CPU. + +## Workload + +The compound scenario constructs the same deterministic graph in both modes: + +- 16 page groups; +- four 64-byte pages per group, for 64 pages total; +- one dependent bundle per page group, for 16 leaf bundles; +- one root bundle over the 16 leaves; +- 81 returned application handles; +- 33 ordered write operations: 16 page waves, 16 leaf waves, and one root wave. + +The `perWave` mode invokes the existing independently retained workspace batch +methods 33 times. The `compound` mode invokes the same bounded page and bundle +services inside one `workspace.batch()` admission and installs one exact final +workspace generation. + +## Results + +| Object format | Mode | Git children | Git interactions | Wall ms | Worker CPU ms | +| ------------- | -------- | -----------: | ---------------: | ------: | ------------: | +| SHA-1 | per-wave | 200 | 380 | 3763.82 | 370.336 | +| SHA-1 | compound | 23 | 238 | 733.116 | 113.672 | +| SHA-256 | per-wave | 200 | 380 | 3709.93 | 373.898 | +| SHA-256 | compound | 23 | 238 | 726.976 | 114.407 | + +| Object format | Process reduction | Interaction reduction | Wall reduction | Worker CPU reduction | +| ------------- | ----------------: | --------------------: | -------------: | -------------------: | +| SHA-1 | 88.5% | 37.368% | 80.522% | 69.306% | +| SHA-256 | 88.5% | 37.368% | 80.405% | 69.402% | + +The semantic digests were equal within each object format: + +- SHA-1: `191507709b1c0ec4027e14d87d7a6096c3dc900a4d1ab7c3b9a253729124c544` +- SHA-256: `f544138550771bb30f3340098d4913e1731807aa6a988e7906e1a1df56f5b785` + +The digest covers every returned handle in construction order. Equality proves +that the batching change did not change page or bundle identity. + +## Child-Process Census + +Both object formats produced the same process topology: + +| Process/session | Per-wave | Compound | +| --------------------- | -------: | -------: | +| `fast-import` | 33 | 1 | +| `hash-object` | 66 | 0 | +| `cat-file` | 1 | 1 | +| `mktree` | 33 | 18 | +| `commit-tree` | 33 | 1 | +| `symbolic-ref` | 33 | 1 | +| `update-ref --stdin` | 1 | 1 | + +The single update-ref session performs 33 checked updates in per-wave mode and +one checked update in compound mode. Compound admission also keeps one scoped +fast-import process for every blob phase and writes the final workspace lease +blob through that same scope. + +The remaining 18 `mktree` processes are not repeated workspace publications. +They arise where interdependent descriptor packs must become visible before +the next Git tree wave. Removing them would require a safe typed tree-writing +protocol or equivalent deterministic object construction; it is a separate +optimization target and must preserve SHA-1/SHA-256 identity and Git's object +validation behavior. + +## Safety and Compatibility Gates + +```bash +npm test +npx eslint . +docker compose run --build --rm test-node \ + npx vitest run test/integration/compound-workspace-admission.test.js \ + --no-file-parallelism +``` + +- 2,172 unit tests passed and two were skipped at the exact implementation + checkpoint. +- The focused Docker integration passed for SHA-1 and SHA-256. +- Each integration case observed one checked ref publication, closed the + scoped fast-import session, ran `git prune --expire=now`, and read the + retained dependent graph successfully. +- Invalid bounds, empty operations, operation overflow, callback failure, + staged failure, distinct concurrent failures, checked-ref failure, escaped + scope use, prior-generation preservation, and queued-work poisoning have + deterministic regression coverage. + +This change is additive and migration-free. It changes neither application +handles nor stored object bytes, descriptor schemas, ref namespaces, existing +workspace methods, or read paths. Existing repositories and active v6.5.8 +workspace refs remain readable without rewriting or cutover. diff --git a/docs/design/README.md b/docs/design/README.md index 110fb8e9..2f0ef992 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -11,6 +11,7 @@ process in [docs/method/process.md](../method/process.md). ## Active METHOD Cycles +- [0060-compound-workspace-admission - compound-workspace-admission](./0060-compound-workspace-admission/compound-workspace-admission.md) - [0054-batched-page-retention - batched-page-retention](./0054-batched-page-retention/batched-page-retention.md) - [0050-lazy-bundle-reference-reads - lazy-bundle-reference-reads](./0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md) - [0049-scoped-staging-workspaces — scoped-staging-workspaces](./0049-scoped-staging-workspaces/scoped-staging-workspaces.md) diff --git a/docs/releases/v6.5.9.md b/docs/releases/v6.5.9.md new file mode 100644 index 00000000..93db9687 --- /dev/null +++ b/docs/releases/v6.5.9.md @@ -0,0 +1,106 @@ +# git-cas v6.5.9 Release Notes + +v6.5.9 adds bounded compound staging-workspace admission for applications that +construct dependency-ordered page and bundle graphs. It replaces one temporary +retention publication per construction wave with one exact final workspace +generation, without changing application handles, stored object bytes, +workspace descriptors, ref layout or namespaces, readers, or existing +workspace methods. Successful compound admission deliberately reduces +workspace-ref update frequency. + +## Compound Workspace Admission + +`workspace.batch()` owns one bounded callback and one operation-scoped Git +persistence view: + +```js +const admitted = await workspace.batch({ + maxOperations: 3, + operation: async (scope) => { + const pages = await scope.pages.putBatch({ pages: pageRequests }); + const leaves = await scope.bundles.putOrderedBatch({ + bundles: buildLeafRequests(pages), + }); + return ( + await scope.bundles.putOrderedBatch({ + bundles: [buildRootRequest(leaves)], + }) + )[0]; + }, +}); + +admitted.value; +admitted.retention; +``` + +The scope exposes only the existing bounded page-batch and ordered-bundle-batch +operations. Calls serialize by invocation order and return frozen handle +arrays for later dependency waves. The default scope limit is 64 operations; +the exported hard limit is 1,024. Every call also retains its existing page, +bundle, member, object, and byte ceilings. + +Success installs the union of previously retained workspace targets and every +new compound target exactly once. The callback value becomes caller-visible +only after scoped Git resources close and the paired retention result names the +exact final generation. Existing staging methods continue to retain their +result independently and remain appropriate when an intermediate handle must +leave private construction code. + +## Failure and Trust Boundary + +Invalid bounds, empty operations, callback failure, staged-write failure, +session-close failure, and checked final-retention failure return no admitted +value and do not move the prior workspace generation. The first failure stops +later queued work. Distinct callback/write and operation/close failures remain +available through `AggregateError`. Immutable objects written before refusal +may remain unreachable for Git's ordinary reclamation. + +The callback is trusted JavaScript. git-cas closes an escaped scope and never +returns the callback value before retention, but it cannot prevent callback +code from assigning a provisional handle into external state as a side effect. +Such a leaked handle has no compound retention witness and is outside the +contract. + +## Process-Topology Witness + +The committed five-sample witness compares the same 33-operation graph under +independent per-wave retention and compound admission. Each mode produces 64 +pages, 16 leaf bundles, one root bundle, and the same ordered sequence of 81 +application handles in fresh SHA-1 and SHA-256 bare repositories. + +| Format | Mode | Git children | Git interactions | Median wall ms | Worker CPU ms | +| ------- | -------- | -----------: | ---------------: | -------------: | ------------: | +| SHA-1 | per-wave | 200 | 380 | 3763.820 | 370.336 | +| SHA-1 | compound | 23 | 238 | 733.116 | 113.672 | +| SHA-256 | per-wave | 200 | 380 | 3709.930 | 373.898 | +| SHA-256 | compound | 23 | 238 | 726.976 | 114.407 | + +Both formats reduced Git child creation by 88.5% and median wall time by about +80.5% on the measured host. `fast-import`, `commit-tree`, symbolic-ref +containment checks, and checked ref updates fell from one per wave to one per +compound admission. The semantic handle digests matched exactly. + +The remaining 18 `mktree` children preserve validation across interdependent +descriptor-pack visibility boundaries. Further reduction requires a separately +proved typed tree-writing protocol or equivalent deterministic construction; +this release does not bypass Git's object validation. + +See the [machine witness](../design/0060-compound-workspace-admission/witness/compound-workspace-admission.json) +and [verification narrative](../design/0060-compound-workspace-admission/witness/verification.md) +for the exact source, environment, counts, digests, and method. + +## Reachability Proof + +Real-Git integration tests cover SHA-1 and SHA-256 success and refusal. They +verify one checked publication, zero active scoped fast-import sessions after +settlement, immediate-prune readability while retained, reclamation after +checked release, and no workspace generation after a dependent wave fails. + +## Compatibility + +This release is additive and requires no application or stored-data migration. +Existing repositories open in place. It introduces no new object format, +descriptor version, ref namespace, reader, transaction log, authority cutover, +or mixed-version rewrite. Applications may leave every existing workspace call +unchanged and adopt `workspace.batch()` only where intermediate handles remain +private to one bounded construction. diff --git a/index.d.ts b/index.d.ts index 986890ad..18acedf0 100644 --- a/index.d.ts +++ b/index.d.ts @@ -1614,6 +1614,27 @@ export interface WorkspaceCheckpointResult { readonly witnesses: ReadonlyArray; } +export const DEFAULT_WORKSPACE_COMPOUND_OPERATIONS: 64; +export const MAX_WORKSPACE_COMPOUND_OPERATIONS: 1024; + +export interface WorkspaceCompoundScope { + readonly pages: { + putBatch( + options: Parameters[0], + ): Promise>; + }; + readonly bundles: { + putOrderedBatch( + options: Parameters[0], + ): Promise>; + }; +} + +export interface WorkspaceCompoundResult { + readonly value: T; + readonly retention: WorkspaceCheckpointResult; +} + export interface WorkspaceReleaseResult { readonly changed: boolean; readonly ref: string; @@ -1691,6 +1712,10 @@ export declare class StagingWorkspace { options: Parameters[0], ): Promise>; }; + batch(options: { + operation(scope: WorkspaceCompoundScope): T | Promise; + maxOperations?: number; + }): Promise>>; checkpoint(options: { handles: Iterable }): Promise; renew(): Promise; promoteToCache(options: { diff --git a/index.js b/index.js index 3da202e6..8641a7db 100644 --- a/index.js +++ b/index.js @@ -110,6 +110,10 @@ export { default as RepositoryInspectionPort } from './src/ports/RepositoryInspe export { default as NodeCompressionAdapter } from './src/infrastructure/adapters/NodeCompressionAdapter.js'; export { default as diffManifests } from './src/domain/services/ManifestDiff.js'; export { SCHEME_WHOLE, SCHEME_FRAMED, SCHEME_CONVERGENT } from './src/domain/encryption/schemes.js'; +export { + DEFAULT_WORKSPACE_COMPOUND_OPERATIONS, + MAX_WORKSPACE_COMPOUND_OPERATIONS, +} from './src/domain/services/WorkspaceCompoundScope.js'; /** * High-level facade for the Content Addressable Store library. diff --git a/scripts/diagnostics/measure-bounded-write-waves.js b/scripts/diagnostics/measure-bounded-write-waves.js index 60a3da3f..b7a7c793 100644 --- a/scripts/diagnostics/measure-bounded-write-waves.js +++ b/scripts/diagnostics/measure-bounded-write-waves.js @@ -6,7 +6,7 @@ import os from 'node:os'; import path from 'node:path'; import { performance } from 'node:perf_hooks'; import { fileURLToPath, pathToFileURL } from 'node:url'; -import ContentAddressableStore from '../../index.js'; +import ContentAddressableStore, { MAX_WORKSPACE_COMPOUND_OPERATIONS } from '../../index.js'; import { instrumentGitPlumbing } from './createCountingGitPlumbing.js'; const WORKER = '--worker'; @@ -16,6 +16,7 @@ const DEFAULT_ASSET_CONCURRENCY = 4; const ASSET_BYTES = 2 * 1024; const ASSET_CHUNK_BYTES = 1024; const BUNDLE_MEMBERS = 3; +const COMPOUND_PAGES_PER_GROUP = 4; const CLOCK = Object.freeze({ now: () => new Date('2026-08-23T12:00:00.000Z') }); const scriptPath = fileURLToPath(import.meta.url); const invokedPath = process.argv[1] === undefined ? null : path.resolve(process.argv[1]); @@ -66,9 +67,15 @@ async function measureObjectFormat(options) { right: { ...common, kind: 'workspace-bundles', mode: 'batch' }, samples: options.samples, }); + const compoundWorkspaceAdmission = await compareModes({ + left: { ...common, kind: 'workspace-compound', mode: 'per-wave' }, + right: { ...common, kind: 'workspace-compound', mode: 'compound' }, + samples: options.samples, + }); return { assetWrite: comparison(assetWrite), workspaceBundleWrite: comparison(workspaceBundleWrite), + compoundWorkspaceAdmission: compoundComparison(compoundWorkspaceAdmission), }; } @@ -86,14 +93,36 @@ async function compareModes({ left, right, samples }) { } export function comparison({ left, right }) { + return { + individual: left, + batch: right, + ...comparisonMetrics(left, right), + }; +} + +export function compoundComparison({ left, right }) { + return { + perWave: left, + compound: right, + ...comparisonMetrics(left, right), + }; +} + +export function compoundOperationCount(items) { + const operationCount = items * 2 + 1; + if (operationCount > MAX_WORKSPACE_COMPOUND_OPERATIONS) { + throw new Error('compound benchmark groups exceed the workspace operation ceiling'); + } + return operationCount; +} + +function comparisonMetrics(left, right) { assert.equal( left.semanticDigest, right.semanticDigest, - 'Benchmark semantic digests differ between individual and batch modes', + 'Benchmark semantic digests differ between compared modes', ); return { - individual: left, - batch: right, semanticDigestEqual: true, processReductionPercent: reduction(left.processCount, right.processCount), gitInteractionReductionPercent: reduction( @@ -131,9 +160,7 @@ async function measureWorker(options) { let values; const metrics = await timed(async () => { try { - values = options.kind === 'assets' - ? await writeAssets(cas, options) - : await writeWorkspaceBundles(cas, options); + values = await writeValues(cas, options); } finally { await cas.close(); } @@ -145,6 +172,16 @@ async function measureWorker(options) { } } +async function writeValues(cas, options) { + if (options.kind === 'assets') { + return await writeAssets(cas, options); + } + if (options.kind === 'workspace-bundles') { + return await writeWorkspaceBundles(cas, options); + } + return await writeCompoundWorkspaceGraph(cas, options); +} + async function writeAssets(cas, { items, mode, assetConcurrency }) { const requests = Array.from({ length: items }, assetRequest); const staged = mode === 'batch' @@ -165,6 +202,56 @@ async function writeWorkspaceBundles(cas, { items, mode }) { return staged.map((bundle) => bundle.handle.toString()); } +async function writeCompoundWorkspaceGraph(cas, { items, mode }) { + const operationCount = compoundOperationCount(items); + const workspace = await cas.workspaces.open({ + namespace: 'git-warp/compound-materializations', + ttlMs: 60_000, + }); + if (mode === 'compound') { + const admitted = await workspace.batch({ + maxOperations: operationCount, + operation: async (scope) => await stageCompoundGraph(scope, items), + }); + return admitted.value.map(handleToken); + } + return (await stageCompoundGraph(workspace, items)).map(handleToken); +} + +async function stageCompoundGraph(scope, groupCount) { + const retained = []; + const leaves = []; + for (let group = 0; group < groupCount; group += 1) { + const pages = handlesOf(await scope.pages.putBatch({ + pages: Array.from({ length: COMPOUND_PAGES_PER_GROUP }, (_, index) => ({ + source: payload(group * COMPOUND_PAGES_PER_GROUP + index, 64), + })), + })); + retained.push(...pages); + const [leaf] = handlesOf(await scope.bundles.putOrderedBatch({ + bundles: [{ members: pages.map((page, index) => [paddedPath('page', index), page]) }], + })); + leaves.push(leaf); + retained.push(leaf); + } + const [root] = handlesOf(await scope.bundles.putOrderedBatch({ + bundles: [{ members: leaves.map((leaf, index) => [paddedPath('leaf', index), leaf]) }], + })); + return [...retained, root]; +} + +function handlesOf(staged) { + return staged.map((value) => value.handle ?? value); +} + +function handleToken(handle) { + return handle.toString(); +} + +function paddedPath(prefix, index) { + return `${prefix}/${String(index).padStart(4, '0')}`; +} + async function writeIndividually(requests, write) { const staged = []; for (const request of requests) { @@ -263,7 +350,7 @@ function summarize(samples) { function report({ items, samples, assetConcurrency, plumbingRepo, objectFormats }) { return { - schema: 'git-cas.bounded-write-waves/v1', + schema: 'git-cas.bounded-write-waves/v2', generatedAt: new Date().toISOString(), environment: { node: process.version, @@ -293,6 +380,7 @@ function report({ items, samples, assetConcurrency, plumbingRepo, objectFormats assetBytes: ASSET_BYTES, assetChunkBytes: ASSET_CHUNK_BYTES, bundleMembers: BUNDLE_MEMBERS, + compoundPagesPerGroup: COMPOUND_PAGES_PER_GROUP, }, objectFormats, }; diff --git a/src/domain/services/BundleService.js b/src/domain/services/BundleService.js index 44fc1b0d..c1cb8699 100644 --- a/src/domain/services/BundleService.js +++ b/src/domain/services/BundleService.js @@ -103,6 +103,14 @@ export default class BundleService { * descriptor and tree dependency waves. */ async putOrderedBatch(options = {}) { + return await withWriteScope( + this.#persistence, + async (persistence) => await this.putOrderedBatchWithPersistence(options, persistence), + ); + } + + /** @internal Builds a bounded bundle group through an operation-owned persistence view. */ + async putOrderedBatchWithPersistence(options = {}, persistence) { const batch = BundleService.#batchOptions(options); const staging = new StagingEvidence(); try { @@ -110,15 +118,7 @@ export default class BundleService { if (admitted.length === 0) { return Object.freeze([]); } - return await withWriteScope( - this.#persistence, - async (persistence) => await this.#writeAdmittedBatch({ - admitted, - batch, - persistence, - staging, - }), - ); + return await this.#writeAdmittedBatch({ admitted, batch, persistence, staging }); } catch (error) { throw augmentError(error, { staging: staging.snapshot() }); } diff --git a/src/domain/services/PageService.js b/src/domain/services/PageService.js index eb1f0a6e..40a46bf5 100644 --- a/src/domain/services/PageService.js +++ b/src/domain/services/PageService.js @@ -56,10 +56,19 @@ export default class PageService { * @returns {Promise} */ async put({ source, maxBytes }) { + return await this.#put({ source, maxBytes }, this.#persistence); + } + + /** @internal Stores one page through an operation-owned persistence view. */ + async putWithPersistence({ source, maxBytes }, persistence) { + return await this.#put({ source, maxBytes }, persistence); + } + + async #put({ source, maxBytes }, persistence) { const limit = this.#effectiveLimit(maxBytes); const observedAt = this.#observedAt(); const bytes = await PageService.#collect(source, limit); - const oid = await this.#persistence.writeBlob(bytes); + const oid = await persistence.writeBlob(bytes); return recordStagedTarget(new StagedPage({ handle: new PageHandle({ oid }), size: bytes.length, diff --git a/src/domain/services/StagingWorkspace.js b/src/domain/services/StagingWorkspace.js index 0b17f502..ba54f062 100644 --- a/src/domain/services/StagingWorkspace.js +++ b/src/domain/services/StagingWorkspace.js @@ -18,6 +18,7 @@ export default class StagingWorkspace { #assets; #bundles; #clock; + #compound; #descriptorCodec; #expiresAt = null; #generation = null; @@ -27,26 +28,30 @@ export default class StagingWorkspace { #released = false; #resolveHandle; #rootSet; + #rootSetForPersistence; #tail = Promise.resolve(); #targets = new Map(); #ttlMs; #workspaceRef; - constructor({ workspaceRef, ttlMs, rootSet, refs, assets, pages, bundles, publications, - resolveHandle, descriptorCodec, clock = DEFAULT_CLOCK }) { + constructor({ workspaceRef, ttlMs, rootSet, rootSetForPersistence, refs, assets, pages, + bundles, publications, resolveHandle, descriptorCodec, compound, clock = DEFAULT_CLOCK }) { StagingWorkspace.#assertDependencies({ rootSet, + rootSetForPersistence, refs, assets, pages, bundles, resolveHandle, descriptorCodec, + compound, clock, }); this.#workspaceRef = WorkspaceRef.from(workspaceRef); this.#ttlMs = ttlMs; this.#rootSet = rootSet; + this.#rootSetForPersistence = rootSetForPersistence; this.#refs = refs; this.#assets = assets; this.#pages = pages; @@ -54,8 +59,13 @@ export default class StagingWorkspace { this.#publications = publications; this.#resolveHandle = resolveHandle; this.#descriptorCodec = descriptorCodec; + this.#compound = compound; this.#clock = clock; + this.#initializeCapabilities(); + Object.freeze(this); + } + #initializeCapabilities() { this.assets = Object.freeze({ put: (options) => this.#enqueue(() => this.#stage(this.#assets, 'put', options)), putBatch: (options) => ( @@ -76,7 +86,6 @@ export default class StagingWorkspace { this.#enqueue(() => this.#stageBatch(this.#bundles, 'putOrderedBatch', options)) ), }); - Object.freeze(this); } get id() { @@ -95,6 +104,18 @@ export default class StagingWorkspace { return this.#expiresAt; } + batch(options = {}) { + return this.#enqueue(async () => { + this.#assertActive(); + return await this.#compound.admit({ + ...options, + install: async (staged, persistence) => ( + await this.#retainCompound(staged, persistence) + ), + }); + }); + } + checkpoint({ handles }) { return this.#enqueue(async () => { this.#assertActive(); @@ -282,7 +303,31 @@ export default class StagingWorkspace { })); } - async #install(targets) { + async #retainCompound(staged, persistence) { + const targets = new Map(this.#targets); + for (const artifact of staged) { + const target = stagedTargetOf(artifact) ?? await this.#resolveTarget(artifact.handle); + targets.set(target.handle.toString(), target); + } + try { + return await this.#install([...targets.values()], persistence); + } catch (error) { + if (error?.code === ErrorCodes.WORKSPACE_TTL_INVALID) { + throw error; + } + throw createCasError( + 'Workspace compound admission could not establish retention', + ErrorCodes.WORKSPACE_RETENTION_FAILED, + { + workspaceId: this.id, + stagedCount: staged.length, + originalError: error, + }, + ); + } + } + + async #install(targets, persistence = null) { if (targets.length > MAX_WORKSPACE_TARGETS) { throw createCasError( 'Workspace target count exceeds the supported maximum', @@ -292,13 +337,10 @@ export default class StagingWorkspace { } const observedAt = this.#observedAt(); const expiresAt = this.#expiryFrom(observedAt); - const descriptor = await this.#pages.put({ - source: this.#descriptorCodec.encode({ - ref: this.#workspaceRef.toString(), - createdAt: this.#workspaceRef.createdAt, - expiresAt, - targetCount: targets.length, - }), + const descriptor = await this.#writeDescriptor({ + expiresAt, + targetCount: targets.length, + persistence, }); const entries = [ { @@ -309,7 +351,10 @@ export default class StagingWorkspace { }, ...targets.map(StagingWorkspace.#targetEntry), ]; - const mutation = await this.#rootSet.replaceExact({ + const rootSet = persistence === null + ? this.#rootSet + : this.#rootSetForPersistence(persistence); + const mutation = await rootSet.replaceExact({ entries, expectedHeadOid: this.#generation, }); @@ -331,6 +376,20 @@ export default class StagingWorkspace { }); } + async #writeDescriptor({ expiresAt, targetCount, persistence }) { + const options = { + source: this.#descriptorCodec.encode({ + ref: this.#workspaceRef.toString(), + createdAt: this.#workspaceRef.createdAt, + expiresAt, + targetCount, + }), + }; + return persistence === null + ? await this.#pages.put(options) + : await this.#pages.putWithPersistence(options, persistence); + } + #witness({ target, entries, observedAt }) { const name = StagingWorkspace.#targetName(target.handle); const index = entries.findIndex((entry) => entry.name === name); @@ -580,16 +639,18 @@ export default class StagingWorkspace { }); } - static #assertDependencies({ rootSet, refs, assets, pages, bundles, resolveHandle, - descriptorCodec, clock }) { + static #assertDependencies({ rootSet, rootSetForPersistence, refs, assets, pages, bundles, + resolveHandle, descriptorCodec, compound, clock }) { const missing = [ ['rootSet', StagingWorkspace.#hasMethods(rootSet, ['replaceExact'])], + ['rootSetForPersistence', typeof rootSetForPersistence === 'function'], ['refs', StagingWorkspace.#hasMethods(refs, ['deleteRef'])], ['assets', StagingWorkspace.#hasMethods(assets, ['put', 'adopt'])], ['pages', StagingWorkspace.#hasMethods(pages, ['put', 'putBatch'])], ['bundles', StagingWorkspace.#hasMethods(bundles, ['put', 'putOrdered'])], ['resolveHandle', typeof resolveHandle === 'function'], ['descriptorCodec', StagingWorkspace.#hasMethods(descriptorCodec, ['encode'])], + ['compound', StagingWorkspace.#hasMethods(compound, ['admit'])], ['clock', StagingWorkspace.#hasMethods(clock, ['now'])], ].filter(([, valid]) => !valid).map(([name]) => name); if (missing.length > 0) { diff --git a/src/domain/services/StagingWorkspaceRegistry.js b/src/domain/services/StagingWorkspaceRegistry.js index fe8ba0b5..63cc1a5f 100644 --- a/src/domain/services/StagingWorkspaceRegistry.js +++ b/src/domain/services/StagingWorkspaceRegistry.js @@ -8,6 +8,7 @@ import RootSet from './RootSet.js'; import RootSetMetadataCodec from './RootSetMetadataCodec.js'; import RootSetPersistence from './RootSetPersistence.js'; import StagingWorkspace from './StagingWorkspace.js'; +import WorkspaceCompoundAdmission from './WorkspaceCompoundAdmission.js'; import WorkspaceDescriptorCodec, { WORKSPACE_DESCRIPTOR_ENTRY, } from './WorkspaceDescriptorCodec.js'; @@ -37,6 +38,7 @@ export default class StagingWorkspaceRegistry { #assets; #bundles; #clock; + #compound; #crypto; #descriptorCodec; #pages; @@ -68,6 +70,7 @@ export default class StagingWorkspaceRegistry { this.#crypto = crypto; this.#clock = clock; this.#descriptorCodec = descriptorCodec; + this.#compound = new WorkspaceCompoundAdmission({ persistence, pages, bundles }); Object.freeze(this); } @@ -81,6 +84,7 @@ export default class StagingWorkspaceRegistry { workspaceRef, ttlMs, rootSet: this.#rootSet(workspaceRef), + rootSetForPersistence: (persistence) => this.#rootSet(workspaceRef, persistence), refs: this.#ref, assets: this.#assets, pages: this.#pages, @@ -88,6 +92,7 @@ export default class StagingWorkspaceRegistry { publications: this.#publications, resolveHandle: this.#resolveHandle, descriptorCodec: this.#descriptorCodec, + compound: this.#compound, clock: this.#clock, }); } @@ -271,19 +276,19 @@ export default class StagingWorkspaceRegistry { return logicalBytes; } - #rootSet(workspaceRef) { + #rootSet(workspaceRef, persistence = this.#persistence) { const ref = WorkspaceRef.from(workspaceRef).toString(); const metadataCodec = new RootSetMetadataCodec({ refType: WorkspaceRef }); - const persistence = new RootSetPersistence({ + const rootSetPersistence = new RootSetPersistence({ rootSetRef: ref, - persistence: this.#persistence, + persistence, ref: this.#ref, refType: WorkspaceRef, metadataCodec, }); return new RootSet({ ref, - persistence, + persistence: rootSetPersistence, refType: WorkspaceRef, metadataCodec, }); diff --git a/src/domain/services/WorkspaceCompoundAdmission.js b/src/domain/services/WorkspaceCompoundAdmission.js new file mode 100644 index 00000000..81d58d66 --- /dev/null +++ b/src/domain/services/WorkspaceCompoundAdmission.js @@ -0,0 +1,61 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import WorkspaceCompoundScope from './WorkspaceCompoundScope.js'; + +/** Executes one bounded provisional graph build before exact workspace retention. */ +export default class WorkspaceCompoundAdmission { + #bundles; + #pages; + #persistence; + + constructor({ persistence, pages, bundles }) { + WorkspaceCompoundAdmission.#assertDependencies({ persistence, pages, bundles }); + this.#persistence = persistence; + this.#pages = pages; + this.#bundles = bundles; + Object.freeze(this); + } + + async admit({ operation, maxOperations, install } = {}) { + if (typeof install !== 'function') { + throw createCasError( + 'Workspace compound admission requires an installation callback', + ErrorCodes.INVALID_OPTIONS + ); + } + return await withWriteScope(this.#persistence, async (persistence) => { + const scope = new WorkspaceCompoundScope({ + pages: this.#pages, + bundles: this.#bundles, + persistence, + maxOperations, + }); + const prepared = await scope.execute(operation); + const retention = await install(prepared.staged, persistence); + return Object.freeze({ value: prepared.value, retention }); + }); + } + + static #assertDependencies({ persistence, pages, bundles }) { + const missing = [ + ['persistence', persistence === null || typeof persistence !== 'object'], + ['pages.putBatch', typeof pages?.putBatch !== 'function'], + ['bundles', bundles === null || typeof bundles !== 'object'], + ] + .filter(([, absent]) => absent) + .map(([name]) => name); + if (missing.length > 0) { + throw createCasError( + 'Workspace compound admission requires complete dependencies', + ErrorCodes.INVALID_OPTIONS, + { missing } + ); + } + } +} + +async function withWriteScope(persistence, operation) { + return typeof persistence.withWriteScope === 'function' + ? await persistence.withWriteScope(operation) + : await operation(persistence); +} diff --git a/src/domain/services/WorkspaceCompoundScope.js b/src/domain/services/WorkspaceCompoundScope.js new file mode 100644 index 00000000..4f36b696 --- /dev/null +++ b/src/domain/services/WorkspaceCompoundScope.js @@ -0,0 +1,193 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; + +export const DEFAULT_WORKSPACE_COMPOUND_OPERATIONS = 64; +export const MAX_WORKSPACE_COMPOUND_OPERATIONS = 1_024; + +/** Bounded provisional page and bundle writes owned by one compound admission. */ +export default class WorkspaceCompoundScope { + #active = true; + #abortFailure; + #aborted = false; + #bundles; + #failed = false; + #failure; + #maxOperations; + #operationCount = 0; + #overflowFailure = null; + #pages; + #persistence; + #staged = []; + #tail = Promise.resolve(); + + constructor({ pages, bundles, persistence, maxOperations }) { + WorkspaceCompoundScope.#assertDependencies({ pages, bundles, persistence }); + this.#pages = pages; + this.#bundles = bundles; + this.#persistence = persistence; + this.#maxOperations = WorkspaceCompoundScope.#operationLimit(maxOperations); + this.api = Object.freeze({ + pages: Object.freeze({ + putBatch: (options) => + this.#enqueue('pages.putBatch', async () => await this.#putPages(options)), + }), + bundles: Object.freeze({ + putOrderedBatch: (options) => + this.#enqueue('bundles.putOrderedBatch', async () => await this.#putBundles(options)), + }), + }); + Object.freeze(this); + } + + async execute(operation) { + if (typeof operation !== 'function') { + throw createCasError( + 'Workspace compound admission requires an operation callback', + ErrorCodes.INVALID_OPTIONS + ); + } + let value; + let callbackError; + let callbackFailed = false; + try { + value = await operation(this.api); + } catch (error) { + callbackFailed = true; + callbackError = error; + this.#aborted = true; + this.#abortFailure = error; + } + this.#active = false; + await this.#tail; + if (callbackFailed && this.#failed && callbackError !== this.#failure) { + throw new AggregateError( + [callbackError, this.#failure], + 'Workspace compound callback and staged operation both failed' + ); + } + if (callbackFailed) { + throw callbackError; + } + if (this.#failed) { + throw this.#failure; + } + if (this.#staged.length === 0) { + throw createCasError( + 'Workspace compound admission staged no handles', + ErrorCodes.INVALID_OPTIONS + ); + } + return Object.freeze({ + value, + staged: Object.freeze([...this.#staged]), + operationCount: this.#operationCount, + }); + } + + #enqueue(method, operation) { + if (!this.#active) { + return Promise.reject( + createCasError('Workspace compound scope is closed', ErrorCodes.WORKSPACE_STATE_INVALID, { + method, + }) + ); + } + if (this.#overflowFailure !== null) { + return Promise.reject(this.#overflowFailure); + } + this.#operationCount += 1; + if (this.#operationCount > this.#maxOperations) { + this.#overflowFailure = createCasError( + 'Workspace compound operation count exceeds the configured maximum', + ErrorCodes.INVALID_OPTIONS, + { + operationCount: this.#operationCount, + maxOperations: this.#maxOperations, + method, + } + ); + this.#recordFailure(this.#overflowFailure); + return Promise.reject(this.#overflowFailure); + } + const result = this.#tail.then(async () => { + if (this.#failed) { + throw this.#failure; + } + if (this.#aborted) { + throw this.#abortFailure; + } + try { + return await operation(); + } catch (error) { + this.#recordFailure(error); + throw error; + } + }); + this.#tail = result.then( + () => undefined, + () => undefined, + ); + return result; + } + + #recordFailure(error) { + if (!this.#failed) { + this.#failed = true; + this.#failure = error; + } + } + + async #putPages(options) { + const staged = await this.#pages.putBatchWithPersistence(options, this.#persistence); + return this.#record(staged, 'page'); + } + + async #putBundles(options) { + const staged = await this.#bundles.putOrderedBatchWithPersistence(options, this.#persistence); + return this.#record(staged, 'bundle'); + } + + #record(staged, kind) { + if (!Array.isArray(staged) || staged.some((artifact) => !artifact?.handle)) { + throw createCasError( + `Workspace compound ${kind} batch did not return staged handles`, + ErrorCodes.WORKSPACE_STATE_INVALID, + { kind } + ); + } + this.#staged.push(...staged); + return Object.freeze(staged.map((artifact) => artifact.handle)); + } + + static #operationLimit(value) { + const limit = value ?? DEFAULT_WORKSPACE_COMPOUND_OPERATIONS; + if (!Number.isSafeInteger(limit) || limit <= 0 || limit > MAX_WORKSPACE_COMPOUND_OPERATIONS) { + throw createCasError( + 'Workspace compound operation limit is outside the supported range', + ErrorCodes.INVALID_OPTIONS, + { maxOperations: value, hardMaximum: MAX_WORKSPACE_COMPOUND_OPERATIONS } + ); + } + return limit; + } + + static #assertDependencies({ pages, bundles, persistence }) { + const missing = [ + ['pages.putBatchWithPersistence', typeof pages?.putBatchWithPersistence !== 'function'], + [ + 'bundles.putOrderedBatchWithPersistence', + typeof bundles?.putOrderedBatchWithPersistence !== 'function', + ], + ['persistence', persistence === null || typeof persistence !== 'object'], + ] + .filter(([, absent]) => absent) + .map(([name]) => name); + if (missing.length > 0) { + throw createCasError( + 'Workspace compound scope requires complete dependencies', + ErrorCodes.INVALID_OPTIONS, + { missing } + ); + } + } +} diff --git a/test/integration/compound-workspace-admission.test.js b/test/integration/compound-workspace-admission.test.js new file mode 100644 index 00000000..5a980f5d --- /dev/null +++ b/test/integration/compound-workspace-admission.test.js @@ -0,0 +1,171 @@ +/** + * Real-Git proof for compound workspace admission and immediate-prune safety. + * + * MUST run inside Docker (GIT_STUNTS_DOCKER=1). Refuses to run on the host. + */ + +import { spawnSync } from 'node:child_process'; +import { mkdtempSync, rmSync } from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { describe, expect, it, vi } from 'vitest'; +import ContentAddressableStore from '../../index.js'; +import { createCountingGitPlumbing } from '../../scripts/diagnostics/createCountingGitPlumbing.js'; +import { ErrorCodes } from '../../src/domain/errors/index.js'; + +if (process.env.GIT_STUNTS_DOCKER !== '1') { + throw new Error( + 'Integration tests MUST run inside Docker (GIT_STUNTS_DOCKER=1). ' + + 'Use: npm run test:integration:node' + ); +} + +vi.setConfig({ testTimeout: 30_000 }); + +describe.each(['sha1', 'sha256'])('real-Git %s compound workspace admission', (objectFormat) => { + it('retains dependent waves through one checked ref publication', async () => { + await proveCompoundAdmission(objectFormat); + }); + + it('publishes no generation when a dependent wave fails', async () => { + await proveFailureContainment(objectFormat); + }); +}); + +async function proveCompoundAdmission(objectFormat) { + const repo = mkdtempSync(path.join(os.tmpdir(), `cas-compound-${objectFormat}-`)); + initializeRepository(repo, objectFormat); + const counted = await createCountingGitPlumbing({ cwd: repo, sessions: true }); + const cas = new ContentAddressableStore({ plumbing: counted.plumbing }); + try { + const workspace = await cas.workspaces.open({ + namespace: `integration/compound-${objectFormat}`, + ttlMs: 60_000, + }); + const admitted = await admitGraph(workspace); + await assertAdmission({ admitted, cas, counted, repo, workspace }); + } finally { + await cas.close(); + rmSync(repo, { recursive: true, force: true }); + } +} + +async function proveFailureContainment(objectFormat) { + const repo = mkdtempSync(path.join(os.tmpdir(), `cas-compound-failure-${objectFormat}-`)); + initializeRepository(repo, objectFormat); + const cas = new ContentAddressableStore({ + plumbing: (await createCountingGitPlumbing({ cwd: repo })).plumbing, + }); + try { + const namespace = `integration/compound-failure-${objectFormat}`; + const workspace = await cas.workspaces.open({ namespace, ttlMs: 60_000 }); + let provisional; + await expect(workspace.batch({ + operation: async (scope) => { + [provisional] = await scope.pages.putBatch({ + pages: [{ source: Buffer.from('unretained after failure') }], + }); + await scope.bundles.putOrderedBatch({ + bundles: [{ members: [['invalid', 'not-an-application-handle']] }], + }); + }, + })).rejects.toMatchObject({ code: ErrorCodes.HANDLE_KIND_MISMATCH }); + await expect(cas.workspaces.inspect({ namespace, limit: 10 })).resolves.toMatchObject({ + returned: 0, + }); + git(repo, ['prune', '--expire=now']); + expect(objectExists(repo, provisional.oid)).toBe(false); + } finally { + await cas.close(); + rmSync(repo, { recursive: true, force: true }); + } +} + +async function admitGraph(workspace) { + return await workspace.batch({ + maxOperations: 3, + operation: async (scope) => await stageGraph(scope), + }); +} + +async function stageGraph(scope) { + const pages = await scope.pages.putBatch({ + pages: Array.from({ length: 8 }, (_, index) => ({ + source: Buffer.from(`compound-page-${index}`), + })), + }); + const leaves = await scope.bundles.putOrderedBatch({ + bundles: pages.map((page, index) => ({ + members: [[`payload/${index}`, page]], + })), + }); + return ( + await scope.bundles.putOrderedBatch({ + bundles: [ + { + members: leaves.map((leaf, index) => [`leaves/${index}`, leaf]), + }, + ], + }) + )[0]; +} + +async function assertAdmission({ admitted, cas, counted, repo, workspace }) { + const retainedOids = admitted.retention.handles.map((handle) => handle.oid); + const counts = counted.snapshot(); + expect(count(counts, 'update-ref') + count(counts, 'session:update-ref')).toBe(1); + expect(count(counts, 'session:fast-import')).toBe(1); + expect(count(counted.activeSessions(), 'fast-import')).toBe(0); + expect(git(repo, ['rev-parse', admitted.retention.ref])).toBe(admitted.retention.generation); + expect(reachableOids(repo, admitted.retention.ref)).toEqual( + expect.arrayContaining(admitted.retention.handles.map((handle) => handle.oid)) + ); + + git(repo, ['prune', '--expire=now']); + const firstLeaf = await cas.bundles.getMember({ + handle: admitted.value, + path: 'leaves/0', + }); + await expect( + cas.bundles.getMember({ + handle: firstLeaf.handle, + path: 'payload/0', + }) + ).resolves.toMatchObject({ handle: admitted.retention.handles[0] }); + + await workspace.release(); + git(repo, ['reflog', 'expire', '--expire=now', '--all']); + git(repo, ['gc', '--prune=now']); + expect(retainedOids.every((oid) => !objectExists(repo, oid))).toBe(true); +} + +function initializeRepository(repo, objectFormat) { + git(repo, ['init', '--bare', `--object-format=${objectFormat}`]); + git(repo, ['config', 'fastimport.unpackLimit', '100']); +} + +function git(repo, args) { + const result = spawnSync('git', args, { cwd: repo, encoding: 'utf8' }); + if (result.error) { + throw result.error; + } + if (result.status !== 0) { + throw new Error((result.stderr || result.stdout || 'git failed').trim()); + } + return result.stdout.trim(); +} + +function reachableOids(repo, ref) { + return git(repo, ['rev-list', '--objects', ref]) + .split('\n') + .filter(Boolean) + .map((line) => line.split(' ')[0]); +} + +function objectExists(repo, oid) { + return spawnSync('git', ['cat-file', '-e', oid], { cwd: repo }).status === 0; +} + +function count(counts, operation) { + return counts.get(operation) ?? 0; +} diff --git a/test/unit/domain/services/StagingWorkspace.compound.test.js b/test/unit/domain/services/StagingWorkspace.compound.test.js new file mode 100644 index 00000000..43c97cb0 --- /dev/null +++ b/test/unit/domain/services/StagingWorkspace.compound.test.js @@ -0,0 +1,305 @@ +import { describe, expect, it, vi } from 'vitest'; +import BundleService from '../../../../src/domain/services/BundleService.js'; +import { ErrorCodes } from '../../../../src/domain/errors/index.js'; +import PageService from '../../../../src/domain/services/PageService.js'; +import StagingWorkspaceRegistry from '../../../../src/domain/services/StagingWorkspaceRegistry.js'; +import parseApplicationHandle from '../../../../src/domain/value-objects/ApplicationHandle.js'; +import JsonCodec from '../../../../src/infrastructure/codecs/JsonCodec.js'; +import MemoryPersistenceAdapter from '../../../helpers/MemoryPersistenceAdapter.js'; +import MemoryRefAdapter from '../../../helpers/MemoryRefAdapter.js'; + +const CLOCK = Object.freeze({ now: () => new Date('2026-08-24T17:00:00.000Z') }); + +function fixture({ withWriteScope = true } = {}) { + const persistence = new MemoryPersistenceAdapter(); + if (!withWriteScope) { + Object.defineProperty(persistence, 'withWriteScope', { value: undefined }); + } + const ref = new MemoryRefAdapter(); + const pages = new PageService({ persistence, maxPageSize: 4096, clock: CLOCK }); + const services = {}; + const resolveHandle = vi.fn(async (value, context) => { + const handle = parseApplicationHandle(value); + return handle.kind === 'page' + ? await pages.resolveRoot(handle) + : await services.bundles.resolveRoot(handle, context); + }); + services.bundles = new BundleService({ + persistence, + codec: new JsonCodec(), + pages, + resolveHandle, + openHandle: (handle) => pages.open({ handle }), + clock: CLOCK, + }); + const registry = new StagingWorkspaceRegistry({ + persistence, + ref, + assets: { put: vi.fn(), putBatch: vi.fn(), adopt: vi.fn() }, + pages, + bundles: services.bundles, + resolveHandle, + crypto: { randomBytes: (length) => new Uint8Array(length) }, + clock: CLOCK, + }); + return { persistence, ref, registry }; +} + +describe('StagingWorkspace compound admission', () => { + it('anchors dependent page and bundle waves in one exact generation', async () => { + const { persistence, ref, registry } = fixture(); + const writeScope = vi.spyOn(persistence, 'withWriteScope'); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + const admitted = await workspace.batch({ + maxOperations: 2, + operation: async (scope) => { + const pages = await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([1, 2, 3]) }], + }); + expect(Object.isFrozen(scope)).toBe(true); + expect(Object.isFrozen(pages)).toBe(true); + const bundles = await scope.bundles.putOrderedBatch({ + bundles: [{ members: [['leaf/data', pages[0]]] }], + }); + expect(Object.isFrozen(bundles)).toBe(true); + return bundles[0]; + }, + }); + + expect(admitted.value.toString()).toMatch(/^git-cas:1:bundle:/u); + expect(admitted.retention.handles.map((handle) => handle.toString())).toHaveLength(2); + expect(new Set(admitted.retention.witnesses.map((witness) => witness.root.generation))).toEqual( + new Set([admitted.retention.generation]) + ); + expect(updateRef).toHaveBeenCalledOnce(); + expect(writeScope).toHaveBeenCalledOnce(); + }); + +}); + +describe('StagingWorkspace compound persistence compatibility', () => { + it('uses direct persistence when write-scope support is absent', async () => { + const { registry } = fixture({ withWriteScope: false }); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + const admitted = await workspace.batch({ + operation: async (scope) => + ( + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([1, 2, 3]) }], + }) + )[0], + }); + + expect(admitted.value.toString()).toMatch(/^git-cas:1:page:/u); + expect(admitted.retention.handles).toHaveLength(1); + }); +}); + +describe('StagingWorkspace compound retention and bounds', () => { + it('retains prior workspace targets in the one compound generation', async () => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + const prior = await workspace.pages.put({ source: new Uint8Array([0]) }); + + const admitted = await workspace.batch({ + operation: async (scope) => + ( + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([1]) }], + }) + )[0], + }); + + expect(admitted.retention.handles.map((handle) => handle.toString())).toEqual([ + prior.handle.toString(), + admitted.value.toString(), + ]); + expect(updateRef).toHaveBeenCalledTimes(2); + }); + + it.each([ + ['zero', 0], + ['fractional', 1.5], + ['above the hard maximum', 1025], + ])('rejects a %s operation bound without moving the workspace ref', async (_, maxOperations) => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + await expect( + workspace.batch({ + maxOperations, + operation: async (scope) => + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([1]) }], + }), + }) + ).rejects.toMatchObject({ code: 'INVALID_OPTIONS' }); + expect(updateRef).not.toHaveBeenCalled(); + }); +}); + +describe('StagingWorkspace compound admission constraints', () => { + it('rejects an empty compound operation without moving the workspace ref', async () => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + await expect(workspace.batch({ operation: async () => 'empty' })).rejects.toMatchObject({ + code: 'INVALID_OPTIONS', + }); + expect(updateRef).not.toHaveBeenCalled(); + }); + + it('fails the whole admission when its operation count exceeds the bound', async () => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + await expect( + workspace.batch({ + maxOperations: 1, + operation: async (scope) => + await Promise.all([ + scope.pages.putBatch({ pages: [{ source: new Uint8Array([1]) }] }), + scope.pages.putBatch({ pages: [{ source: new Uint8Array([2]) }] }), + ]), + }) + ).rejects.toMatchObject({ code: 'INVALID_OPTIONS' }); + expect(updateRef).not.toHaveBeenCalled(); + }); +}); + +describe('StagingWorkspace compound scope lifecycle', () => { + it('closes an escaped scope before returning its retained value', async () => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + let escapedScope; + + const admitted = await workspace.batch({ + operation: async (scope) => { + escapedScope = scope; + return ( + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([1]) }], + }) + )[0]; + }, + }); + + expect(admitted.retention.handles).toHaveLength(1); + await expect( + escapedScope.pages.putBatch({ + pages: [{ source: new Uint8Array([2]) }], + }) + ).rejects.toMatchObject({ code: 'WORKSPACE_STATE_INVALID' }); + expect(updateRef).toHaveBeenCalledOnce(); + }); +}); + +describe('StagingWorkspace compound failure containment', () => { + it('does not move the ref when a provisional write fails', async () => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + await expect( + workspace.batch({ + operation: async (scope) => + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array(4097) }], + }), + }) + ).rejects.toMatchObject({ code: 'PAGE_TOO_LARGE' }); + expect(updateRef).not.toHaveBeenCalled(); + }); + + it('does not retain a successful page wave when its dependent bundle wave fails', async () => { + const { ref, registry } = fixture(); + const updateRef = vi.spyOn(ref, 'updateRef'); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + await expect( + workspace.batch({ + operation: async (scope) => { + await scope.pages.putBatch({ pages: [{ source: new Uint8Array([1]) }] }); + return await scope.bundles.putOrderedBatch({ + bundles: [{ members: [['invalid', 'not-an-application-handle']] }], + }); + }, + }) + ).rejects.toMatchObject({ code: ErrorCodes.HANDLE_KIND_MISMATCH }); + expect(updateRef).not.toHaveBeenCalled(); + }); +}); + +describe('StagingWorkspace compound retention failure', () => { + it('preserves the previous generation when final retention fails', async () => { + const { ref, registry } = fixture(); + const retentionFailure = new Error('checked ref update failed'); + const updateRef = vi.spyOn(ref, 'updateRef').mockRejectedValueOnce(retentionFailure); + const workspace = await registry.open({ + namespace: 'git-warp/materializations', + ttlMs: 60_000, + }); + + await expect( + workspace.batch({ + operation: async (scope) => + ( + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([1]) }], + }) + )[0], + }) + ).rejects.toMatchObject({ + code: 'WORKSPACE_RETENTION_FAILED', + }); + + const retry = await workspace.batch({ + operation: async (scope) => + ( + await scope.pages.putBatch({ + pages: [{ source: new Uint8Array([2]) }], + }) + )[0], + }); + expect(retry.retention.handles.map((handle) => handle.toString())).toEqual([ + retry.value.toString(), + ]); + expect(updateRef).toHaveBeenCalledTimes(2); + expect(updateRef.mock.calls[1][0].expectedOldOid).toBeNull(); + }); +}); diff --git a/test/unit/domain/services/WorkspaceCompoundScope.test.js b/test/unit/domain/services/WorkspaceCompoundScope.test.js new file mode 100644 index 00000000..a96384b5 --- /dev/null +++ b/test/unit/domain/services/WorkspaceCompoundScope.test.js @@ -0,0 +1,187 @@ +import { describe, expect, it, vi } from 'vitest'; +import WorkspaceCompoundScope from '../../../../src/domain/services/WorkspaceCompoundScope.js'; + +const PERSISTENCE = Object.freeze({ kind: 'test-persistence' }); + +function artifact(id) { + return Object.freeze({ + handle: Object.freeze({ id, toString: () => id }), + }); +} + +function ignoreRejection(promise) { + void promise.catch(() => undefined); +} + +function fixture({ putPages, putBundles, maxOperations } = {}) { + const pages = { + putBatchWithPersistence: vi.fn(putPages ?? (async ({ id }) => [artifact(`page:${id}`)])), + }; + const bundles = { + putOrderedBatchWithPersistence: vi.fn( + putBundles ?? (async ({ id }) => [artifact(`bundle:${id}`)]) + ), + }; + const scope = new WorkspaceCompoundScope({ + pages, + bundles, + persistence: PERSISTENCE, + maxOperations, + }); + return { bundles, pages, scope }; +} + +describe('WorkspaceCompoundScope ordering', () => { + it('serializes concurrently started operations by invocation order', async () => { + const order = []; + let releaseFirst; + const firstGate = new Promise((resolve) => { + releaseFirst = resolve; + }); + const { pages, scope } = fixture({ + putPages: async ({ id }) => { + order.push(`start:${id}`); + if (id === 'first') { + await firstGate; + } + order.push(`finish:${id}`); + return [artifact(`page:${id}`)]; + }, + }); + + const result = await scope.execute(async (api) => { + const first = api.pages.putBatch({ id: 'first' }); + const second = api.pages.putBatch({ id: 'second' }); + await vi.waitFor(() => expect(order).toEqual(['start:first'])); + releaseFirst(); + return await Promise.all([first, second]); + }); + + expect(order).toEqual(['start:first', 'finish:first', 'start:second', 'finish:second']); + expect(pages.putBatchWithPersistence).toHaveBeenCalledTimes(2); + expect(result.operationCount).toBe(2); + expect(result.staged).toHaveLength(2); + }); +}); + +describe('WorkspaceCompoundScope operation bounds', () => { + it('refuses calls after overflow without extending the bounded queue', async () => { + let releaseFirst; + const firstGate = new Promise((resolve) => { + releaseFirst = resolve; + }); + const { bundles, pages, scope } = fixture({ + maxOperations: 1, + putPages: async () => { + await firstGate; + return [artifact('page:first')]; + }, + }); + + const execution = scope.execute(async (api) => { + const first = api.pages.putBatch({ id: 'first' }); + const overflow = api.pages.putBatch({ id: 'overflow' }); + const afterOverflow = api.bundles.putOrderedBatch({ id: 'after-overflow' }); + const settled = Promise.allSettled([first, overflow, afterOverflow]); + const refused = afterOverflow.then( + () => ({ status: 'fulfilled' }), + (error) => ({ status: 'rejected', error }), + ); + try { + const outcome = await Promise.race([ + refused, + new Promise((resolve) => setTimeout(() => resolve({ status: 'pending' }), 0)), + ]); + expect(outcome).toMatchObject({ + status: 'rejected', + error: { code: 'INVALID_OPTIONS' }, + }); + } finally { + releaseFirst(); + } + await settled; + }); + + await expect(execution).rejects.toMatchObject({ code: 'INVALID_OPTIONS' }); + expect(pages.putBatchWithPersistence).not.toHaveBeenCalled(); + expect(bundles.putOrderedBatchWithPersistence).not.toHaveBeenCalled(); + }); +}); + +describe('WorkspaceCompoundScope failure ordering', () => { + it('poisons later queued work after the first staged failure', async () => { + const failure = new Error('first operation failed'); + const { bundles, scope } = fixture({ + putPages: async () => { + throw failure; + }, + }); + + await expect( + scope.execute( + async (api) => + await Promise.all([ + api.pages.putBatch({ id: 'first' }), + api.bundles.putOrderedBatch({ id: 'second' }), + ]) + ) + ).rejects.toBe(failure); + expect(bundles.putOrderedBatchWithPersistence).not.toHaveBeenCalled(); + }); + + it('reports distinct callback and staged failures together', async () => { + const callbackFailure = new Error('callback failed'); + const stagedFailure = new Error('staged operation failed'); + const { scope } = fixture({ + putPages: async () => { + throw stagedFailure; + }, + }); + + const failure = await scope + .execute(async (api) => { + await api.pages.putBatch({ id: 'first' }).catch(() => undefined); + throw callbackFailure; + }) + .catch((error) => error); + + expect(failure).toBeInstanceOf(AggregateError); + expect(failure.errors).toEqual([callbackFailure, stagedFailure]); + }); +}); + +describe('WorkspaceCompoundScope callback failure ordering', () => { + it('poisons queued work as soon as the callback fails', async () => { + const callbackFailure = new Error('callback failed'); + const { bundles, pages, scope } = fixture(); + + await expect(scope.execute((api) => { + ignoreRejection(api.pages.putBatch({ id: 'first' })); + ignoreRejection(api.bundles.putOrderedBatch({ id: 'second' })); + throw callbackFailure; + })).rejects.toBe(callbackFailure); + + expect(pages.putBatchWithPersistence).not.toHaveBeenCalled(); + expect(bundles.putOrderedBatchWithPersistence).not.toHaveBeenCalled(); + }); +}); + +describe('WorkspaceCompoundScope falsy failure evidence', () => { + it('preserves a staged undefined rejection after the callback handles it', async () => { + const { scope } = fixture({ + putPages: async () => { + throw undefined; + }, + }); + + const outcome = await scope.execute(async (api) => { + await api.pages.putBatch({ id: 'first' }).catch(() => undefined); + return 'must not escape'; + }).then( + (value) => ({ rejected: false, value }), + (error) => ({ rejected: true, error }), + ); + + expect(outcome).toEqual({ rejected: true, error: undefined }); + }); +}); diff --git a/test/unit/facade/ContentAddressableStore.application-storage.test.js b/test/unit/facade/ContentAddressableStore.application-storage.test.js index e1f2135c..c1bba32e 100644 --- a/test/unit/facade/ContentAddressableStore.application-storage.test.js +++ b/test/unit/facade/ContentAddressableStore.application-storage.test.js @@ -5,8 +5,10 @@ import ContentAddressableStore, { CacheHit, CachePolicy, CacheSet, + DEFAULT_WORKSPACE_COMPOUND_OPERATIONS, ExpiringMarker, ExpiringSet, + MAX_WORKSPACE_COMPOUND_OPERATIONS, PageHandle, RetentionWitness, StagedAsset, @@ -73,6 +75,13 @@ describe('ContentAddressableStore application storage capabilities', () => { }); }); +describe('ContentAddressableStore compound workspace bounds', () => { + it('exports the default and hard operation limits', () => { + expect(DEFAULT_WORKSPACE_COMPOUND_OPERATIONS).toBe(64); + expect(MAX_WORKSPACE_COMPOUND_OPERATIONS).toBe(1024); + }); +}); + describe('ContentAddressableStore page cache configuration', () => { it.each([ ['pageCacheEntries', { pageCacheEntries: 0 }], diff --git a/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js b/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js index e65474e7..696876bf 100644 --- a/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js +++ b/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js @@ -490,6 +490,35 @@ describe('GitPersistenceAdapter operation-owned write scopes', () => { }); +describe('GitPersistenceAdapter operation-owned failure evidence', () => { + it('preserves operation and session-close failures together', async () => { + const operationError = new Error('operation failed'); + const closeError = new Error('fast-import close failed'); + const fastImport = { + writeBlobs: vi.fn().mockResolvedValue(['a'.repeat(40)]), + checkpoint: vi.fn().mockResolvedValue(undefined), + close: vi.fn().mockRejectedValue(closeError), + abort: vi.fn().mockResolvedValue(undefined), + }; + const adapter = new GitPersistenceAdapter({ + plumbing: sessionPlumbing({ fastImportSession: fastImport }), + policy: noPolicy, + }); + + const failure = await adapter.withWriteScope(async (persistence) => { + await persistence.writeBlob(Buffer.from('staged')); + throw operationError; + }).catch((error) => error); + + expect(failure).toBeInstanceOf(AggregateError); + expect(failure.errors).toContain(operationError); + expect(failure.errors.some((error) => ( + error === closeError || error?.errors?.includes(closeError) + ))).toBe(true); + expect(fastImport.abort).toHaveBeenCalledOnce(); + }); +}); + describe('GitPersistenceAdapter operation-owned oversized writes', () => { it('keeps an oversized blob on the genuine one-shot write path', async () => { const fastImport = { diff --git a/test/unit/scripts/measure-bounded-write-waves.test.js b/test/unit/scripts/measure-bounded-write-waves.test.js index 708ea152..0f2c5576 100644 --- a/test/unit/scripts/measure-bounded-write-waves.test.js +++ b/test/unit/scripts/measure-bounded-write-waves.test.js @@ -1,5 +1,10 @@ import { describe, expect, it } from 'vitest'; -import { comparison } from '../../../scripts/diagnostics/measure-bounded-write-waves.js'; +import { MAX_WORKSPACE_COMPOUND_OPERATIONS } from '../../../index.js'; +import { + comparison, + compoundComparison, + compoundOperationCount, +} from '../../../scripts/diagnostics/measure-bounded-write-waves.js'; describe('bounded write-wave benchmark comparison', () => { it('rejects semantically unequal benchmark modes', () => { @@ -8,6 +13,20 @@ describe('bounded write-wave benchmark comparison', () => { right: sample('right-digest'), })).toThrow(/semantic digest/u); }); + + it('rejects semantically unequal compound admission modes', () => { + expect(() => compoundComparison({ + left: sample('per-wave-digest'), + right: sample('compound-digest'), + })).toThrow(/semantic digest/u); + }); + + it('uses the public compound-operation ceiling', () => { + const maximumGroups = Math.floor((MAX_WORKSPACE_COMPOUND_OPERATIONS - 1) / 2); + + expect(compoundOperationCount(maximumGroups)).toBe(maximumGroups * 2 + 1); + expect(() => compoundOperationCount(maximumGroups + 1)).toThrow(/workspace operation ceiling/u); + }); }); function sample(semanticDigest) { diff --git a/test/unit/types/declaration-accuracy.test.js b/test/unit/types/declaration-accuracy.test.js index f1b7d4c9..8eb2508e 100644 --- a/test/unit/types/declaration-accuracy.test.js +++ b/test/unit/types/declaration-accuracy.test.js @@ -125,6 +125,18 @@ describe('Application-storage declaration accuracy', () => { }); }); +describe('Compound workspace declaration accuracy', () => { + it('declares its bounded scope and retained result', () => { + const declarations = read('index.d.ts'); + + expect(declarations).toContain('export interface WorkspaceCompoundScope {'); + expect(declarations).toContain('export interface WorkspaceCompoundResult {'); + expect(declarations).toContain('batch(options: {'); + expect(declarations).toContain('export const DEFAULT_WORKSPACE_COMPOUND_OPERATIONS: 64;'); + expect(declarations).toContain('export const MAX_WORKSPACE_COMPOUND_OPERATIONS: 1024;'); + }); +}); + describe('Persistence lifecycle declaration accuracy', () => { it('declares bounded tree reuse and deterministic resource release', () => { const declarations = read('index.d.ts');