[sfloess]

Summary

Completes the source lifecycle, operational diagnostics, and collaboration capabilities on top of mainline.

Source lifecycle and reclamation

• Reversible logical tombstones with explicit restoration boundaries.
• Pluggable deferred/immediate reclamation strategies and composable retention, retry, and audit decorators.
• Durable actor/reason/request-time/retention metadata and append-only deletion audit events.
• Legal-erasure override bypasses future retention, preserves the requested deadline and override decision, and prevents restoration.
• Atomic SQLite reclamation claims, idempotent retries, restart-safe scheduling, and failure recovery.
• Physical cleanup across Documents, Knowledge, Vectors, and source-specific Graph materializations; shared Graph entities remain.
• Index purge state is written only after all downstream materializations have been removed.

Collaboration

• Provider-neutral human/agent identities, projects, and membership roles.
• Attributable, durable contributions and chronological activity history.
• Human/agent work handoffs with checkpoints, acknowledgement, completion, superseding, and append-only history.
• Provenance-linked project synthesis and refresh.
• Competing proposals, objections, attributable decisions, and enforceable human approval gates.
• Control Center collaboration workbench for participant/project setup, contributions/reviews, activity refresh, handoffs, checkpoints/transitions, synthesis, proposals, and decisions.
• Existing SQLite persistence is reused. No new service, database, orchestration engine, or model-provider dependency was added.

Generic contract audit

• Generic Python deletion strategy/result/decorator contracts preserve context and result types.
• Typed reclamation job contract.
• Foundational store protocols audited; unnecessary generic parameters avoided.
• No Java implementation fabricated because this repository has no Java source or build module. REST remains language-neutral.

Operational work retained in this branch

• Bounded robots-aware same-origin crawling with durable jobs, partition-aware lease recovery, explicit batch ingestion, and recurring schedules.
• Persisted ingestion/retrieval execution traces with bounded metadata and stage timings.
• Weighted reciprocal-rank fusion, offline evaluation, opt-in Thompson-adaptive channel weighting, and Thompson diagnostics.
• Control Center source operations and retrieval/pipeline diagnostics.
• Transcript ingestion uses five-minute renewable owner leases, per-message heartbeats, owner-conditional completion/failure, duplicate-dispatch exclusion, and expiry-only startup recovery. Migration 0038_transcript_job_leases gives pre-lease running jobs a fifteen-minute upgrade grace period. Two-independent-SQLite-instance tests cover live-owner protection, expiry recovery, retry, and stale-owner rejection.

Issue disposition

All repository issues in the PR completion scope are resolved:

• Introduce Generic Contracts Across Java and Python #26 closed as not planned: Python generic contracts audited; no Java implementation exists to mirror.
• Define and Implement a Pluggable Deletion Lifecycle #27 closed as completed: pluggable deletion lifecycle.
• Define project collaboration and contributor identity contracts #43 closed as completed: identity and collaboration contracts.
• Record attributable project contributions and activity history #44 closed as completed: attributable contributions and activity history.
• Implement project work handoffs between humans and agents #45 closed as completed: human/agent work handoffs.
• Generate project synthesis and manage competing proposals and decisions #46 closed as completed: synthesis, proposals, and decisions.

Validation

Latest verified PR head b6923abb6d81637fe2b5cbac5a1abc39ed0e2ba3:

• 365 passed, 1 skipped, 1 warning, 0 failed (GitHub Actions run 42407150262).
• Installer, backup, and restore shell syntax passed.
• Control Center JavaScript syntax passed.
• The end-to-end hybrid partition regression forces a Graph-only candidate into fusion and verifies that foreign-partition candidates never reach results or reranker inputs.
• The single skip is the pinned Nomic ONNX model integration test because its model graph is unavailable on the runner. The single warning is Starlette's httpx TestClient deprecation notice.
• GitGuardian's earlier scan passed: 250 commits scanned, no secrets detected.

SonarCloud is intentionally deferred. Do not run SonarCloud/SonarQube analysis or spend implementation time on its findings until feature parity is complete, every repository issue is resolved, all required unit and integration tests are passing, and the outstanding PR queue is consolidated. Sonar is a final quality gate, not a current implementation gate.

Review and merge policy

• No outside reviewers have been requested; none are to be requested during implementation.
• Do not merge this PR yet. First complete feature parity, resolve all issues, validate unit and integration tests, and merge or close the outstanding PR queue.
• After implementation and PR consolidation are complete, run SonarCloud and bring in outside reviewers. Address all findings before the final merge.

[sfloess]

Focused review findings and current disposition

I rechecked the source lifecycle, crawler, scheduling, trace persistence, and migration paths, then committed the following corrections:

1. P1 - Crawl startup recovery with multiple instances: crawl jobs now use the five-minute renewable heartbeat lease, and startup recovery only interrupts jobs whose lease has expired. Recovery runs in every configured SQLite partition, not just default. Regression coverage verifies that fresh leases survive startup and stale jobs in authenticated partitions are recovered.
2. Transcript checkpoint recovery: automatic startup recovery is deliberately not invoked for transcript jobs yet. Those request-owned jobs do not have a renewable lease, so bulk-marking them interrupted on every replica startup would recreate the multi-instance race. Transcript lease/heartbeat support should precede automatic recovery.
3. P2 - Robots crawl-delay: the five-second cap is removed. The configured delay is honored both before the initial page and between subsequent pages.

Both Python CI runs for the current head passed, as did GitGuardian.

SonarCloud remains a separate failure on the current head: Security Rating C and Reliability Rating C on new code, where the quality gate requires A. The Python workflow has no Sonar step. The GitHub connection cannot change Sonar project settings or read the repository's required-check protection, so I have not claimed it was disabled. That still needs an administrator action if it is blocking the merge.

There is no independent formal review yet. GitHub will not accept a formal review from the PR author. This PR remains draft and has not been merged.

[sfloess]

P0 retrieval-scope regression coverage

The current partition middleware resolves the authorized principal/partition, and InternalRestClient forwards authorization, partition selection, and correlation ID across internal REST calls. Existing tests covered direct search and store isolation, but did not exercise the composed Hybrid endpoint's internal REST path.

Added an integration regression test on integration/source-lifecycle-mainline-2026-11-04 in commit 48cc0e8f428bd65037c431d2523c5f10539db35a. It runs Hybrid search independently as two principals with the same source ID in separate partitions, asserts the response includes the Graph/intent composition fields, and verifies neither principal sees the other's document. CI is running; this is not marked verified until the workflow completes.

[sfloess]

Vector REST contract correction

The first full CI run after the Hybrid partition regression test completed with 351 passed, 1 skipped, and 2 failures. The new Hybrid partition test passed. The two failures exposed a second, independent defect: /api/v1/vector/collections/{name}/query was registered twice. FastAPI matched the earlier legacy handler, bypassing the newer model and dimension checks.

Removed the duplicate handler and consolidated the contract into one route. The consolidated route preserves the legacy limit/generation_id inputs, supports top_k, enforces model and dimensions against the collection, validates finite numeric embeddings, and checks active generation identity. Commits: e79631ee0b84e75368fd7490b2e684e61da5dd39 and e19e2e11ea917c7c7017ac62dc485256ee6f98d7.

The workflow for the consolidated route is running. No test is considered green until CI confirms the latest head.

[sfloess]

CI verification after Vector route consolidation

Latest commit e19e2e11ea917c7c7017ac62dc485256ee6f98d7 completed the Python workflow successfully:

• 353 passed, 1 skipped, 1 warning, 0 failed.
• Installer shell syntax passed.
• Control Center JavaScript syntax passed.
• The Hybrid partition regression test passed.
• The Vector query model-mismatch and invalid-dimension regression tests passed after removing the duplicate route.

The single remaining warning is recorded by CI; no tests were skipped or weakened as part of this correction. PR #47 remains draft and unmerged. SonarCloud and outside review remain deferred.

[sfloess]

Follow-up verification: the Hybrid partition test now asserts actual Graph-expanded candidate sources and chunks in each partition. Latest workflow on f45f680c1dbbe3af2e87433070d5e6bd07dc4cfa passed 353 passed, 1 skipped, 1 warning, 0 failed. The Vector route consolidation and the stricter Hybrid/Graph isolation regression are both green.

[sonarqubecloud]

Quality Gate failed

Failed conditions
E Security Rating on New Code (required ≥ A)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE