Skip to content

feat: expose document processing history and cross-namespace listing - #421

Draft
Frankie-Xu wants to merge 4 commits into
Ontos-AI:mainfrom
Frankie-Xu:docs/Frankie-Xu/document-centric-list-history-rfc
Draft

Frankie-Xu wants to merge 4 commits into
Ontos-AI:mainfrom
Frankie-Xu:docs/Frankie-Xu/document-centric-list-history-rfc

Conversation

@Frankie-Xu

@Frankie-Xu Frankie-Xu commented Sep 14, 2026 •

Copy link
Copy Markdown

Summary

Clients can inspect a document's processing attempts without treating the job ledger as a file registry. GET /api/v1/documents/{document_id}/jobs (also v2) returns a paged ledger summary, including failed/running retries and older revisions. Private document history remains readable after archive. Authorization checks both the materialized document owner and each job owner; canonical result links take priority over metadata provenance.

GET /documents?all_namespaces=true explicitly pages across the credential owner's non-archived private documents so clients can derive labels from documents. Existing omitted/blank namespace calls keep selecting default; an explicit namespace and all_namespaces=true conflict with HTTP 400 INVALID_ARGUMENT. Shared demo documents retain explicit namespace listing and expose an empty processing history.

This implements the document-centric direction discussed in #231 and #232. Namespace remains client-owned, documents remain the corpus resource, and jobs remain the processing ledger. Those PR branches are unchanged. No namespace registry, namespace-filtered job listing or job deletion is added.

Draft design choices for review: the all_namespaces parameter and null envelope namespace; history summary fields; canonical-result/metadata association precedence; empty shared-demo history. Initial uploads with no materialized document still require retaining the returned job ID. Counts/pages are separate reads with existing offset pagination semantics.

Verification

  • 30 actual HTTP/PostgreSQL history/document/job-read contracts passed (7 new history/list cases). These include v1/v2 old revision selection using the result ID returned by history, both before and after archive, tenant isolation, canonical-link precedence, shared demo privacy, namespace defaults and pagination. Synthetic fixtures use task-local storage directories.

  • Repository Ruff (ruff check apps packages) and scoped Pyright passed for all changed/new Python files; registered history/list parameters verified in the actual v1/v2 OpenAPI schema.

  • Full API/worker/shared Pyright in the API-only environment did not pass: 57 missing worker/optional imports, 2 diagnostics in unchanged worker files and 1 source-resolution warning. No changed API file had a diagnostic; full worker dependency validation remains unverified.

  • Independent source/spec review passed, including shared demo boundaries and consuming an older history result through the existing chunks selector before/after archive; Python parsing and git diff --check passed.

  • Worker parsing, model calls and deployment were not exercised for this read-only API change.

  • Current-head GitHub PR CI, Secret Scan and CodeQL workflows report action_required; the PR has no reported check rollup. Remote CI has not passed.

Deployment Notes

  • Additive endpoint/query parameter. No migration, environment variable, storage format or worker change.
  • Existing list defaults and job inspection routes remain compatible. Archive does not mutate or hide job ledger rows.
  • History queries use existing owner/result links and JSON provenance; no new index is introduced in this Draft.

Checklist

  • Tests were added or updated when behavior changed
  • Public docs, examples, or OpenAPI contracts were updated when needed
  • Database migrations are idempotent and safe to deploy — no migration
  • Logs, errors, and validation paths avoid leaking secrets or user data
  • The pull request description explains any breaking or user-visible change

Record maintainer ownership rules from Ontos-AI#232/Ontos-AI#231 so a later API
change does not revive namespace or job-listing shortcuts.

Co-authored-by: Frankie-Xu <Frankie-Xu@users.noreply.github.com>
@Frankie-Xu
Frankie-Xu marked this pull request as draft October 7, 2026 10:47
@Frankie-Xu Frankie-Xu changed the title docs: draft document-centric list and history RFC feat: expose document processing history and cross-namespace listing Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants