Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Latest tagged release: [v1.2 — architecture cohesion release](https://github.c
- [`docs/v0.6-to-v1-artifact-diff.md`](docs/v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff
- [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md): JSON/JSONL schema contracts for reviewer-facing evidence artifacts
- [`docs/schema-compatibility-matrix.md`](docs/schema-compatibility-matrix.md): schema versions, artifact paths, and v1.1-to-v1.2 compatibility notes
- [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md): release artifact diff contract for reviewer-facing outputs
- [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md): release artifact diff contract and executable structured triage report for reviewer-facing outputs
- [`docs/vocabulary.md`](docs/vocabulary.md): cross-demo vocabulary for events, hits, signals, bounded correlation, findings, summaries, reports, and audit traces
- [`docs/README.md`](docs/README.md): current route, supporting docs, and historical release evidence

Expand Down Expand Up @@ -106,6 +106,7 @@ Other demo entrypoints:
Useful inspection commands:

- `telemetry-lab summarize --input data/raw/sample_events.jsonl`
- `python scripts/artifact_contract_diff.py --expected path/to/expected-artifacts --actual path/to/actual-artifacts --json-out artifact-diff.json`

For CSV inputs, pass a `.csv` file to `--input`; use `--timestamp-col` when the timestamp column is not named `timestamp`.

Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ This directory separates the current reviewer route from supporting design notes
- [`v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): additive artifact contract and compatibility diff from the fourth demo to the fifth
- [`evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON/JSONL schema contracts for reviewer-facing evidence artifacts
- [`schema-compatibility-matrix.md`](schema-compatibility-matrix.md): schema versions, artifact paths, and compatibility labels
- [`reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes
- [`reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract and executable structured triage report for reviewer-facing artifact changes
- [`vocabulary.md`](vocabulary.md): cross-demo vocabulary for evidence workflow terms and bounded correlation
- [`architecture.md`](architecture.md): local file-based workflow diagram
- [`roadmap.md`](roadmap.md): v1 reviewer contract stabilization phase
Expand Down
49 changes: 49 additions & 0 deletions docs/reviewer-artifact-diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,55 @@ the local, file-based artifacts listed in [`docs/reviewer-pack.md`](reviewer-pac
and the schema-covered evidence artifacts in
[`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md).

## Executable Triage Report

Use the standalone comparator when a regeneration mismatch needs more context:

```bash
python scripts/artifact_contract_diff.py \
--expected path/to/committed-artifacts \
--actual path/to/regenerated-artifacts \
--json-out artifact-diff.json
```

The human summary and strict
[`artifact-contract-diff/v1`](../schemas/artifact_contract_diff.schema.json)
report expose missing, extra, and changed relative paths. JSON and JSONL entries
add record counts, top-level keys, exact schema/version markers, and run-manifest
digest fields when present. CSV, Markdown, text, JSON, and JSONL normalize CRLF
and CR to LF before comparison. Existing binary files are presence-only, so the
tool does not turn renderer-dependent PNG bytes into a reproducibility claim.

The report is deterministic: it contains no timestamp, artifact body, or
absolute checkout path. Exit status is `0` for no differences, `1` for contract
differences, and `2` for invalid or unreadable input. This tool explains a
mismatch; it does not replace `python scripts/regenerate_artifacts.py --check`,
accept regenerated output, or infer compatibility labels automatically.

Write `--json-out` outside both compared roots. The CLI rejects an output path
inside either tree and atomically replaces an existing external report only
after the new JSON has been written successfully.

### Report Semantics

| Field | Contract |
| --- | --- |
| `status` | `unchanged` requires an empty `differences` array; `changed` requires at least one difference. |
| `summary.*_files` | `expected_files = unchanged_files + missing_files + changed_files + presence_only_files`; the corresponding actual count substitutes `extra_files` for `missing_files`. |
| `unchanged_files` | Counts comparable text/JSON/JSONL files only. It does not include binaries checked for presence. |
| `differences[].status` | `missing` has only an expected snapshot, `extra` has only an actual snapshot, and `changed` has both. |
| `change_reasons` | Missing/extra use their single path reason. Changed comparable artifacts start with `content-changed` and may add structure, schema-version, or run-manifest-digest reasons. |
| `comparison_digest` | SHA-256 of the comparison bytes. Text-like artifacts use strict UTF-8 with CRLF and lone CR normalized to LF. |
| `comparison_size_bytes` | Length of those normalized comparison bytes, not necessarily the on-disk byte size. |
| `structure` | For changed, missing, or extra JSON/JSONL, records the container, record count, union of top-level keys, and validated schema/digest markers when present. |
| `presence_only_paths` | Sorted binary paths present in both trees. Their bytes are intentionally not compared or summarized. |

The local triage contract is bounded: each root may contain at most 10,000
files; structured summaries accept at most 64 MiB per changed JSON/JSONL file,
4,096 structural keys or schema markers, and 10,000 entries per run-manifest
digest map. Invalid digest shapes, unsafe embedded paths, symlinks, special
files, or exceeded limits fail closed with exit `2` and no JSON report.

## Required Release Diff Sections

Each release artifact diff must include:
Expand Down
Loading
Loading