Repository navigation
feat: add inference setup and baseline experiment contract - #14
Merged
Merged
Conversation
Accept inference-summary v2 alongside v1. Preserve prepared-problem and baseline provenance, distinguish cold/warm/resume from compilation/cache state, and refuse incompatible cost comparisons. Agent: Codex; model: GPT-6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inference records currently lack a stable setup/prepared-problem contract, so consumers cannot reliably distinguish sampler cold starts, baseline-informed warm starts and checkpoint resumptions. This adds
inference-summary@2alongside v1, with validated baseline-stage/artifact references, separate preparation and initialization clocks, and explicit work-unit definitions.Comparisons refuse mismatched or unknown start/environment/problem conditions. Archived or unassessed baseline candidates remain visible but cannot become selected trusted references. The live registry stays on v1 until the producer and consumer migration; this is phase 1 of the approved setup-oriented redesign.
Scripts Changed
insight/catalogue.py: setup, prepared problem, initialization provenance and comparison validation.insight/summary.py: v1/v2 routing and comparison integration.REFERENCE.md: producer contract and staged migration.tests/test_catalogue.py,tests/fixtures/lens_summary_v2.json: synthetic baseline/cold/warm/resume evidence, malformed inputs, compatibility and immutable ingest/cache replay.Validation
python -m pytest tests -q).ruff check .andruff format --check .passed.bin/pyauto-insight check --offlinepassed against the unchanged 56-record v1 capture.git diff --checkpassed.release validation incomplete: no rehearsal for current source. No red/yellow reasons; freeze clear. Development shipping allowed by the Heart contract; this is not release readiness.Limits and next phases
Artifact hashes and scientific identities are producer declarations; the executor must verify bytes before reuse. No scientific runs, baseline acceptance, new sampler integration or dashboard redesign occurs in this reader-first phase. Producer/script migration, setup UI, assistant/wiki lookup and curated literature candidates remain tracked in the Mind umbrella
draft/feature/pyautoinsight/inference_setup_redesign.mdand must coordinate with the existing search-extensibility epic.Tier: judge — human
/prm; no merge authorization.Closes #13.