Skip to content

feat: add P1 result-aware experiment iteration - #9

Merged
Air000000 merged 33 commits into
mainfrom
feat/p1a-result-aware-agent
Sep 22, 2026
Merged

Air000000 merged 33 commits into
mainfrom
feat/p1a-result-aware-agent

Conversation

@Air000000

@Air000000 Air000000 commented Sep 20, 2026 •

Copy link
Copy Markdown
Owner

Scope

P1-A/P1-B implementation under the approved plan:

docs/superpowers/plans/2026-09-20-labpilot-p1-result-aware-iteration.md

PR remains Draft until the final P1-B review/full-CI gate.

P1-A implementation review — ACCEPTED

Reviewed head:

e3116ecb7f5799e550317a02ea35d868ffbb23b0

Remote evidence:

  • GitHub CI #300: SUCCESS
  • local-equivalent CI gate: SUCCESS
  • production image/build-context hygiene: SUCCESS
  • local ./scripts/check_ci.sh: 277 passed, 1 skipped, 5 deselected

Task 1 — AgentRun result provenance

ACCEPTED.

  • Alembic head 0013_result_aware_agent_runs
  • nullable source_batch_summary_revision_id
  • RESTRICT FK to exact BatchSummaryRevision
  • initial Agent compatibility preserved

Task 2 — ResultContext + ContextCompiler

ACCEPTED.

  • Agent-owned bounded ResultContext
  • exact summary/result facts participate in canonical request hash
  • initial request/hash semantics preserved when ResultContext absent
  • 128 KiB ResultContext bound under existing 256 KiB request cap

Task 3 — exact result binding adapter

ACCEPTED.

  • same BatchRow FOR UPDATE linearizes latest-at-create
  • latest check + AgentRun insert happen in same UoW
  • exact pinned summary load; no later-latest substitution
  • no raw Artifact/stdout/stderr/raw invalid metric promotion
  • agent_runtime -> results_validation dependency remains unchanged

Non-blocking note:
the lock-race test currently uses short sleeps to order waiters. It is stable in local/CI evidence; harden test synchronization only if flakiness appears.

Task 4 — Follow-up AgentRun service/API

ACCEPTED.

  • optional Follow-up ports preserve all initial Agent construction paths
  • follow-up creation fails closed when runtime ports absent
  • local-only + remote provider blocks before ResultContext load/provider call
  • exact ResultContext reaches allowed provider
  • stop(goal_satisfied) completes without Proposal/Version/Batch writes
  • follow-up propose and persisted propose replay fail closed until P1-B
  • production composition uses one SqlAgentRepository and one shared SqlResultAwareAgentAdapter

P1-B Task 5 — ACCEPTED

Reviewed head:

fdb9738614c36ffa29d995ecc7593bb5366fcb65

Evidence:

  • GitHub CI #302: SUCCESS
  • local proposal service tests: 24 passed
  • proposal loop regression: 8 passed
  • Agent unit regression: 88 passed
  • architecture guard: 12 passed
  • local full gate: 282 passed, 1 skipped, 5 deselected

Verified boundaries:

  • initial create_agent_revision() semantics unchanged
  • follow-up Agent revision stays in the existing Experiment
  • Experiment FOR UPDATE serializes revision-number allocation
  • no second revision allocator introduced
  • no new Experiment is created by follow-up revision creation
  • exact replay requires same payload + Project + Experiment
  • cross-Experiment replay of one origin AgentStep is rejected
  • origin-step unique-conflict recovery returns only the exact winning revision
  • Task 6 remains unimplemented at this reviewed head

Next authorized step

P1-B Task 6 only:

Follow-up ProposeDecision
→ load exact pinned ResultContext
→ derive Experiment from ResultContext
→ create_follow_up_agent_revision()
→ link ProposalRevision to AgentRun
→ proposal_created

Do not verify, approve, materialize ExperimentVersion V2, or create/start a Batch in Task 6.

Explicitly excluded

  • historical Result Tool / P1-C
  • controlled code modification / Worktree
  • automatic Batch start
  • new Agent decision kind
  • UI redesign
  • Slice 2 hardening expansion

P1-B Task 6 — ACCEPTED

Reviewed implementation head:

ae079be252fd62eff415ef040af4228ab45e2d61

Verified boundaries:

  • fresh Follow-up propose derives Experiment only from exact pinned ResultContext
  • proposal Snapshot mismatch fails before Proposal write
  • follow-up materialization uses same-Experiment Proposal service
  • crash after committed R2 replays the same origin AgentStep
  • retry links the same R2; no R3 is created
  • provider is not called again during persisted-proposal replay
  • no automatic verification, approval, ExperimentVersion, or Batch is created

Task 6 is included in later successful CI #310 on the current PR head.

P1-B Task 7 — ACCEPTED

Current reviewed head:

ab73e2c5a96a9b661b418ebf59d1169c23afad11

Remote evidence:

Tranche acceptance proves:

Experiment E
→ R1
→ V1
→ B1
→ exact S1
→ Follow-up AgentRun A2
→ R2 in same E
→ explicit deterministic verification
→ explicit human approval
→ V2 in same E
→ no automatic Batch for V2

Evidence boundary also proves:

  • selected valid metric evidence and validation report identity reach ResultContext
  • excluded Trial/reason and Trial conditions reach ResultContext
  • raw file sentinel does not reach provider-visible context
  • post-summary raw MetricObservation mutation does not override or leak past the exact pinned summary

Stop-path acceptance proves no R2/V2/Batch writes when the Agent returns goal_satisfied.

Next authorized step

Task 8 only: focused architecture/data-boundary/regression gate.

No production implementation changes are expected in Task 8. Any failure must be classified before code is changed.

Task 9 full repository gate / closeout remains not started.

Task 8 — ACCEPTED

Reviewed head:

e8caaf668fed2b5e5878393fa0c47c001ba218e9

Remote evidence:

  • GitHub CI #312: SUCCESS
  • production image/build-context hygiene: SUCCESS
  • PR head matches reviewed head

Task 8 focused regression gates all passed:

  • architecture
  • migration
  • Agent unit/integration
  • same-Experiment Proposal service
  • initial Slice 1 API/proposal loop
  • Result-Aware API
  • Result Plane
  • P1 tranche acceptance
  • OpenCode constructor compatibility
  • Ruff / mypy

The only Task 8 code change was test-only concurrency hardening:

  • replaced timer-based lock ordering with an explicit test barrier after real BatchRow FOR UPDATE acquisition
  • focused race repeated 5x successfully
  • no production code changed

Verified invariants remain:

  • agent_runtime -> results_validation dependency unchanged
  • Agent does not auto-verify
  • Agent does not auto-approve
  • Agent does not auto-create/start Batch
  • initial Proposal path unchanged
  • Follow-up Experiment authority comes from exact pinned ResultContext

Next authorized step

Task 9 only: final repository-equivalent gate, closeout/status documentation, final CI/hygiene verification, then PR merge-readiness review.

Do not reopen global architecture.

Air000000 and others added 30 commits September 20, 2026 14:20
@Air000000 Air000000 changed the title feat: add P1-A result-aware AgentRun foundation feat: add P1 result-aware experiment iteration Sep 22, 2026
@Air000000
Air000000 marked this pull request as ready for review September 22, 2026 06:01
@Air000000
Air000000 merged commit 03d1297 into main Sep 22, 2026
2 checks passed
@Air000000
Air000000 deleted the feat/p1a-result-aware-agent branch September 22, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant