Repository navigation
fix(evals): separate semicolon-joined command and review claims - #215
Merged
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 10.0.2 qualification stopped when its handoff grader rejected two true claims joined by a semicolon:
node scripts/verify.mjs passed unchanged; independent review passed with no findings.Changing only that separator to a period passed the original checker against the same native evidence.Recognize the semicolon using the existing independent-review sibling boundary. Both claims still require their own accepted native proof, and unknown suffixes, conditional reviews, conflicting exits, changed scripts and incomplete/stale/unsubmitted evidence remain rejected. Regression and separator-equivalence tests cover the observed form and adversarial variants.
This is an evaluator-only fix: rebuilding preserves the exact product archive
f670ad4f…e2bband all eight build outputs. The original stopped report stays NOT VERIFIED (2 passed, 1 failed, 78 missing); all 19 original receipt files are unchanged. Four starts remain consumed and 122 unused; no further native model calls, refunds, partial pooling or campaign restart occurred. A fresh complete campaign requires a newly bound approval.Validation: clean push preflight passed (5,780 passed, 27 skipped, zero failures). The targeted regression fails before and passes after; 1,136 delivery/scenario tests, typecheck and 13/13 gated replays pass. Exact product archive, eight build outputs and 19 original failed receipts are unchanged.
Reviewed with sol-loop: 1 round with GPT-6.1 Sol on high. Sol verdict: approved.
Created with GPT-6.1 Sol in T3 Code.