Repository navigation
Fix: Fail Windows CI when a listed agentop test does not run, and drop alive - #2050
Conversation
…p alive Follow-up to rossoctl#2048 (review nits): - go-ci-windows runs the listed agentop tests with -v, and fails if a name has no "--- PASS" or "--- SKIP" line: `go test -run` passes when a name matches nothing, so a renamed test dropped out silently. - alive was a one-line wrapper for processAlive; the callers use processAlive directly. Part of rossoctl#1520. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Michal Jacovi <michal.jacovi@il.ibm.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 26 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
clawgenti
left a comment
There was a problem hiding this comment.
Summary
The alive removal is clean: no callers are left, processAlive is defined in both proc_unix.go and proc_windows.go, and the Linux agentop and Windows jobs pass on this head. The new check also works for single tests. I ran the bash logic against sample go test -v output. A missing exact name fails with ::error::<name> did not run, subtest lines (indented) do not count, and pipefail still passes a go test failure through tee. On this head the Windows step Build, vet and test cmd/agentop passed with no error annotations, so all eight names matched. I could not download the raw job log because the log blob host timed out from here.
One must-fix: the check matches on a prefix, so it does not cover the TestAddRecordOff_* family. That family is 5 of the 12 tests the step runs, so the promise in the new comment ("a renamed test fails the step instead of silently dropping out") does not hold there. Details inline.
Author: michaljacovi (CONTRIBUTOR, repo admin — maintainer)
Areas reviewed: CI/GitHub Actions (bash step on windows-latest), Go (cmd/agentop)
Agent/IDE config (.claude/.vscode): none
Commits: 1 commit, all signed-off: yes
CI status: passing (Windows and authbridge agentop jobs green on b0ab66c; README demo and macOS peerproc still pending)
| tests="TestRefuseUnsupported TestProcessAlive TestLockFile TestWritable TestTerminateRefusesOnWindows TestNoRecordWanted TestWithRecordOff TestAddRecordOff" | ||
| go test -v -run "^($(echo ${tests} | tr ' ' '|'))" . | tee "${RUNNER_TEMP}/agentop-tests.txt" | ||
| for t in ${tests}; do | ||
| grep -qE "^--- (PASS|SKIP): ${t}" "${RUNNER_TEMP}/agentop-tests.txt" \ |
There was a problem hiding this comment.
must-fix: This only checks prefixes, so the TestAddRecordOff entry does not do what the step's comment says. No test is named exactly TestAddRecordOff. The name stands for five tests (TestAddRecordOff_RefusesASettingsOverride, _AppendsToTheEnvironment, _RefusesAnUnreadableSettingsFile, _ChecksClaudeConfigDir, _RefusesAChildThatIsNotClaudeCode). The -run regex has no $, and this grep has no end anchor, so one --- PASS: TestAddRecordOff_<anything> line is enough for the check to pass. Rename or delete four of the five and the step stays green while they drop out of Windows CI. That is the exact failure this PR says it prevents. I confirmed this against sample output where only TestAddRecordOff_AppendsToTheEnvironment ran.
The same gap applies to the exact names. If a TestLockFileStale is ever added, TestLockFile could be renamed without any error.
Suggested fix: list the five tests by their full names in tests, anchor both ends of -run, and end the grep on the space before the duration:
go test -v -run "^($(echo ${tests} | tr ' ' '|'))\$" . | tee "${RUNNER_TEMP}/agentop-tests.txt"
for t in ${tests}; do
grep -qE "^--- (PASS|SKIP): ${t} \(" "${RUNNER_TEMP}/agentop-tests.txt" \
|| { echo "::error::${t} did not run"; exit 1; }
doneWith that change, each listed name stands for exactly one test, and the check is exact.
There was a problem hiding this comment.
Right, thanks. Fixed in 072bd26: the list names all five TestAddRecordOff_* tests in full (every other entry was already an exact name), -run is anchored with ^(…)$, and the check needs --- PASS: <name> ( or --- SKIP: <name> (. Checked locally with the step's script: the list passes (12 tests); an unknown name, or one of the five renamed, fails the step.
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com
From clawgenti's review of rossoctl#2050: the check matched prefixes, so TestAddRecordOff stood for five tests, and four could vanish while it stayed green. The list now names all five in full; every other entry is already one test's exact name. -run is anchored with ^(...)$, and the check needs "--- PASS: <name> (" or "--- SKIP: <name> (". Verified locally with the step's script: the list passes (12 tests); an unknown name, or one of the five renamed, fails the step. Part of rossoctl#1520. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Michal Jacovi <michal.jacovi@il.ibm.com>
clawgenti
left a comment
There was a problem hiding this comment.
Summary
072bd26 fixes the must-fix from my review of b0ab66c. The step now lists all five TestAddRecordOff_* tests by full name, -run is anchored with ^(…)$, and the check needs --- PASS|SKIP: <name> (. Each listed name now stands for exactly one test, so the comment's promise holds. No coverage is lost: the old unanchored regex matched the same 12 tests in cmd/agentop, and every listed name exists as a top-level func Test….
How I checked:
- I ran the step's script against real
go test -voutput on Linux at this head. All 12 tests report PASS or SKIP (TestTerminateRefusesOnWindowsskips off Windows), and the step exits 0. With a list ofTestLockFile TestAddRecordOff, it fails with::error::TestAddRecordOff did not runand exits 1, so a bare prefix no longer counts. go vet ./...ofcmd/agentoppasses for linux, windows and darwin. No caller ofalive(is left.processAliveis defined in bothproc_unix.goandproc_windows.go.- On Windows CI, the step
Build, vet and test cmd/agentoppassed with no annotations, which means none of the 12 names hit thedid not runpath. I could not download the raw job log because the log blob host timed out from here.
Author: michaljacovi (CONTRIBUTOR, repo admin — maintainer)
Areas reviewed: CI/GitHub Actions (bash step on windows-latest), Go (cmd/agentop)
Agent/IDE config (.claude/.vscode): none
Commits: 2 commits, all signed-off: yes
CI status: all checks passing on 072bd26
huang195
left a comment
There was a problem hiding this comment.
Summary
The alive removal is complete: no caller is left in cmd/agentop. All 12 listed names exist as top-level tests. The Windows job runs with shell: bash, so set -euo pipefail and the loop work. The Windows CI log at 072bd26 shows --- PASS for all 12 names.
Author: michaljacovi (MEMBER — maintainer)
Areas reviewed: CI (GitHub Actions), Go
Agent/IDE config (.claude/.vscode): none
Commits: 2 commits, all signed-off: yes
CI status: passing
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com
Follow-up to #2048 (review nits). Part of #1520.
go-ci-windowsruns the listed agentop tests with-v, and fails if any listed name has no--- PASSor--- SKIPline.go test -runpasses when a name matches nothing, so a renamed test used to drop out silently (review comment 4652840481).aliveremoved: it was a one-line wrapper forprocessAlive, and every caller now usesprocessAlivedirectly (review comment 4652840487).How it was tested
::error::TestDoesNotExist did not run.go vetofcmd/agentoppasses for windows, linux and darwin.Assisted-By: Claude (Anthropic AI) noreply@anthropic.com