Repository navigation
🤖 test: make Kind E2E reruns idempotent, read previous and Coder logs, add MCP failure rows - #403
Conversation
…CP failure rows (#390) Change-Id: I238039e70250ec6c8023a1928d747b92c86510ba Signed-off-by: Thomas Kosiewski <tk@coder.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex security review |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
…lete (#390) The CoderAIProvider and CoderMCPServer row groups forced a 2 s namespace delete timeout on every row. Rows that need the delete to finish (ai-ns-no-release, mcp-ns-no-release) could time out on a loaded host before they reached the ControlPlaneGone release check. Only the *-ns-stuck rows keep 2 s. agent-start-error's wall-clock bound becomes 120 s: still far below the 300 s agent timeout it guards against. Change-Id: I9c8b6668bb2b8cb5ee89158083c58fcb67e7e4b5 Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex review |
|
@codex security review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 10e5d3957e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
A rerun that reused e2e-mcp-llm still POSTed a new stub-model chat model, so each rerun added another default model. The driver now lists the organization's chat models and reuses stub-model of the provider when it is enabled and the default. Change-Id: I0b59c66deefbd5ded62340d57129375473104f36 Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex review |
|
@codex security review |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
The Kind E2E driver now survives a local rerun in a namespace that a failed run kept, reads the operator log across a container restart, and checks the Coder pod log for the MCP server key. The offline suite gains failure rows for three MCP checks that no test could make fail, and the CI failure dump now shows the
CoderMCPServerobjects and the MCP stub log.Fixes #390
Refs #156 (the two E2E follow-ups in the advisor comment of #370)
Background
#390 lists four follow-ups from the final assessment of #389. The last comment of #156 adds two from the advisor check of #370:
gw-leak-checkalmost always passes on unlicensed Kind.e2e-ns-keepControlPlaneGone check readkubectl logs deploy/coder-k8s, which covers only the current container. An operator restart loses the earlier lines.Implementation
hack/e2e-workspace-lifecycle.sh:coder_ensure: it lists the collection first and reuses an object with the same name or slug, else it POSTs. It is used fore2e-taken,e2e-mcp-llmande2e-mcp-taken. A reusede2e-mcp-llmmust be enabled with base URLhttp://mcp-stub.<ns>.svc:8080/v1, or the driver fails.key_secret createreplaces a Secret that already exists (AlreadyExists). Without that, a rerun fails atcannot create Secret e2e-ai-key(AI P1) before it reaches the first 409.pod_logs <ns> <selector> <file>: for each container of each non-terminating pod it reads the current log, and also--previouswhenrestartCount > 0. All four operator log reads (AI rotation, AI Gateway leak check, MCP P1 leak check, namespace ControlPlaneGone checks) use it.app.kubernetes.io/name=coder-control-plane,app.kubernetes.io/instance=coder)..github/workflows/ci.yaml(#390.3): the failure dump addskubectl -n coder get codermcpservers -o yamlandkubectl -n coder logs deploy/mcp-stub --tail=200. The objects hold no key values (the driver checks that), and the stub logs only request methods and tool counts.hack/e2e-workspace-lifecycle_test.sh(#390.1 and coverage for the rest):managerwithrestartCount: 1, and the ControlPlaneGone release lines and thegw-log-leakline appear only in its previous log. So thepassscenario fails if the driver does not read--previous. The fake Coder pod (restartCount: 0) refuses--previous, so the driver must not ask for a previous log when there is none.AlreadyExistsfor an existing Secret, as the real ones do.mcp-unlisted-writesa CoderMCPServer of an unlisted organization wrote to Codermcp-conflict-writesupdated_atin Codera refused CoderMCPServer wrote to Codermcp-ns-no-releasethe operator log has no ControlPlaneGone release of MCP server e2e-mcp-keepmcp-coder-log-leakCoderMCPServer e2e-mcp, the operator log or the Coder log holds the MCP server key (1 lines)rerun-kept-namespace(passes)e2e-taken,e2e-mcp-llm,e2e-mcp-takenin the fake Coder and the five driver SecretsValidation
passfails withno ControlPlaneGone release of AI provider e2e-ns-keep,rerun-kept-namespacefails withcannot create Secret e2e-ai-key, andmcp-coder-log-leak,mcp-ns-no-releaseandgw-log-leakfail with a different message than the row expects.mcp-unlisted-writesandmcp-conflict-writesalready fail correctly: they add coverage for existing checks.make test-scripts(which runsbash ./hack/e2e-workspace-lifecycle_test.sh): ALL OFFLINE TESTS PASSED, 103 scenarios, 137 checks, 12 min on this host.shellcheckon both scripts: clean.go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.10: clean.make verify-vendor,make test,make build,make lint: pass (withGOROOTpinned to Go 1.26.10 on this host).hack/e2e-workspace-lifecycle*.sh. I did not run Kind locally.Risks
pod_logsfails the step ifkubectl logs --previousfails after a restart (for example, the kubelet removed the dead container). That is a false failure, never a false pass.--previouscovers container restarts inside one pod only.Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:high