Repository navigation
ci: pin kind-e2e runner to ubuntu-24.04 - #6225
Conversation
ubuntu-latest now resolves to the ubuntu-26.04 runner image. Its AppArmor fusermount3 profile only allows FUSE mounts under a few host paths, so the JindoRuntime FUSE mount used by the kind e2e tests is denied and every kind-e2e-test matrix job fails. Pin the kind-e2e-test job to ubuntu-24.04 as a temporary mitigation. The real fix is tracked in #6224. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: cheyang <cheyang@163.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6225 +/- ##
=======================================
Coverage 65.69% 65.69%
=======================================
Files 486 486
Lines 34423 34423
=======================================
Hits 22614 22614
Misses 10057 10057
Partials 1752 1752 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟢 Approval recommended
There are no active review issues for the Ubuntu 24.04 runner pin and its documented reversion condition.
No open issues 🎉
What changed in this PR
Temporarily pins the Kind end-to-end test runner to Ubuntu 24.04 to avoid the reported JindoRuntime FUSE mount failures on Ubuntu 26.04.
Changes:
- Changes
kind-e2e-testfromubuntu-latesttoubuntu-24.04. - Adds a
TODO(#6224)comment to restoreubuntu-latestonce the mount issue is resolved.
| File | Description |
|---|---|
.github/workflows/kind-e2e.yml |
Pins the runner image and documents the condition for reverting the pin. |
Review effort: Balanced
cheyang
left a comment
cheyang
left a comment
There was a problem hiding this comment.
Both reviewers independently confirmed the premise from the Actions API: ubuntu-latest flipped to the ubuntu-26.04 image between Oct 24 and Oct 27, and on 26.04 all 5 kind-e2e-test jobs fail with fusermount3: mount failed: Permission denied in the jindofs-fuse container (verbatim in failing run 40474966878's own diagnostic artifact), while the same workflow on the 24.04 image (run 40125302787, same triggering PR) and this PR's pinned run (40885813005) are green on all 5 Kubernetes versions across all 5 suites. Scope is correct: backward-compatibility-e2e stays on ubuntu-latest and is green on 26.04 (its script is Alluxio-only, no Jindo FUSE), and no other workflow runs FUSE workloads. CI at the review head is fully green (24/24 check runs plus CircleCI; tide pending only for the missing approved/lgtm labels — the review gate, not a failing check). No blockers or majors from either reviewer, and the debate had no factual disagreements; Reviewer C (Copilot) also raised no findings.
Two non-blocking notes, both already tracked by open issue #6224: (1) from merge day, CI exercises the JindoRuntime FUSE path only on 24.04, so the underlying product failure is invisible to CI until #6224 lands — though the lost 26.04 coverage was a two-day, all-red accidental lane created by the image flip, never a green signal; (2) when the CI side of #6224 is fixed, keep the issue (or a successor) open until the production-impact assessment is done, rather than auto-closing it with the CI fix. Nit: consider a target date on the un-pin checkbox in #6224 — optional, since the un-pin trigger is already explicit there.
Next round
[MUST] (none — merge is not gated on anything from this review)
[SHOULD] Keep #6224 open through its production-impact follow-up when the CI-side fix lands; optionally date the un-pin checkbox in #6224.
[Filed] (none)
Non-blocking observations (3) — not part of this round's ask; fix now, follow up, or file an issue as you see fit
- .github/workflows/kind-e2e.yml:25 [minor] — kind-e2e loses all ubuntu-26.04 coverage of the JindoRuntime FUSE path
F1 (consensus — raised by Claude, confirmed by Codex in cross-critique): pinning runs-on to ubuntu-24.04 removes ubuntu-26.04 from the only workflow that exercises the JindoRuntime FUSE mount, so CI can no longer detect the underlying failure: it stays broken in the product (jindo-fuse image + fusermount3 + hosts with AppArmor >= 5.0 shipping /etc/apparmor.d/fusermount3) and invisible to CI until #6224 is resolved. Codex's sharpening nuance, adopted: the lost 26.04 coverage was a ~2-day, all-red accidental lane created by the October image flip — never a designed green signal — and restoring it is verbatim #6224's second follow-up checkbox. This is the deliberate trade-off of a mitigation, tracked in open issue #6224, and does not gate merge. Some 26.04 signal survives in backward-compatibility-e2e.yml, which is intentionally left on ubuntu-latest (run 40885813116, job 12348908856 re-pulled live: image ubuntu-26.04, 5/5 SUCCESS). - [minor] — Keep the production-impact angle of #6224 visible
Q1 (consensus — raised by Codex, confirmed by Claude in cross-critique): the pin fixes CI, but issue #6224 also tracks a production question: any FUSE runtime that mounts through fusermount3 (e.g. JindoRuntime with -oauto_unmount) will likely fail the same way on user nodes running Ubuntu 26.04 or other distros with AppArmor >= 5.0. When the CI side of #6224 is eventually fixed, please keep the issue (or a successor) open until the production-impact assessment follow-up is resolved, rather than auto-closing it with the CI fix. Not a defect in this PR — a scoping guard for the tracker it now references; complementary to F1. - .github/workflows/kind-e2e.yml:25 [nit] — The pin has no expiry signal
F2 (consensus — raised by Claude, confirmed as nit by Codex with two mitigations): the TODO(#6224) marker has no date or review trigger; a dated follow-up checkbox in #6224 would keep the pin from outliving its purpose. Mitigations, adopted from the cross-critique: the un-pin trigger is already explicit in #6224's follow-ups ('Make kind-e2e pass on ubuntu-26.04 ..., then switch the workflow back to ubuntu-latest'), so the pin cannot be forgotten while the bug-labelled issue stays open; and a future ubuntu-24.04-label retirement would fail loudly via GitHub's deprecation brownouts, not silently. Anchoring corrected to line 25, where the TODO comment sits.
Verified review delta
- F1: open (initial-pr)
- Q1: open (initial-pr)
- F2: open (initial-pr)
Code: clear. CI: success; attribution: baseline. Required CI checks still apply.
…d-cloudnative#6219) Round 4: PR rebased onto 96a3765 (fluid-cloudnative#6225 runner pin moved to base; PR content otherwise byte-identical to round 3). All five harness contracts green on the new head; TS2 canary still-present as designed (fluid-cloudnative#6221 OPEN). No new findings, no harness changes needed. Co-Authored-By: Claude Code <noreply@anthropic.com>
…elta; all findings still fixed; runner-pin nit closed via base fluid-cloudnative#6225; advance .last-reviewed
…ase onto 96a3765); F1 still open/pre-existing; CI fully green - re-verify.sh grafted onto c805f16: L1 pass, L2 pins resolve; F1 contract still red - explicit per-ref runs on new base 96a3765 / previous head 9fe48f1 / head c805f16 - PR content byte-identical to round 1 (only kind-e2e.yml differs, via new base fluid-cloudnative#6225) - CI: all green on new head incl. 5/5 kind-e2e-test; Analyze (go) 11m54s - production code untouched; advance .last-reviewed to c805f16
Ⅰ. Describe what this PR does
Pins the
kind-e2e-testjob ofE2E Functionality Check(.github/workflows/kind-e2e.yml) fromubuntu-latesttoubuntu-24.04, and adds aTODO(#6224)comment to switch back later.ubuntu-latestnow resolves to theubuntu-26.04runner image, where all 5kind-e2e-testmatrix jobs fail (for example, run 40474966878 for #6219; the same PR passed onubuntu-24.04in run 40125302787). Ubuntu 26.04's AppArmor 5.0.2 package installs/etc/apparmor.d/fusermount3. That profile is attached to/usr/bin/fusermount3and only allows FUSE mounts under$HOME,/mnt,/run/user,/media,/tmpand/cvmfs, so the jindo-fuse container's mount of/jfs/jindofs-fuseis denied (fusermount3: mount failed: Permission denied). Ubuntu 24.04 (AppArmor 4.0.1) does not ship this profile.backward-compatibility-e2e.ymlis not changed because it passes onubuntu-26.04.Ⅱ. Does this pull request fix one issue?
NONE
This is a temporary mitigation for #6224, which stays open to track the real fix.
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
No test cases are added because this PR only changes the CI runner image.
Ⅳ. Describe how to verify it
Check this PR's own
E2E Functionality Checkrun. Everykind-e2e-testjob should showubuntu-24.04in the "Runner Image" section of its log and pass, for all 5 Kubernetes versions (v1.33.2, v1.30.13, v1.28.15, v1.24.17 and v1.22.17).Ⅴ. Special notes for reviews
After this merges, open PRs such as #6219 need a new push or a rebase to pick up the pin. Re-running an old workflow run reuses its original merge commit, so it still runs on
ubuntu-latest.