Skip to content

test(workspace): stabilize Windows provider update tests - #1556

Open
skevetter wants to merge 1 commit into
mainfrom
fix/provider-update-windows-flakes
Open

skevetter wants to merge 1 commit into
mainfrom
fix/provider-update-windows-flakes

Conversation

@skevetter

Copy link
Copy Markdown
Contributor

Problem

Two Windows unit-test flakes in pkg/workspace (job "Provider Update Tests (windows-latest)"), both observed on main/PR CI over the last two days:

  1. TestProviderUpdateAbsoluteReferenceUsesFinalStagedTarget/outside_reference_remains_untouched_while_managed_tree_updates: require.NoError(f.t, ctx.Err(), "update must terminate by concrete result, not deadline") failed with context deadline exceeded" after 5.76s. preservationFixture.updateboundsupdateProviderwith a hardcontext.WithTimeout(..., 5*time.Second); 5s is too tight for a cold Windows -race` runner under Defender, so the fixture's own guard — not the implementation — fires first.
  2. TestProviderUpdateSameNameHTTPFailurePreservesExisting: the whole test binary crashed with Exception 0xc0000005 (PC=0xc0005243c0, runtime: g 152 ...: unknown pc) right after the test's own assertions passed, during t.Cleanup closing the owned httptest.Server. The crashing PC is literally the g-struct pointer of a different, still-live goroutine (goroutine 151) — i.e. memory that should hold a return address on goroutine 152's stack instead holds another goroutine's object pointer. This matches known upstream Go runtime bugs: windows/amd64 stack corruption under -race during goroutine stack shrink/copy, reported in runtime: Windows crash with Go 1.26.0, 1.26.1 golang/go#77975 (Windows crash with Go 1.26.0/1.26.1, fixed in #78041/CL 753480) and runtime: intermittent ACCESS_VIOLATION in (*unwinder).next during shrinkstack on Windows amd64 (Go 1.26.0) golang/go#77955 (ACCESS_VIOLATION in shrinkstack/scanstack on Windows, also reproduced on go1.25.7). Our CI runner is already on go1.26.9 (post-#78041 backport) and still crashes, so this is a distinct, still-open class of the same Windows+race stack-corruption bug family, not a bug in our test or in net/http/httptest (the crash is deep inside Go runtime/stdlib internals, not in any devsy-owned code path).

Fix

  • Replace every hard-coded 3s/5s/10s context.WithTimeout used to bound updateProvider/provider.DownloadBinaries calls in the pkg/workspace provider-update preservation fixtures with a single shared updateTestHangGuard = 60 * time.Second constant — a hang-safety bound, not a target duration, generous enough to never race a slow CI host.
  • Drop -race from the Windows provider-update go test invocation in hack/provider_update_ci/discovery.go (narrowest viable mitigation for a confirmed upstream Go runtime bug, documented with issue references). pkg/workspace and pkg/provider keep full -race coverage on ubuntu-latest and macos-latest via the existing "Unit Tests" job, so this does not reduce race coverage, only removes a Windows-only runtime crash that isn't a real data race in our code.

Validation

  • go test -race -count=20 -run 'TestProviderUpdate' ./pkg/workspace/ — 20x clean on macOS (does not reproduce the Windows-only runtime crash; included per the review checklist).
  • GOOS=windows go vet ./pkg/workspace/ ./hack/provider_update_ci/... — clean.
  • go test -race -count=1 ./hack/provider_update_ci/... — clean (covers the updated arg-shape test).
  • mise run cli:lint:ci — 0 issues.

Replace the hard-coded 5s/10s/3s update deadlines shared by the
provider-update preservation fixtures with a single 60s hang guard,
since tight wall-clock deadlines flake under slow Windows -race CI.

Drop -race from the Windows provider-update test run: it intermittently
crashes net/http server teardown during goroutine stack shrink/copy on
windows/amd64 (golang/go#77975, #77955, #78041), still reproducing on
go1.26.9. pkg/workspace and pkg/provider keep -race coverage via the
Unit Tests job on ubuntu-latest and macos-latest.
@netlify

netlify Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for devsydev canceled.

Name Link
🔨 Latest commit e3416c3
🔍 Latest deploy log https://app.netlify.com/projects/devsydev/deploys/6ad50f4cc33cbf726ce2e1c0

@coderabbitai

coderabbitai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 29 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fabeb242-759f-4270-9cea-168d70ce64d5
📥 Commits

Reviewing files that changed from the base of the PR and between 7357c5a and e3416c3.

📒 Files selected for processing: 5
  • hack/provider_update_ci/discovery.go
  • hack/provider_update_ci/main_test.go
  • pkg/workspace/provider_update_extended_test.go
  • pkg/workspace/provider_update_prepared_test.go
  • pkg/workspace/provider_update_preservation_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@netlify

netlify Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for images-devsy-sh canceled.

Name Link
🔨 Latest commit e3416c3
🔍 Latest deploy log https://app.netlify.com/projects/images-devsy-sh/deploys/6ad50f4c6185276c29fcb6a8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant