[davecheney]

The Windows and macOS workflows put the custom -only-current-os flag in GOTESTFLAGS before the package list. Go stops parsing packages at an unknown flag, so these jobs ran only the root package tests. The builder, cgo, compileopts, compiler, interp, loader, and transform packages were not tested.

Use the standard -short flag in these two workflows. Go recognizes this flag and processes the complete package list. Local make test does not change because GOTESTFLAGS is empty by default.

Short-mode coverage

In TestBuild, -short now skips only the cross-target cases (EmulatedCortexM3, EmulatedRISCV, AVR, WebAssembly, WASIp1, and WASIp2). Linux CI runs these cases without -short, so other hosts do not need to run them again. The host-specific cases, such as Windows386, continue to run in short mode. TestWASIPanicTraceback and the cross-target cases in TestTest also skip in short mode.

The -only-current-os flag is removed because nothing uses it now.

Workflow dependencies

The Windows test job no longer installs wasmtime because short mode does not use it. The macOS test job keeps qemu because TestBuild/opt=0-gc-cortex-m-qemu runs in short mode.

Prerequisites

All three prerequisites are merged:

• loader: use absolute paths in recorded path tests #5913 uses absolute paths in the recorded-path tests.
• loader: preserve the working directory in go list #5914 preserves the working directory passed to go list.
• builder: accept source paths in size tests without trimpath #5915 corrects the size-output path assertions.

[davecheney]

I spotted this one as part of #5872. The net result was several packages were not tested on macOS and windows, and there were a few bugs, mainly around file path handling that crept in. I've restored the test runner to use -short as the synonym for "skip long tests", as this is known to go test and is safe to use when testing multiple packages. I've also fixed the failing tests and hardened them a little by using the newer testing.TB helpers.

[deadprogram]

Thanks for tracking this down @davecheney! Good catch on the flag ending the package list. The following is edited from an automated review:

1. -short skips more than -only-current-os did. TestTest only adds the EmulatedCortexM3, EmulatedRISCV, WASM and WASI targets when testing.Short() is false, and TestWASIPanicTraceback skips in short mode. The macOS job installs qemu and the Windows job installs wasmtime, so those subtests did run there before this change and now they do not. Maybe that is fine given the Linux job still runs them, but should we note it in the commit message? Or keep them by using an environment variable instead of a flag?

2. The commit message only talks about the CI change. Perhaps add a line or two on the test fixes that came along with it, such as the loader.List PWD fix for symlinked dirs like /var on macOS, the absolute path case in TestSizeFull, and the new wasm-tools check in emuCheck?

3. TestSizeFull passed on Linux only by chance, since /home/runner/work/tinygo/tinygo/lib/nrfx/mdk happens to match ^[a-z/]+$. The new check covers that too, so it is more correct now. Should the comment mention that it also explains why Linux was passing?

4. The emuCheck wasm-tools check only looks at spec.WITPackage and spec.WITWorld, while the build also honors Options.WITPackage and Options.WITWorld. No test sets those today, so it is fine as is. Just mentioning it in case that changes.

What do you think of all this? Thanks again for working on it.

[davecheney]

@deadprogram you're right, this PR is a hot mess. Let me pull it apart some, merge the various test fixes, then rebase this PR to make it smaller.

[davecheney]

Thanks for the review. I split the PR up and changed the approach for point 1.

1. I kept -short because go test recognizes it in all packages. In TestBuild, -short now skips only the cross-target cases, such as EmulatedCortexM3, RISC-V, AVR, and WASM/WASI. Linux CI runs them, so it is not necessary to run them on every host. The host-specific cases, such as Windows386, continue to run on Windows. Before this change, the short check returned before the host-specific block. The Windows test job no longer installs wasmtime because short mode does not use it. macOS keeps qemu because TestBuild/opt=0-gc-cortex-m-qemu runs in short mode. Local make test continues to run all tests.
2. The test fixes moved to loader: use absolute paths in recorded path tests #5913, loader: preserve the working directory in go list #5914, and builder: accept source paths in size tests without trimpath #5915. Each one has its own commit message.
3. builder: accept source paths in size tests without trimpath #5915 adds the absolute path case to TestSizeFull. Its description explains why Linux passed by chance.
4. I removed the emuCheck wasm-tools change from this PR.

On Windows CI, the PR now tests all eight packages. On dev, it tests only the root package.

[deadprogram]

Thanks for splitting this up @davecheney, it is much easier to follow now. I compared the CI logs against dev and the macOS and Windows test jobs now run all 8 packages. One minor note edited from an automated review:

test-compat in .github/workflows/compat.yml also runs go test -short on Linux. Before this change -short returned before the runtime.GOOS == "linux" block in TestBuild, so that job skipped the other Linux arches and the MIPS test. Now that block runs there too. The emulated arches should skip since qemu-user is not installed, but X86Linux probably runs natively, and the root package time in that job went from about 206s to 339s. Is that intended? If not, perhaps the Linux block could also check testing.Short().

[davecheney]

Thanks for splitting this up @davecheney, it is much easier to follow now. I compared the CI logs against dev and the macOS and Windows test jobs now run all 8 packages. One minor note edited from an automated review:

test-compat in .github/workflows/compat.yml also runs go test -short on Linux. Before this change -short returned before the runtime.GOOS == "linux" block in TestBuild, so that job skipped the other Linux arches and the MIPS test. Now that block runs there too. The emulated arches should skip since qemu-user is not installed, but X86Linux probably runs natively, and the root package time in that job went from about 206s to 339s. Is that intended? If not, perhaps the Linux block could also check testing.Short().

Thanks for the second review. Let me take another swing at this tomorrow