Skip to content

ci: skip the example tests when testing built wheels - #147

Merged
GuillaumeLagrange merged 1 commit into
masterfrom
cod-4044-release-pytest-codspeed-patch-with-pytest-92-support
Oct 30, 2026
Merged

GuillaumeLagrange merged 1 commit into
masterfrom
cod-4044-release-pytest-codspeed-patch-with-pytest-92-support

Conversation

@adriencaccia

Copy link
Copy Markdown
Member

The wheel tests collect tests/examples directly, where pytest-benchmark's own fixture reads node._nodeid, removed in pytest 9.2 and still read by pytest-benchmark 5.3.0. This failed the v5.0.4 release run; skip them in wheel tests, as the local pytest config already does.

Refs COD-4044, which stays open until 5.0.4 is on PyPI.

tests/examples holds files that other tests copy through pytester. Collected
directly, they run pytest-benchmark's own fixture, which reads `node._nodeid`,
removed in pytest 9.2 and still read by pytest-benchmark 5.3.0. The wheel tests
install the latest pytest, so the v5.0.4 release build failed. The local pytest
config already ignores tests/examples.

Refs COD-4044
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge.

Summary

Adds --ignore={project}/tests/examples to the wheel test command.

  • Wheel checks skip example tests when testing built packages.

Reviews (1) · Last reviewed commit: "ci: skip the example tests when testing ..." · Reviewed by Greptile

@codspeed

codspeed Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 9.74%

3 warnings, 1 note

⚠️ 6 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 3 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 16 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

ℹ️ 37 benchmarks have no flamegraph

The flamegraph wasn't generated: the symbols of the profiling trace could not be resolved, because no information about the language frames was collected.

See affected benchmarks

⚡ 6 improved benchmarks
❌ 10 (👁 10) regressed benchmarks
✅ 261 untouched benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Memory test_multiprocessing_map[100] 4.1 MB 3.6 MB +14.53%
⚡ Walltime test_tcp_connection[1.1.1.1-53] 867.2 µs 771.3 µs +12.44%
⚡ Memory test_multiprocessing_map[10] 3.8 MB 3.5 MB +8.6%
⚡ Walltime test_pipe_communication[10000] 12 ms 11.3 ms +6.39%
⚡ Walltime test_pipe_communication[1000] 6.2 ms 5.9 ms +5.89%
⚡ Memory test_multiprocessing_map[10000] 3.2 MB 3 MB +5.88%
👁 Walltime test_sudoku[initial_grid0] 7.9 µs 9.2 µs -14.63%
👁 Walltime test_generate_parenthesis[3] 11.1 µs 13.1 µs -15.12%
👁 Walltime test_multiprocessing_map[10000] 52.3 ms 56.3 ms -6.98%
👁 Walltime test_is_complete[board0] 1.8 µs 1.9 µs -6.87%
👁 Walltime test_multiprocessing_map[1000] 49.1 ms 56.3 ms -12.81%
👁 Walltime test_solve_power_sum[13-2] 7 µs 7.5 µs -6.38%
👁 Walltime test_multiprocessing_map[100000] 92.7 ms 289.1 ms -67.94%
👁 Walltime test_hostname_resolution[amazon.com] 326.6 µs 344.3 µs -5.14%
👁 Walltime test_sum_of_squares[sum_of_squares_for_loop_power] 189.6 µs 220.7 µs -14.07%
👁 Walltime test_sum_of_squares[sum_of_squares_sum_comprehension_power] 240.6 µs 276.7 µs -13.04%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cod-4044-release-pytest-codspeed-patch-with-pytest-92-support (8975067) with master (24605df)

Open in CodSpeed

@GuillaumeLagrange GuillaumeLagrange left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 😬

@GuillaumeLagrange
GuillaumeLagrange merged commit 8f93e8f into master Oct 30, 2026
79 of 81 checks passed
@GuillaumeLagrange
GuillaumeLagrange deleted the cod-4044-release-pytest-codspeed-patch-with-pytest-92-support branch October 30, 2026 17:13
adriencaccia added a commit that referenced this pull request Oct 31, 2026
tests/examples holds files that other tests copy through pytester. Collected
directly, they run pytest-benchmark's own fixture, which reads `node._nodeid`,
removed in pytest 9.2 and still read by pytest-benchmark 5.3.0. The wheel tests
install the latest pytest, so the v5.0.4 release build failed. The local pytest
config already ignores tests/examples.

Refs COD-4044

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants