[adriencaccia]

tests/examples/test_addition_fixture.py was only input for pytester-based tests, but its test_ name made pytest collect it whenever the path-relative --ignore options didn't apply, as in the wheel tests that failed the v5.0.4 release build. Its source now lives in an ADDITION_BENCHMARK constant that each test writes into its pytester sandbox with makepyfile, so both tests/examples ignores, including the one from #147, go away.

Closes COD-4080

[greptile-apps]

Confidence Score: 5/5

[Low impact] The PR appears safe to merge; no actionable issues were found.

Summary

Moves the addition benchmark into an ADDITION_BENCHMARK string that tests write inside their pytester sandboxes.

• Plugin tests create their sample benchmark inside their sandboxes.

_{Reviews (1) · Last reviewed commit: "test: write the example benchmark inside..." · Reviewed by Greptile}

[codspeed]

Merging this PR will degrade performance by 6.81%

3 warnings, 1 note

⚠️ 5 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 3 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 18 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

ℹ️ 12 benchmarks have no flamegraph

The flamegraph wasn't generated: the symbols of the profiling trace could not be resolved, because no information about the language frames was collected.

See affected benchmarks

⚡ 8 improved benchmarks
❌ 10 regressed benchmarks
✅ 259 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Walltime	test_multiprocessing_map[100000]	246.4 ms	431.5 ms	-42.9%
❌	Memory	test_multiprocessing_map[1000]	2.9 MB	4.2 MB	-31.03%
❌	Memory	test_multiprocessing_map[10000]	3.2 MB	4.1 MB	-21.25%
❌	Memory	test_multiprocessing_map[100]	3.1 MB	3.7 MB	-17.75%
❌	Memory	test_multiprocessing_map[10]	3.5 MB	4.2 MB	-17.49%
❌	Walltime	test_multiprocessing_map[100]	51.3 ms	56.6 ms	-9.4%
❌	Walltime	test_noop_lambda_decorated	1.2 µs	1.3 µs	-7.73%
❌	Walltime	test_recursive_fibo_20	5.2 ms	5.5 ms	-6.24%
❌	Walltime	test_solve_crossword[puzzle0-words0]	2.1 µs	2.3 µs	-6.21%
❌	Walltime	test_array_alloc[100]	1.1 µs	1.2 µs	-5.5%
⚡	Walltime	test_tcp_connection[1.1.1.1-53]	889.1 µs	774.3 µs	+14.84%
⚡	Walltime	test_sum_squares_slow	339.7 µs	307.8 µs	+10.35%
⚡	Walltime	test_make_peak	6.8 µs	6.2 µs	+9.34%
⚡	Walltime	test_sudoku[initial_grid0]	8.3 µs	7.7 µs	+8.39%
⚡	Walltime	test_make_lowpass	5.8 µs	5.4 µs	+7.77%
⚡	Walltime	test_make_highpass	5.7 µs	5.4 µs	+6.43%
⚡	Walltime	test_sum_of_squares[sum_of_squares_sum_comprehension_power]	241.1 µs	228.1 µs	+5.7%
⚡	Walltime	test_mmap_operation[4096]	38.2 µs	36.1 µs	+5.56%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-4080-pytest-codspeed-example-benchmark-is-collected-as-a-real (c13dfad) with master (8f93e8f)}