Repository navigation
Description
Release performance regressed at f7c1e76 on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/41324417425
Release performance: 0.186.2 (candidate) vs 0.186.2 (baseline)
Same runner: AMD EPYC 9V45 96-Core Processor, 4 CPUs; load average 3.95/3.58/2.08 before and 1.50/2.07/2.04 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.186.2), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
| Command | Wall (s) | Wall ratio | CPU (s) | CPU ratio | Peak RSS (MiB) | RSS ratio | Result |
|---|---|---|---|---|---|---|---|
compile (no cache) |
3.80 → 5.30 | 1.395x | 8.31 → 11.88 | 1.430x | 923 → 1010 | 1.094x | ❌ regressed |
compile (warm cache) |
0.18 → 0.19 | 1.056x | 0.12 → 0.11 | 0.917x | 71 → 71 | 1.005x | ✅ ok |
plan --json |
5.49 → 5.55 | 1.011x | 7.27 → 7.33 | 1.008x | 1004 → 1023 | 1.019x | ✅ ok |
lineage hub downstream |
0.19 → 0.17 | 0.895x | 0.26 → 0.25 | 0.962x | 157 → 162 | 1.032x | ✅ ok |
lineage rollup upstream |
0.19 → 0.17 | 0.895x | 0.26 → 0.25 | 0.962x | 156 → 171 | 1.091x | ✅ ok |
lineage column trace |
1.87 → 1.88 | 1.005x | 2.57 → 2.62 | 1.019x | 920 → 916 | 0.997x | ✅ ok |
dag --json |
2.52 → 2.51 | 0.996x | 3.21 → 3.17 | 0.988x | 972 → 976 | 1.003x | ✅ ok |
scope --json |
0.19 → 0.19 | 1.000x | 0.18 → 0.17 | 0.944x | 118 → 124 | 1.048x | ✅ ok |
compile (one-model edit) |
0.81 → 0.83 | 1.025x | 0.69 → 0.70 | 1.014x | 359 → 358 | 0.997x | ✅ ok |
dense compile (warm cache) |
0.31 → 0.30 | 0.968x | 0.18 → 0.19 | 1.056x | 94 → 94 | 1.001x | ✅ ok |
dense compile (one-model edit) |
1.25 → 1.20 | 0.960x | 1.06 → 1.05 | 0.991x | 434 → 440 | 1.014x | ✅ ok |
dense compile (no cache) |
6.45 → 6.49 | 1.006x | 15.87 → 15.77 | 0.994x | 1368 → 1418 | 1.036x | ✅ ok |
build (empty warehouse) |
30.14 → 30.14 | 1.000x | 30.25 → 30.24 | 1.000x | 1732 → 1746 | 1.008x | ✅ ok |
Result: failed. compile (no cache): wall 1.395x exceeds 1.15x; compile (no cache): CPU 1.430x exceeds 1.15x.
Activity
github-actions commentedon Nov 1, 2026
Release performance regressed at 45f102e on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/41327968958
Release performance: 0.186.2 (candidate) vs 0.186.2 (baseline)
Same runner: AMD EPYC 7763 64-Core Processor, 4 CPUs; load average 3.69/3.79/2.67 before and 1.24/1.62/2.14 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.186.2), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): wall 1.366x exceeds 1.15x; compile (no cache): CPU 1.433x exceeds 1.15x.
github-actions commentedon Nov 1, 2026
Release performance regressed at 69bf510 on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/41328954763
Release performance: 0.186.2 (candidate) vs 0.186.2 (baseline)
Same runner: AMD EPYC 9V74 80-Core Processor, 4 CPUs; load average 4.10/3.86/2.65 before and 1.13/1.52/2.05 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.186.2), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): wall 1.341x exceeds 1.15x; compile (no cache): CPU 1.415x exceeds 1.15x.
github-actions commentedon Nov 1, 2026
Release performance regressed at b66b83f on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/41338371912
Release performance: 0.186.2 (candidate) vs 0.186.2 (baseline)
Same runner: INTEL(R) XEON(R) PLATINUM 8573C, 4 CPUs; load average 4.71/4.14/2.84 before and 1.34/1.57/2.07 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.186.2), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): wall 1.292x exceeds 1.15x; compile (no cache): CPU 1.375x exceeds 1.15x.
github-actions commentedon Nov 2, 2026
Release performance regressed at 3b77897 on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/41375594941
Release performance: 0.186.2 (candidate) vs 0.186.2 (baseline)
Same runner: AMD EPYC 9V74 80-Core Processor, 4 CPUs; load average 4.33/4.01/2.69 before and 1.28/1.57/2.09 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.186.2), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): wall 1.353x exceeds 1.15x; compile (no cache): CPU 1.406x exceeds 1.15x.
github-actions commentedon Nov 2, 2026
Release performance regressed at 50382ae on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/41378766538
Release performance: 0.187.0 (candidate) vs 0.186.2 (baseline)
Same runner: AMD EPYC 9V45 96-Core Processor, 4 CPUs; load average 3.94/3.49/2.03 before and 1.23/1.74/1.86 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.186.2), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): wall 1.370x exceeds 1.15x; compile (no cache): CPU 1.484x exceeds 1.15x.
github-actions commentedon Nov 10, 2026
Release performance regressed at c642221 on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/42825566391
Release performance: 0.207.0 (candidate) vs 0.207.0 (baseline)
Same runner: AMD EPYC 9V45 96-Core Processor, 4 CPUs; load average 1.13/2.63/3.27 before and 1.26/1.80/2.61 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.207.0), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): CPU 1.224x exceeds 1.15x.
kvlonge commentedon Nov 10, 2026
Run https://github.com/sqlstack-io/sqlbuild/actions/runs/42825566391 (main at c642221, after #1397) flagged
compile (no cache)CPU at 1.224x against 0.207.0 (12.79 s -> 15.65 s; wall 1.093x, peak RSS 0.988x).dense compile (no cache)CPU is 1.124x, within its limit, and every other command is at or below 1.0x.This is the known cost of moving model analysis onto SQLQuack's native parsers in #1397. It was accepted by the maintainers as a temporary allowance when the cutover was approved, and the analysis-sharing change in the same PR already reduced it (same-machine A/B on the 3,000-model compiler-only case: cold CPU 17.46 s -> 15.62 s). A follow-up to bring cold no-cache CPU back within 1.10x of the previous release is tracked; the comparison limits themselves are unchanged.
github-actions commentedon Nov 10, 2026
Release performance regressed at 5b4cbbf on main.
Comparison run: https://github.com/sqlstack-io/sqlbuild/actions/runs/42874254488
Release performance: 0.207.0 (candidate) vs 0.207.0 (baseline)
Same runner: AMD EPYC 9V74 80-Core Processor, 4 CPUs; load average 0.98/2.28/3.14 before and 1.14/1.50/2.29 after. 5 interleaved runs per version; wall and CPU are medians, peak RSS is the worst run.
Benchmark projects are generated per side: the baseline runs projects from its own generator (v0.207.0), the candidate from the candidate's.
Limits: wall +25% (ignored under 0.5 s), CPU +25% (ignored under 0.5 s), peak RSS +25% (ignored under 32 MiB). Cold, warm and one-model edit compiles use +15% for wall and CPU.
compile (no cache)compile (warm cache)plan --jsonlineage hub downstreamlineage rollup upstreamlineage column tracedag --jsonscope --jsoncompile (one-model edit)dense compile (warm cache)dense compile (one-model edit)dense compile (no cache)build (empty warehouse)Result: failed. compile (no cache): CPU 1.165x exceeds 1.15x.
kvlonge commentedon Nov 11, 2026
Closing: the last flagged regression (compile without cache, CPU 1.22x at 0.207.0) is the known cost of moving model analysis onto SQLQuack's native parsers, explained above, and later release comparisons up to 0.211.0 passed. The release performance workflow opens a new issue if a regression comes back.