Skip to content

docs(perf): measure the 0.12.0 candidate against 0.11.0 - #1740

Merged
HuiJun merged 12 commits into
developfrom
docs/perf-record-0.12.0
Oct 30, 2026
Merged

HuiJun merged 12 commits into
developfrom
docs/perf-record-0.12.0

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What and why

The release-gate performance record for 0.12.0: docs/project/performance-release-0.12.0-vs-0.11.0.md, following the method of the 0.11.0 record (every benchmark package, benchstat over six interleaved rounds, whole-binary timings on generated and real models, process floor, the tests/stressmodel constellation), wired into mkdocs.yml (not-in-nav, after the 0.11.0 record), docs/project/README.md, the "latest is" link of docs/internals/performance.md, and a changes/unreleased/perf-0.12.0-record.added.md fragment.

The record measures three columns, because the toolchain moved from go1.25.11 to go1.27.3 between the releases and dominates a naive comparison: 0.11.0 as released, the 0.11.0 source rebuilt with go1.27.3, and the candidate — first develop at 1c6bb1972 (which holds #1699), the pre-fix measurement that found the regressions below, then the release candidate itself, cc1d477ba (develop ab88ea950 after the seven fixes + #1730), measured with the full method in "The release candidate" with the pre-fix column kept so the recovery is visible row by row. Every statement says which columns it rests on; source regressions are column 2 against 3 (and 2 against the release candidate).

What it finds, in short: the candidate is faster on nearly every row a user notices (loads −8…−13 %, WriteAPIJSON −10…−37 %, action chains −35…−59 %, LookupQualified 150 → 43 ns), and allocates more on a handful of paths, each bisected to its merge — #1462 (LoadModel/0), #1478 (every load row, +5 % bytes), #1514/#1461 (AssignmentLoopStep), #1619 (DerivedReadWriteRead), #1508/#1512 (the binding walks), #1626 (FeaturesOf, Instantiate, BatchSatisfy) — and the +31 MiB resident set on the largest generated model is #1441 holding a submission's parse trees through its analysis. The fix for each is a separate perf/<slug> PR; the record's Fixes section is filled in as they open, and its Re-runs section with the quiet interleaved re-run of the rows the loaded machine left in doubt.

The fixes (#1741–#1747) are listed as in the candidate with their merge SHAs; the export rows were re-probed on 5e0322e0c when #1760 landed on internal/translate/export; "After the measured heads" accounts for every merge up to develop b4d0a35c4 — by git diff --stat … -- '*.go' go.mod go.sum for the client/pom/workflow merges and by #1771's own six-round benchstat for the one runtime refactor. The record also says what it does not decide: the afternoon's quiet re-run ran under 3–8 % hypervisor steal, so the first pass's time verdicts stand, and one open finding (a -run-query run leaves the next -validate of the same binary with a cold library record cache) is named and not attributed.

How it was verified

python3 scripts/check-doc-links.py reports one broken link, docs/project/third-party-notices.md → ../assets/landing/libavoid-js.LICENSE.txt, which is pre-existing on develop (same result on a clean checkout of 54e590985) and untouched here; the new record's links all resolve. The raw per-round benchmark outputs (six rounds × three configurations, the release-candidate rounds and re-probes, whole-binary runs, bisect logs) are kept outside the repository and attached to the gate report, as the 0.11.0 record's were.

Checklist

  • make test and make lint pass locally — documentation only; no Go source changes
  • Tests added or updated for the change — not applicable
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved — no gate count moved
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/15408827f8104c7fbbdbd1b2c852ed2c
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/15408827f8104c7fbbdbd1b2c852ed2c?variant=devin
Requested by: @HuiJun

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 30, 2026 04:50

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 3 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

devin-ai-integration Bot and others added 4 commits October 30, 2026 04:55
…ation count does not bend

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…leaving them blank

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration Bot and others added 5 commits October 30, 2026 08:30
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…he quiet host

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ort rows

The record gains a second pass: the release candidate cc1d477 (develop
ab88ea9 + #1730) measured with the full method against the two 0.11.0
columns, with 1c6bb19 kept as the pre-fix measurement so the seven
fixes' recovery is visible row by row; the export rows re-probed on
5e0322e after #1760 landed on them; every merge up to develop
b4d0a35 accounted for by diff or by its own benchstat; the quiet
re-run's steal-noise stated rather than read; the record-cache cold-load
effect found by the whole-binary sequences named.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@HuiJun
HuiJun merged commit cf16528 into develop Oct 30, 2026
30 checks passed
@HuiJun
HuiJun deleted the docs/perf-record-0.12.0 branch October 30, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant