Skip to content

fix(tools): the savings floor measures without the linker's alignment fill - #302

Merged
angeloINTJ merged 1 commit into
mainfrom
fix/savings-without-fill
Oct 30, 2026
Merged

fix(tools): the savings floor measures without the linker's alignment fill#302
angeloINTJ merged 1 commit into
mainfrom
fix/savings-without-fill

Conversation

@angeloINTJ

@angeloINTJ angeloINTJ commented Oct 30, 2026

Copy link
Copy Markdown
Owner

Issue: #296. The nightly SIMUT savings watermark went red on 2026-10-28 (one feature) and 2026-10-29 (nine of 16). No code had left a switch; the linker's alignment fill had moved.

What was measured

The change

The metric. The floor now compares used minus every *fill* in .boot2, .text, .rodata and .data of the linker map. flash_fill( ) generalizes the bank_pad( ) of #285.

The refusals. A map is refused, never read as 0, when its sections do not add up to used, when a section is missing, or when it has a fill outside its section or a line it cannot parse.

The tolerance. A flash drop of up to FLASH_TOL = 8 B passes:

The configurator's costs. --matrix keeps the whole used, because a cost is what the image takes in its slot. tools/feature_costs.json is untouched.

The 16 floors are re-recorded with --update on e776822: 13 fall by 1 to 102 B and 3 rise by 1 to 5 B. The unit changed from aligned bytes to content bytes. The RAM floors and the bases are unchanged.

Files. tools/measure_savings.py, tools/test_measure_savings.py, tools/feature_savings.json, docs/analysis/ECONOMIA_DE_RECURSOS.md (Living: a section on the fill, with the table per merge), and the text of AGENTS.md, tools/README.md and one step of .github/workflows/build.yml. In that step only the comment and the name change; no job name changes. Nothing changes under src/, test/, data/ or keys/, so no image changes.

Proof

All of it ran on this host, with a pio shim (the local PlatformIO install is broken by python 3.14). The nightly job was not run on this branch.

Tests first. tools/test_measure_savings.py goes from 21 to 45 cases: the old tool fails 32 of the 45, and the new one passes 45/45. Each of 15 mutations of the tool breaks exactly the expected cases.

Measurements:

  • --update, then --check: "16 recursos, nenhum abaixo do piso".
  • A leak planted on purpose (two PromMetrics symbols kept by main.cpp): exactly one finding, web_metrics 4,745 B < 4,955 B, −210 B.
  • A replay of the nine runs with the new metric (six commits of main, the two version swaps and the leak): 0 false findings in 81 measurements, and the leak is the only finding. The old gate gave 9, 5 and 9 findings on runs with no leak.

Gates. check_angulo.py passes, and the local ci_gates_local --quick --raso passes 40/40.

Review. A revisor gave COM RESSALVAS, with nothing blocking, and all six findings are fixed:

  • The revisor's own reader, written from scratch, matches flash_fill on 145 real maps.
  • The 16 floors recomputed from the stored ELFs and maps equal the JSON.
  • With cmd_check called directly, an 8 B drop passes, a 9 B drop fails, and RAM goes from 64 B passing to 65 B failing.

After the merge

Dispatch savings.yml on main, check it is green, and close #296.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uy7K2mzEmmjBnpxpdYNDFu

Ângelo Moisés Alves @claude
fix(tools): the savings floor measures without the linker's alignment…
735a867 
… fill

The nightly savings watermark went red on 2026-10-28 (concurrency_asserts) and on 2026-10-29 (nine
of 16, issue #296) with no code leaving a switch. Measured on this host (shim pio) the three CI logs
reproduce exactly: 48 of 48 (feature, commit) pairs equal on flash, RAM and the BTstack bank fill. Over
six commits of main (6adf102..229d5d5) the nine savings moved 8-16 B per commit while, in 45
(feature, step) cells, the bytes a switch removes, adds or merges (strings) did not move by one
byte: only the linker's *fill* did, the pad in front of 8-aligned sections (.fini.__stub in .text,
the time_t[] tables behind the string pools in .rodata), which follows the size of everything
before it and differs between the base and the variant. The one real move is 4 B of
AppManager::core0Yield( ) for tft_graph at #294, undone at #295. The same 229d5d5 with one
character of the version string changed ("2.12.1", no code) moves six of the nine by 8 B; "2.13.0"
moves none.

The floor now compares `used` minus every *fill* in .boot2, .text, .rodata and .data of the linker
map (the .boot2 fill is 0 B in all 145 maps): flash_fill( ) generalizes bank_pad( ), whose BTstack-bank
fill (up to 4,095 B, #285) was the biggest term. It refuses, never reads 0, a map whose boot2+text+rodata+data do not add up to `used`, a map
without one of them, and a *fill* outside any section, outside its own, or on a line it cannot
parse. A flash drop of up to FLASH_TOL = 8 B passes: with the fills out what is left is real code and
moves 0-12 B (core0Yield; cli_full +12, sound_buzzer -4/+8, web_history +1 over three commits): that
residue is what 8 B cover. The smallest drop the floor ever recorded was 24 B (#234, a code change,
measured on 2026-10-22 with the fill still in); alone, with the new number, #234 takes 38 B from
sound_buzzer (4,999 -> 4,961 B). --matrix keeps the whole `used` (the cost of a switch is what the
image takes in the slot) and tools/feature_costs.json is untouched.

Proof, all on this host with the shim pio (the nightly job was not run on this branch):
- tools/test_measure_savings.py 21 -> 45 cases. Against the old tool 32 of 45 fail (the 13 that pass
  pin what must not change); against the new one 45/45. 15 small mutations of the tool (no fill
  subtraction, tolerance 0 or 9, no self-check, missing section accepted, every entry or every
  section counted as fill, scan from 0, --update writing the floor minus the tolerance, the --matrix
  build returning used - 1, ...) each break only the cases written for them.
- flash_fill( ) reads the 145 real maps of the investigation with no refusal, to the same bytes as an
  independent reader; PlatformIO's `used` is boot2+text+rodata+data in 208 of 208 images.
- 16 floors re-recorded with --update on e776822: 13 fall by 1-102 B and 3 rise by 1-5 B, because
  the unit changed (bytes of content, no longer multiples of 8); the RAM floors are unchanged.
  --check on the branch: "16 recursos, nenhum abaixo do piso".
- A leak planted on purpose (main.cpp keeps PromMetrics::lineF and escapeLabel, two helpers of the
  web_metrics switch): --check fails with exactly one finding, web_metrics 4745 B < 4955 B.
- The gate replayed offline over the six commits, the two version-string builds and the leak:
  0 false findings in 81 measurements, and the leak is the one finding.

Also touched, because each describes the metric this changes: .github/workflows/build.yml (the
comment and the name of the step that runs the test, which said the fill in front of the BTstack
bank; the job is unchanged), AGENTS.md (the paragraph on the savings floor), tools/README.md (the
two rows) and docs/analysis/ECONOMIA_DE_RECURSOS.md (Living: a section on the alignment fill, the
floors table, the intro and "Como rodar").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uy7K2mzEmmjBnpxpdYNDFu
@github-actions github-actions Bot added documentation Improvements or additions to documentation ci Continuous integration and automation tools Build tools, scripts, and developer tooling labels Oct 30, 2026
@angeloINTJ
angeloINTJ merged commit cf97b95 into main Oct 30, 2026
14 checks passed
@angeloINTJ angeloINTJ mentioned this pull request Oct 30, 2026
Closed
angeloINTJ pushed a commit that referenced this pull request Oct 30, 2026
Ângelo Moisés Alves @claude
Merge main cf97b95 (v2.12.0, #302) into the branch: sync 8 (S10c)
88ddd89 
First parent: 22d83de. What lands from main (c902894..cf97b95): #299 (96a5af0), the Files page says a refused upload was refused: doUpload( ) in WebUI.h, fil_up_refused in both packs' @webdict, tools/test_webui_upload_result.py;
#300 (229d5d5), chore v2.12.0: the version in src/SystemDefs_Limits.h, both CHANGELOGs, the READMEs, the manual and the site; #301 (e776822), the release gate's docs; #302 (cf97b95), the savings floor without the linker's alignment fill:
tools/measure_savings.py (flash_fill( ), FLASH_TOL = 8), tools/test_measure_savings.py (45 cases), tools/feature_savings.json re-recorded (16 floors), AGENTS.md, tools/README.md, docs/analysis/ECONOMIA_DE_RECURSOS.md, a step's comment and name in build.yml.

Conflicts (7), as resolved
- data/lang/language_pt-BR.lng, data/lang/language_es-ES.lng (one line each, the @webdict): the union by key against the base (c902894's): 557 keys = the branch's 556 (538 + the 18 thm_) + main's fil_up_refused, put after fil_up_err where main put it;
  every other line the branch's (main changed that line only). check_lang_packs.py OK, pt-BR 52,706 B and es-ES 54,727 B of the 65,536 B file ceiling.
- docs/analysis/PLANO_STABLE.md 3.1: the branch's table (ten rows, the two Angulo images included); every row is what tools_s10c/budget_set.py writes from this merge's builds.
- tools/flash_budget.json (five hunks): the branch's file; measured_on = the branch's string + the 633 characters main appended to the base's (measured_at_commit: main appended none) + this merge's sentence; every number from the builds
  (budget_set.py for ten images; pico_w_test_https is unchanged, 1,029,188 B used = its budget).
- tools/features.toml (one comment): the pico_w_test headroom with both figures, main's 3.427 B on cf97b95 (3.539 B before the refused-upload message) and this tree's 3.267 B, which PLANO_STABLE's row now says.
- tools/README.md: the branch's rows; measure_savings.py = the branch's row with main's "Desde 30/10" sentence in the place of its "Desde 22/10" one, and the date; test_measure_savings.py = main's row (the branch left it as the base).
- tools/test_measure_savings.py: main's rewrite with the branch's cases kept inside it: the SIMUT_SAVINGS_PIO block after main's tagged [g3] line in pio_run_cases( ), flips_cases( ) beside table_cases( ), and main( ) runs it last.
  21 cases at the base, 42 on the branch, 45 on main, 66 merged = 45 + (42 - 21): none dropped, none duplicated.
Merged clean: WebUI.h, tools/measure_savings.py (below), src/SystemDefs_Limits.h, AGENTS.md, build.yml, both CHANGELOGs, the READMEs, the manual, ECONOMIA_DE_RECURSOS.md, tools/feature_savings.json.

tools/measure_savings.py, how the two halves meet (the merged file read whole)
- Main's half (201 changed lines) is the floor: flash_fill( ), FLASH_TOL, pio_build( ) returning 'fill', gate_flash( ) = flash - fill, measure( ) keeping fill_base/fill_off, print_table( ), cmd_check( ) letting a drop of FLASH_TOL pass.
  The branch's half (98 changed lines) is the matrix: the ui=<id> flips (CHOICES, flip_key, cost_env, flips_of, plan_flip), env_text( ) naming the Angulo's static-RAM budget, the 13th check, SIMUT_SAVINGS_PIO.
  The matrix builds through pio_build_full( ), which keeps the whole `used` (the slot holds the fill), so the halves share no function body: the changed lines of the merged file against main's are exactly the branch's 98, and against
  the branch's exactly main's 201 (compared as sets of added and removed lines: identical both ways). They meet in pio_run( ): pio_build( ) asks for the map through it and the branch picks the pio binary in it; the suite holds both in pio_run_cases( ).
  --check/--update never take the ui= branch, --matrix/--checks never read 'fill'; no code that uses bank_pad, a 'pad' key or BANK_SECTION is left in tools/ (the label of main's case [e5] still says "it was 0 for bank_pad", as history).
- test_measure_savings.py 66/66. Main's 15 mutations (sav_20261029/mutate.py) on the merged file: each caught by exactly the cases its table names (m1..m15 "== expected"). Its harness copies only tools/, and the branch's flips_cases( ) read src/ and the manifest,
  so the unmutated copy crashed (rc=1) after main's cases; stages/sync8/scripts/mutate_merged.py links the rest of the tree beside the copy and the unmutated copy is rc=0, none failed.

The eleven images (from scratch, the shim first in PATH, no PLATFORMIO_*, under flock; the baseline is the same eleven at 22d83de built first the same way: sha256-equal to sync 7's in all eleven, the instrument's control)
  image                 22d83de -> merge        main's pair (c902894 -> cf97b95)
  pico_w_release        1,035,956 -> 1,036,060  +104   +112
  pico_w_test           1,036,772 -> 1,036,876  +104   +112
  pico_w_test_https     1,041,220 -> 1,041,220     0      0 (other sha256)
  pico_w_asserts        1,038,260 -> 1,038,364  +104   +112
  pico_w_alpha            990,380 ->   990,380     0      0 (other sha256)
  pico_w_air            1,035,340 -> 1,035,340     0      0 (other sha256)
  pico2_w_release         999,936 -> 1,000,048  +112   +112
  pico_wire               604,312 ->   604,424  +112   +112
  pico_wiretft            787,664 ->   787,776  +112   +112
  pico_wiretft_angulo     827,504 ->   827,616  +112   (not a main image: like pico_wiretft)
  pico2_w_angulo        1,036,752 -> 1,036,864  +112   (not a main image: like pico2_w_release)
Decoded (nm -S, objdump -h, stages/sync8/decode): in every image that carries the Files page exactly two symbols move, WebUI_GZ::FILE_PAGE_GZ (+109..+112 B, #299) and WebUI_GZ::LOGIN_PAGE_GZ (+1 B, the version string of #300), and only .rodata changes.
The three Pico W images that grow 104 B where main's grow 112 B (pico_w_release, pico_w_test, pico_w_asserts) are the same two symbols: .rodata = the symbols (+110 B) + the bytes no symbol covers (-6 B here, +2 B on main's tree), counted per ELF
(pico_w_release 414,288 -> 414,392 = +110 - 6; main's 414,296 -> 414,408 = +110 + 2; pico2_w_release +113 - 1 here, +111 + 1 on main's): the alignment of the entries behind the page blobs, not a symbol. pico_w_alpha and pico_w_air carry the page too (the same two symbols grow, +109 and +1 B, and no section changes size: the bytes no symbol covers give back what they take) and pico_w_test_https has no Files page (LOGIN_PAGE_GZ +1 B, no section
changes size): the three keep their size with another sha256, as main's do.

The savings floors (tools/measure_savings.py --check, SIMUT_SAVINGS_PIO = the shim's pio): 15 of 16 hold. Measured - floor in flash: 0 for twelve, web_history -6 (inside FLASH_TOL), tel_mqtt +2, syslog +3; RAM +0 for all.
tft_graph saves 20,365 B against its floor of 20,433 B: 68 B below, 60 past the tolerance. It is not this merge: the same tool on main's tree (cf97b95) gives 20,433 B (main's floor, to the byte), and on 22d83de, before the merge, 20,365 B (stages/sync8/tg_*.txt).
It is the branch's UI-as-driver code in the release images: what the switch takes out of DisplayManager::handleTouch( ) is 1,804 B here against 1,832 B on main and out of DisplayManager::loopCore1( ) 436 B against 484 B (-28, -48), and a few
compiler switch tables differ (CSWTCH.*, 116 B on main's side). feature_savings.json is NOT updated here: the floor is main's number and the nightly runs on main; the --update, with this explanation, belongs to the merge that takes the branch into main.
A first run of the floors let PlatformIO's library installer start a git clone of one of the project's own libraries for a temporary environment (network, which this step does not allow): it was interrupted (SIGINT; TempEnvs put profiles.ini back, tools/generated
unchanged) and the run was repeated with GIT_ALLOW_PROTOCOL=file and the proxies on a closed port. The figures above are from the repeat.

Proofs (host only; records in stages/sync8)
- check_flash_budget.py <env> <log> right after each build: eight over their budgets by 104..112 B before budget_set.py (pico_w_alpha, pico_w_air and pico_w_test_https at 0), exit 0 for all eleven after it (run on the saved images: the floors had
  rewritten .pio/build by then); --table OK (11 images, 1 exempt); test_flash_budget.py 24 ok; gen_features.py --check; check_readme_numbers.py (772 cases in 11 suites, unchanged: main adds no native case); git diff --check; no conflict marker anywhere.
  pico_w_test is 3,267 B and pico_w_asserts 1,779 B under the OTA ceiling (the gate's WARN: under one sector, as before).
- tools/test_webui_upload_result.py PASS and tools/test_webui_theme_names.py PASS with the union in both packs. V-PIX: 951 of 951 frames equal to the records, six contact sheets and base en/es equal, its control bites. tsv_check.py: 394 rows, 0 findings.
  tools/tft_ui tests: 8 files, 0 fail. check_lang_packs.py, check_ui_texts.py (0 findings): clean.
- ci_gates_local.py --raso (the shim first in PATH, a depth-1 clone with the working tree laid over it): 68 of 69 steps passed; 1 skipped (the AddressSanitizer/UBSan host suites: they need PLATFORMIO_BUILD_FLAGS, CI only);
  the one failure is the fuzz step (clang++ cannot load libclang-cpp.so.18.1 on this host, as in the sync 6 and 7 records). The eleven native suites: 366 + 62 + 34 + 14 + 59 + 54 + 17 + 13 + 51 + 19 + 83 = 772 cases, all succeeded.

The host proofs above (V-PIX, tsv_check, the tft_ui tests, the packs, check_ui_texts, the authorization matrix) ran on the tree
before budget_set.py rewrote PLANO_STABLE section 3.1, flash_budget.json and two comment lines of features.toml, files none of them
reads. The review re-ran tsv_check (394 rows, 0 findings), check_ui_texts, check_lang_packs, the two web tests,
check_readme_numbers (772 cases), test_flash_budget (24 ok), check_flash_budget.py --table and gen_features.py --check on the
committed tree: all pass. Floor run: tft_graph saves 20,365 B here against main's floor of 20,433 B; the tree before the merge
(22d83de) gives the same 20,365 B and main's tree 20,433 B. That is the branch's UI-as-driver code in six DisplayManager symbols,
not the merge. It is left for the branch's merge into main; feature_savings.json is main's, unchanged.

Not verified on hardware.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uy7K2mzEmmjBnpxpdYNDFu
angeloINTJ pushed a commit that referenced this pull request Oct 30, 2026
Ângelo Moisés Alves @claude
tools: the savings test says which of its cases carry an id
4061cd6 
After sync 8 the suite holds the 45 cases of #302, each with an id the
mutations of that change are matched to, and the 21 cases of the
configurator's matrix side, which carry none. The docstring of
tools/test_measure_savings.py and its row in tools/README.md said that
every case carries an id (the review of sync 8). Text only: 66/66 ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uy7K2mzEmmjBnpxpdYNDFu
@angeloINTJ angeloINTJ mentioned this pull request Oct 30, 2026
Merged
@angeloINTJ
angeloINTJ deleted the fix/savings-without-fill branch October 31, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ci Continuous integration and automation documentation Improvements or additions to documentation tools Build tools, scripts, and developer tooling

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Savings watermark failed on main

1 participant

@angeloINTJ