Repository navigation
ESP32: LVGL's memory pools never reach the firmware #10
Description
On ESP32, lv_mem_core_micropython.c is meant to keep LVGL's allocations in multi_heap pools held in GC blocks, so a refresh doesn't walk the GC allocation table past every live block. The pools are compiled in only when MICROPY_MALLOC_USES_ALLOCATED_SIZE is 1, and MicroPython's esp32 port leaves it at the default, 0. So the pool code compiles out, lv_malloc_core calls m_malloc directly, and no pool root pointer is registered.
To see it, build ESP32_GENERIC_S3 (SPIRAM_OCT) with this module (MicroPython 1.29/1.30 preview, ESP-IDF 5.5) and disassemble lv_malloc_core in micropython.elf: its only call is to m_malloc, and the generated root_pointers.h has no lvmp_pool_mem.
What it costs: refresh time grows with what the application keeps in the heap. On the unix port, a full refresh of 48 buttons at 1024x600 takes 3.9 ms with a fresh heap and 7.1 ms with 2 MB of bytearrays alive behind a free block; on an STM32H743 with a 28 MB heap the same pattern went from 54 ms to 575 ms (#6).
Still open: setting MICROPY_MALLOC_USES_ALLOCATED_SIZE to 1 on esp32 doesn't build as is (esp32_rmt.c calls m_free() and m_realloc() without sizes), and #6 reports that a USER_C_MODULES build compiles this module's sources a second time without -DESP_PLATFORM. The TLSF pools from #6 don't depend on that flag, so turning those on for ESP32 builds may be the simpler route.
Activity
MicroPython: LVGL's memory pools on ESP32, opt-in (#15)
nuraci commentedon Nov 10, 2026
S3 numbers, now on #15 (c34fbee). ESP32-4848S040: ESP32-S3 at 240 MHz, 8 MB octal PSRAM, 480x480 RGB panel, about 4.6 MB of free GC heap. MicroPython 1.29.0, ESP-IDF 5.5.2. Two builds from one tree: no pools, and LVMP_POOL_TLSF=1 in mpconfigboard.h. Your layout script, with multimer.hold() replaced by a context manager that stops and restarts our display timer; two runs per build, each after a reset, ranges over both. The 48 buttons don't all fit on 480x480, so the refresh draws only the visible part.
Unlike your P4, here the median refresh moves: about 6-7x slower without pools once the heap has holes, as on our H743, and flat with them.
I had also run the same script on v9.5.53 with one 256 KB pool (on multi_heap or TLSF, turned on by hand; the two measured the same). The growing pools keep the refresh exactly as that did, at a much lower cost on a fresh heap:
The soak has animations, allocation churn, 2 MB of application data and a gc.collect() every 10 s, but no staged holes, so with #15 it no longer costs anything there. On both builds the same tests passed: audio with LVGL animating, WiFi (TCP downloads, reconnect, DNS + HTTP) and the soak (0 errors, memory flat). Our shell and weather demo also run on the pools build.
A side note, not pool-related: from v9.5.53 to c34fbee the free GC heap after the display starts dropped by 128 KB, with or without pools. The firmware grew by 55 KB, which I'd guess is mostly the file and function names the assert handler of #13 keeps. This board uses CONFIG_SPIRAM_XIP_FROM_PSRAM, so code and constants are copied into PSRAM in 64 KB pages, and that might explain the 128 KB. I haven't checked it section by section, so take it as a guess. It may matter for boards with less PSRAM.
Logs: docs/log/esp32-pools-c34fbee-4848S040.txt (#15) and docs/log/esp32-pools-4848S040.txt (v9.5.53)
nuraci commentedon Nov 10, 2026
Same layout script on the STM32H743 (STM32H7_CORE V1.0, 480 MHz, 28 MB GC heap in SDRAM, 1024x600 RGB panel), now built on c34fbee's ports/micropython with no patches of ours; LVMP_POOL_TLSF=1 from the board's make flags. Fresh / one hole / many holes, median of 20 refreshes, 2 runs each (identical within 1 ms):
With the growing pools the cost dropped from about 250 KB and 6 ms per collection (fixed 256 KB pools, our old patch) to 25 KB and 1.5 ms, and the refresh no longer depends on how fragmented the heap is. The build without pools gives the same numbers as the old binding.
Also on c34fbee: soft resets with LVGL live, 300 callbacks reachable only through LVGL surviving collections and lv.deinit()/lv.init(), a 15 minute soak (35 refreshes/s, 0 errors, memory flat) and our 10 examples — all fine.
bdbarnett commentedon Nov 10, 2026
Thank you for the S3 and H743 runs: an RGB panel on the S3 and a 28 MB heap on the H7 are two cases we couldn't measure, and in both the pools keep the refresh flat once the heap has holes.
Your guess about the 128 KB is right, and the 64 KB pages are the whole story. We built ESP32_GENERIC_S3 (SPIRAM_OCT, ESP-IDF 5.5.4) with only the LVGL module, twice, changing nothing but the omnipython-lvgl commit, and compared the two images:
.flash.text).flash.rodata).data,.bss)Nearly all of it is the assert handler from #13. This build has about 1,230 LVGL assert sites. Each one used to be a three-byte jump to itself. Each now tests the weak handler, loads the file name, line and function name, and calls it, about 20 bytes: that's the 23 KB of code. The 27 KB of constants are the names, 879 function names (19.9 KB) and 98 source paths (6.2 KB). The pool changes in #15 come to a few hundred bytes. The paths are as long as the path the compiler was given, so a tree built from deeper in the filesystem grows more, which may be the gap between our 51 KB and your 55 KB.
With
CONFIG_SPIRAM_XIP_FROM_PSRAM, ESP-IDF copies the flash-mapped constants and code into PSRAM at boot, a whole 64 KB MMU page at a time, and keeps those pages out of the PSRAM it gives the heap. That'ss_xip_psram_placement()in components/esp_psram/system_layer/esp_psram.c, which subtractsMMU_PAGE_TO_BYTES(used_page)for each of the two; the page counts come fromCache_Count_Flash_Pages()in xip_impl/mmu_psram_flash.c. Code and constants are rounded up separately, so any growth costs 0, 64 or 128 KB depending on where each ended. In our build the code had 13.8 KB left in its last page and the constants 13.2 KB, and both overflowed. Built again with XIP on, the two commits take the same 30 + 10 and 31 + 11 pages: 2,560 KB of PSRAM against 2,688 KB. MicroPython's esp32 port grows its GC heap from that same PSRAM pool, up to its largest free block, so the most the GC heap can reach drops by the same 128 KB.Without XIP nothing is copied, and since internal RAM didn't change, the heap doesn't move at all. That's the default on MicroPython's S3 boards, so this shows only on builds like yours. If you want to see it on the board, ESP-IDF's boot log prints the pool size ("Adding pool of ...K of PSRAM memory to heap allocator").
#19, now merged, makes the handler much cheaper. It passes only the file's base name (
__FILE_NAME__) and the line, and is declarednoreturnand not weak, so each site is a load, a move and a call. File and line are enough to find the assert, so the function name was redundant. Built the same way, that comes to 6.3 KB over v9.5.53 instead of 50.8 KB (code +4.7 KB, constants +1.6 KB), within the same pages as v9.5.53 here, so with XIP it costs no PSRAM in this build. The message is nowRuntimeError: LVGL assert failed at lv_display.c:614, and the MicroPython smoke test triggers one to check it.nuraci commentedon Nov 11, 2026
Thanks for taking the 128 KB apart page by page. The separate rounding of code and constants
also explains something we had not looked at closely enough.
We moved our pin to 665dec5 (#19) and wrote the three images, one after the other, on the
same ESP32-4848S040 (ESP-IDF 5.5.2,
CONFIG_SPIRAM_XIP_FROM_PSRAM). The boot log doesn'treach our UART, so the pool size is the PSRAM region from
esp32.idf_heap_info(esp32.HEAP_DATA):So in our build only the constants crossed a page, and the step from v9.5.53 to c34fbee was
64 KB of PSRAM region, not 128 as I wrote before. That figure came from the free GC heap on
the other board, not from the region itself, and I haven't tracked down the other 64 KB. With 665dec5 the region is back to the v9.5.53 size
to the byte, and
gc.mem_free()after boot goes up by exactly 65,536 B over c34fbee.The message works here too:
display.set_tile_cnt(256)givesRuntimeError: LVGL assert failed at lv_display.c:614and LVGL keeps running.Elsewhere #19 saves 46,336 B of flash on the STM32H743 boards and the same on the RP2350.
Panel rate and refresh times on the S3 are unchanged (42.20 fps, full refresh 93.7 ms
average); the H743 and RP2350 boards are next on the bench.
665dec5 goes into our next release, which also moves the RP2350 board to omnipython-lvgl.
nuraci commentedon Nov 11, 2026
Follow-up: 665dec5 is now in our v0.1.6 release, on all six boards (four STM32H743, the RP2350 and the ESP32-S3). Each image was checked on hardware, including the assert from your smoke test: https://github.com/Copper-And-Code/mpy-lvgl-boards/releases/tag/v0.1.6
The PSRAM page counts per build are in docs/log/esp32-psram-665dec5.txt, in case they're useful.
bdbarnett commentedon Nov 11, 2026
The fix described above (#19) is released in v9.5.54: https://github.com/PyDevices/omnipython-lvgl/releases/tag/v9.5.54