[nuraci]

The ESP32 keeps LVGL's allocations in TLSF pools held in GC blocks
(ports/micropython/src/lv_mem_core_micropython.c, LVMP_POOL), because gc_alloc()
walks the allocation table from the first free block. Other ports with a large GC heap
have the same problem, and have no multi_heap. This lets a board turn the pools on with
LVGL's own TLSF. Off by default: a board opts in with CFLAGS += -DLVMP_POOL_TLSF=1.

Why

STM32H743, MicroPython 1.29.0, 28 MB GC heap in a 16-bit SDRAM, LVGL 9.5, a 1024x600
display with 48 themed buttons, full refresh with lv.refr_now(), without the pools:

GC heap state	8 KB bytearray	full refresh
fresh	181 us	54 ms
2 MB of bytearrays alive behind a free block	611 us	575 ms
30 000 small objects alive	82 us	53 ms
the same objects freed	609 us	574 ms

A refresh makes about 1200 LVGL allocations; a multi-block gc_alloc() does not move
gc_last_free_atb_index, so each one walks past every live block after the first hole.
So the frame time depends on what the application left in the heap.

The change

• src/lvmp_pool.h (new): the LVMP_POOL switch and pool sizes, shared by three files.
  LVMP_POOL is now also on when the build defines LVMP_POOL_TLSF=1.
• src/lv_mem_core_micropython.c: unchanged on ESP32 (multi_heap); otherwise the same
  calls mapped to lv_tlsf_*.
• src/lvmp_tlsf.c (new, in micropython.mk and micropython.cmake): LVGL compiles
  lv_tlsf.c only with LV_STDLIB_BUILTIN, so this file includes it in a translation
  unit of its own with that setting, and LV_MEM_SIZE = one pool. Nothing in LVGL or
  lv_conf.h changes.
• The root pointer lvmp_pool_mem moves to src/lvgl_micropython_build.c: make builds
  put lv_mem_core_micropython.c in SRC_USERMOD_LIB_C, which is not scanned for
  MP_REGISTER_ROOT_POINTER (with the pools on, the build fails with "mp_state_vm_t has
  no member named lvmp_pool_mem").

Tested

On this branch (92f8c0a + this commit), STM32H743 board, IPS 1024x600 panel, 2 MB of
application data kept in the heap:

Display mode	full refresh, before	with the pools
DIRECT, 2 buffers	273 ms	53-64 ms
PARTIAL, 200 lines	243 ms	78 ms

• 300 event callbacks that only LVGL refers to survive garbage and collections, also
  after lv.deinit()/lv.init(); 3 soft resets with LVGL left initialised, heap back to
  the same value each time; a touch game played by hand.
• Build, no warnings: STM32 (make, pools on), RP2350 (CMake, pools off, no pool symbols in
  the firmware).
• RP2350 board, pools off, on this branch against the same board on the old code:
  LVGL smoke test with soft resets, panel at 57.8 fps with no underruns, a touch game
  played by hand. Refresh times match within 7 us; the out-of-memory test behaves the
  same on both.
• pytest ports/micropython/tests: 5 passed.

ESP32-S3 (16 MB flash, 8 MB PSRAM, no display attached), MicroPython 1.29, ESP-IDF 5.5.2:
the LVGL smoke test with soft resets and the 300 callbacks test pass with the pools on
(lv.init() plus a display takes 262 208 B of GC heap, so a pool is held, and the
multi_heap code is in the ELF) and with a stock build (pools off).

To get the pools on I had to change three things in my test copy, none in this PR: see
"Pools off on a stock ESP32 build" below.

Pools off on a stock ESP32 build

Not caused by this change, and left alone here: with stock MicroPython 1.29 and
USER_C_MODULES the pools do not reach the firmware, for two reasons:

• MICROPY_MALLOC_USES_ALLOCATED_SIZE is 0 on the esp32 port, and setting it to 1 does
  not build as is: esp32_rmt.c calls m_free() and m_realloc() without the size.
• The sources of the module are compiled twice, for __idf_main and for micropython.elf,
  and the second has no -DESP_PLATFORM. The objects linked are those, so LVMP_POOL is
  0 in them even with the flag on. The root pointer is still registered, unused.

Your builds may differ (I did not try micropython-pydevices). If it happens there too,
I can open a separate issue.

The "before" figures were measured on PyDevices/lvgl-micropython 3e287e7, the code this
moved from. Tests, logs and the board code:
Copper-And-Code/mpy-lvgl-boards,
docs/stm32h743.md, tests/hil/stm32/t05_lvgl_gc.py.

🤖 Generated with Claude Code

[bdbarnett]

Thank you, this is a careful change and a clear write-up.

I checked it merged onto today's main. With the pools off it behaves like main, and an ESP32-S3 (SPIRAM_OCT) firmware with this module builds with no warnings from the new files. On a Raspberry Pi 4 (unix port, 32 MB heap, 48 buttons at 1024x600, partial rendering), a full refresh with 2 MB of bytearrays alive behind a free block took 25.7 ms with the pools off and 5.9 ms with -DLVMP_POOL_TLSF=1, the same as a fresh heap. 300 callbacks that only LVGL refers to survived collections both ways, and five lv.deinit()/lv.init() cycles with collections in between ran clean.

You were right about stock ESP32 builds too: ours compile the pool code out as well, since MICROPY_MALLOC_USES_ALLOCATED_SIZE is 0 on that port. That's #10 now.

[nuraci]

Thanks a lot for the thorough testing, especially the Pi 4 numbers and the deinit/init cycles. 25.7 → 5.9 ms is a nicer confirmation than I could have produced myself.

Glad #10 is tracked. I don't have ESP32 hardware to test on, so I'll leave that one with you.

On my side I'll keep focusing on the STM32 port (H7 boards with LTDC/SDRAM). If it's useful, I can report back on how the pools behave there once my board setup is running.

[bdbarnett]

Yes please, H7 figures would be welcome. We have one data point there already, and it may save you some time.

On an Arduino GIGA R1 (STM32H747) with its display shield, the GC heap is 8 MB in SDRAM, and the pools didn't help. With -DLVMP_POOL_TLSF=1, a forced full refresh took 87.6 ms against 85.2 ms without, and gc.collect() took 106.1 ms against 102.9 ms. A collection there is dominated by sweeping the large SDRAM heap, not by LVGL's objects, so taking LVGL out of the walk barely registers. The Pi 4 gained because there LVGL's objects were most of what the collector walked. Details are in PyDevices/micropython-pydevices#292.

If your H7 board keeps a smaller heap in internal RAM, I'd expect a result closer to the Pi's, and that's the case we haven't measured.

One trap we hit on the way, in case you drive LVGL from a script: calling lv.refr_now() from Python lets MicroPython run scheduled callbacks between the flush callback's bytecodes. If one of them touches a widget, LVGL's "Invalidate area is not allowed during rendering" assert fires, and the default handler spins forever with the scheduler locked, which looks like a hard hang. That's #12 here.

[nuraci]

Thanks, that's useful, and it explains the difference from our numbers rather than contradicting them. On our STM32H743 (28 MB heap in SDRAM) the pools didn't matter on a fresh heap either: the gain showed up with 2 MB of bytearrays behind a free block, where every one of the ~1200 allocations of a refresh walked the allocation table from that free block (575 ms → 50-51 ms, against 54 ms on a fresh heap). Was there a free block below the bytearrays on the GIGA? Without one the walk is short and I'd expect exactly what you saw.

Caveat: those H7 numbers are from lvgl-micropython with a backport of these pools, not from omnipython-lvgl yet. I'll redo them on omnipython-lvgl once the board is moved over, and I can also try a smaller heap in AXI SRAM, the case you haven't measured.

On #10: I now have an ESP32-S3 board (4848S040, 8 MB PSRAM) and firmwares built with the pools, multi_heap and TLSF. Results soon.

Our flush callbacks are in C, so #12 shouldn't bite us, but thanks for the warning.

[bdbarnett]

No, there wasn't: the GIGA run had a live app on the heap and nobody staged a hole, so I ran your layout on it today. The two builds came from the same tree, one with -DLVMP_POOL_TLSF=1 and one without. The heap is 8 MB in SDRAM with about 1.2 MB in use, the screen has 48 buttons with labels and 6 sliders, and each figure is the range over two runs.

	fresh	one hole	many holes
bytearray(100), either build	22-54 µs	1.1-1.2 ms	1.9-2.1 ms
50 LVGL labels created and deleted, no pools	22 ms	16-57 ms	65-73 ms
same, pools	16 ms	16 ms	17-18 ms
full refresh, median, no pools	31.8 ms	32.0-32.1 ms	33.2-39.4 ms
full refresh, worst of 20, no pools	64-65 ms	33-176 ms	215-221 ms
full refresh, median, pools	33.0-33.1 ms	33.1-33.2 ms	33.4-33.5 ms
full refresh, worst of 20, pools	33 ms	33-42 ms	51 ms
gc.collect(), no pools	34-37 ms	82-83 ms	78-79 ms
gc.collect(), pools	40 ms	85-87 ms	82-84 ms

"One hole" is yours: a small bytearray freed below 1024 live 2 KB bytearrays. "Many holes" interleaves the same 2 MB with small bytearrays and frees those, more like a heap that has run for a while.

So the walk is real here too, and the pools take LVGL out of it. LVGL's own allocations get three to four times slower on the fragmented heap without pools, and stay flat with them. The one-hole case varies between runs, probably because a single hole stops being the first free block as soon as something small fills it.

Where we differ is the refresh. Its median barely moves here, which suggests that once LVGL's caches are warm, a full refresh of this screen makes few allocations. The cost lands on the refreshes that do allocate: the worst of 20 went to about 220 ms without pools and stayed near 50 ms with them. On a fresh heap the pools cost about a millisecond per refresh and 3-5 ms per collection, which is what we saw in #292. Your 575 → 50 ms suggests your refresh allocates on every pass, so I'd expect the median to move for you where it doesn't for us.

That changes our conclusion for the GIGA. Our earlier "no help" was a fresh heap. A long-running app is more likely to look like "many holes", and there the pools bound the worst refresh.

Thanks for the S3 runs on #10. Here's the script, if you want to compare layouts directly. display_driver and multimer are our display glue and timer scheduler, so swap in your own setup and drop the hold().

heap_layout.py

import gc
import time

import lvgl as lv
import display_driver  # noqa: F401  (display, input, LVGL loop)
import multimer

REFRESHES = 20
COLLECTS = 10
PROBES = 300
PAD = 1024  # 1024 x 2 KB = 2 MB


def median(xs):
    xs = sorted(xs)
    return xs[len(xs) // 2]


def build_screen():
    scr = lv.obj()
    scr.set_flex_flow(lv.FLEX_FLOW.ROW_WRAP)
    for i in range(48):
        b = lv.button(scr)
        b.set_size(110, 60)
        lv.label(b).set_text("Btn %d" % i)
    for i in range(6):
        s = lv.slider(scr)
        s.set_width(300)
        s.set_value(i * 15, 0)
    lv.screen_load(scr)
    return scr


def probe_alloc():
    gc.collect()
    t = time.ticks_us()
    for _ in range(PROBES):
        x = bytearray(100)  # noqa: F841
    return time.ticks_diff(time.ticks_us(), t) / PROBES


def churn(scr):
    with multimer.hold():
        t = time.ticks_us()
        for _ in range(5):
            objs = [lv.label(scr) for _ in range(10)]
            for o in objs:
                o.delete()
        return time.ticks_diff(time.ticks_us(), t) / 1000


def measure(name, scr):
    a = probe_alloc()
    c = churn(scr)
    refr = []
    with multimer.hold():
        for _ in range(REFRESHES):
            scr.invalidate()
            t = time.ticks_us()
            lv.refr_now(None)
            refr.append(time.ticks_diff(time.ticks_us(), t) / 1000)
    coll = []
    for _ in range(COLLECTS):
        t = time.ticks_us()
        gc.collect()
        coll.append(time.ticks_diff(time.ticks_us(), t) / 1000)
    print("%-10s alloc_us=%.1f labels50_ms=%.1f refresh_med=%.1f max=%.1f collect_med=%.1f" % (
        name, a, c, median(refr), max(refr), median(coll)))


scr = build_screen()
with multimer.hold():
    lv.refr_now(None)
gc.collect()
measure("fresh", scr)

gc.collect()
hole = bytearray(8)
pad = [bytearray(2048) for _ in range(PAD)]
del hole
gc.collect()
measure("one-hole", scr)
del pad
gc.collect()

keep = []
tmp = []
for _ in range(PAD):
    keep.append(bytearray(2048))
    tmp.append(bytearray(8))
del tmp
gc.collect()
measure("many-holes", scr)

[nuraci]

No hole in our GIGA question then, thanks for staging one. I ran your script on an STM32H7_CORE V1.0 (STM32H743, 28 MB GC heap in SDRAM, 1024x600 RGB565), two builds from the same tree with and without LVMP_POOL_TLSF=1, two runs each with a hard reset before every run; both runs gave the same figures. The only change to the script: multimer.hold() became a context manager that stops and restarts our display timer. Note this is still lvgl-micropython with our pool patch, not omnipython-lvgl: moving the STM32 boards over is next.

	fresh	one hole	many holes
bytearray(100), either build	101-130 µs	66-79 µs	862-869 µs
50 LVGL labels created and deleted, no pools	15.6 ms	31.6 ms	93.2-93.7 ms
same, pools	14.5 ms	14.5 ms	17.8-17.9 ms
full refresh, median, no pools	68.6 ms	359.3 ms	360.0 ms
full refresh, worst of 20, no pools	68.7-68.8 ms	359.6-359.8 ms	369.6-370.7 ms
full refresh, median, pools	68.6 ms	68.6 ms	68.6 ms
full refresh, worst of 20, pools	68.6-68.7 ms	68.7 ms	68.6-68.7 ms
gc.collect(), no pools	31.7-31.8 ms	80.8-81.0 ms	77.0-77.1 ms
gc.collect(), pools	37.6 ms	87.0 ms	82.9 ms

So your guess holds: here the median moves too, 68.6 → 360 ms without pools once there is a hole, and stays at 68.6 ms with them. Our refresh allocates on every pass. On a fresh heap the pools cost nothing on the refresh and about 6 ms per collection. One difference from your table: the plain bytearray(100) probe is not slower with one hole here, only with many.

About #10: the ESP32-S3 runs are not posted yet, sorry if I gave that impression. The three builds (no pools, multi_heap, TLSF) are ready and I'll run this same script on them as soon as the board is back on the bench.