Repository navigation
Speed up JPEG decoding on ESP32-P4 and ESP32-S31 - #11634
mikeysklar wants to merge 4 commits into
Conversation
tannewt
left a comment
tannewt
left a comment
There was a problem hiding this comment.
Do we want to be explicit about whether the user wants hardware acceleration? Or do you think it is ok to do through speed testing?
I had a couple comments about structure.
|
|
||
| #if CIRCUITPY_JPEGIO_HW | ||
| // The JPEG engine decodes straight into Bitmaps whose pixels start on a cache line. | ||
| #define CIRCUITPY_DISPLAYIO_BITMAP_DATA_ALIGNMENT (64) |
There was a problem hiding this comment.
I don't really want to add this. For camera I just have the VM and port_malloc always alloc aligned to 64 bytes. JPEGIO can check to just make sure.
This is how to do it:
https://github.com/adafruit/circuitpython/pull/11614/changes#diff-c9bb8741d3fb6b2364b58fa90638114b80d0dc512982dc3fccb21cc2087f20b8R233
| void espressif_jpegio_reset(void) { | ||
| if (decoder_engine != NULL) { | ||
| jpeg_del_decoder_engine(decoder_engine); | ||
| decoder_engine = NULL; | ||
| } | ||
| } |
There was a problem hiding this comment.
Lets not do a bulk reset here. Instead, change the allocation in shared-bindings to trigger a finaliser and have it deallocate.
| #if CIRCUITPY_JPEGIO_HW | ||
| espressif_jpegio_reset(); | ||
| #endif | ||
|
|
There was a problem hiding this comment.
Without the bulk reset then remove this.
| self->stride = stride(width, bits_per_value); | ||
| self->data_alloc = false; | ||
| if (!data) { | ||
| data = m_malloc_without_collect(self->stride * height * sizeof(uint32_t)); |
There was a problem hiding this comment.
Don't change this. Just align all allocations.
CIRCUITPY_JPEGIO_HW (default 0) means the port provides
jpegio_jpegdecoder_decode_hw(). decode() calls it first when open() was
given an in-memory buffer. If it returns false, TJpgDec decodes the image
as before. The espressif port turns it on for esp32p4 and esp32s31, the
targets with a JPEG decoder peripheral, and implements it with
esp_driver_jpeg. Other targets are unchanged.
CIRCUITPY_DISPLAYIO_BITMAP_DATA_ALIGNMENT (default 4) sets the alignment
of the pixel memory a Bitmap allocates. The two chips raise it to the
64 byte cache line, so a decode of the whole image at 0,0 into a Bitmap of
the same size goes straight into the Bitmap. The pixels stay in the GC
heap. Other decodes go through a temporary buffer that is freed before
decode() returns. Skip indexes, and input the DMA can't read, use TJpgDec.
The engine is created on the first decode and deleted on VM reset. The
color conversion is full range BT601, as JFIF specifies. This needs
yuv_range in jpeg_decode_cfg_t, added in adafruit/esp-idf.
1280x720, median of 10, hardware from bytes vs TJpgDec from io.BytesIO:
ESP32-P4 (Adafruit P4 GPIO) 4:2:0 9.2 ms (302.1), 4:2:2 12.2 (327.5),
4:4:4 18.2 (385.2)
ESP32-S31 (ESP32-S31-Korvo-1) 4:2:0 16.0 ms (374.8), 4:2:2 20.9 (409.4),
4:4:4 31.0 (477.4)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Points at adafruit/esp-idf#19, which also merges #18. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Use 64-byte GC blocks and align the PSRAM heap areas to them, as the camera change does, so Bitmap pixels start on a cache line without a Bitmap-specific alignment option. Revert the Bitmap allocation change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each JpegDecoder now owns its JPEG engine and releases it when it is collected, instead of sharing one engine that is freed on VM reset. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
a730902 to
d69ef5f
Compare
|
Thanks! Changes:
I'd keep hardware decode on by default. It costs under 1% of flash and RAM on P4 and S31, and 720p decodes in 9 to 16 ms instead of 300 to 375 ms. Builds can opt out with Re-ran the hardware tests after these changes, rebased on main. All pass on P4 and S31. |
2 participants
What
jpegionow uses the chip's JPEG hardware on ESP32-P4 and ESP32-S31.Why
Software decode is too slow for video on these chips.
Needs adafruit/esp-idf#18 (adds
yuv_range) and #19 (S31 ROM fix).The esp-idf submodule points at #19, which includes both.
MICROPY_BYTES_PER_GC_BLOCKport_mallocchange as #11614.CIRCUITPY_JPEGIO_HWTJpgDec is timed from
io.BytesIO; the hardware frombytes.open()got bytes,scale=0, 16-bit Bitmap, baseline color JPEGdecode()returnsdecode()onlytests/circuitpython-manual/jpegio/hw_decode.pyscale=1fall back and match exactly; progressive raises as beforeAI assistance: written with Claude Code. I ran the tests above on hardware.