feat(thumbnails): Tika-based audio cover art extraction - #3362
Closed
dschmidt wants to merge 22 commits into
Closed
Conversation
Rename RawImageDecoder to RawTiffDecoder and qualify candidates by their actual stream instead of trusting the tags: CR2 keeps its full-size JPEG as an IFD0 strip, DNG stores the raw sensor payload as lossless JPEG (SOF3) which starts with a regular SOI. A JPEG segment walk to the SOF marker accepts only the DCT processes common decoders render.
Cap the IFD queue at maxIFDs so a file packed with huge SubIFD counts can no longer grow it without bound, and walk the embedded JPEG by a bounded number of header segments so a preview whose SOF sits past 64KB of leading metadata is still found instead of being scanned only within a fixed byte window.
Both are TIFF-based camera raw formats handled by the same generic walker; NRW detection needs the reva mimetype mapping in opencloud-eu/reva#773, SRF is already mapped. No sample files on hand for these two, added by format structure alongside their verified siblings.
Reject a preview whose declared length exceeds 100MB rather than serving it; previews are camera-generated JPEGs, so this bounds the output independently of the input file size.
…ogging Add big-endian, largest-candidate, orientation-direction, isRenderableJPEG classification and maxPreviewLength tests plus a ForType raw-dispatch check; log a raw file without an embedded preview at debug instead of error; name the TIFF magic constant and de-enumerate the decoder doc comment.
… width Review of #3332 surfaced two BigTIFF-only defects in the embedded-preview walker (attacker-controlled input): - 64-bit IFD and SubIFD-array offsets were bounds-checked with `off + n > len`, which wraps for a crafted offset near 2^64, bypassing the guard and slicing out of range -> panic (recovered by the framework into a 500 + stack log on every crafted request, defeating the no-panic goal). Now overflow-safe. - a tag value was always read as an 8-byte Uint64 in BigTIFF; a LONG (4-byte) offset in a big-endian BigTIFF was thereby shifted. Read it at its declared type width instead. Adds tests for the overflow paths (assert ErrNoImageFromRawFile, no panic), the big-endian LONG offset, and strengthens the truncation test to assert the error. Trims a few over-long comments.
isLongType accepted only LONG/LONG8, so a SubIFDs entry using the dedicated IFD (13) or, in BigTIFF, IFD8 (18) pointer type (both standards-compliant and common) was skipped and its embedded preview never discovered. Accept those pointer types for the SubIFDs tag via isSubIFDType, keeping the stricter isLongType on length fields so they never accept a pointer type. IFD8 reads at 8-byte width like LONG8.
Replace the in-process TIFF/BigTIFF walker (rawtiff.go) with a Tika-based extractor: raw images are sent to a Tika server's /unpack endpoint, which returns the embedded JPEG previews (Tika's RawTiffParser emits them as embedded documents). The largest renderable JPEG is decoded through the image pipeline. This aligns preview extraction with the search service (same Tika, same format coverage) and drops a large hand-rolled parser. Raw thumbnails now require THUMBNAILS_TIKA_TIKA_URL; without it, raw types fall back to the default decoder (no thumbnail). Audio artwork extraction stays in process for now and moves in a follow-up.
Tika sniffs TIFF-based raws (NEF/DNG/ARW/PEF) as generic image/tiff by content; only the filename extension routes them to the raw parser. Thread the source name through to the unpack request via Content-Disposition.
… CI check requires
… Convert interface The Tika preview extractor does a network call; it was using context.Background(). Pass the request context from the gRPC handler through Convert so the unpack call respects cancellation and deadlines. Non-I/O decoders ignore the context.
dschmidt
added
Type:Maintenance
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 26 |
| Duplication | -28 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
… no-image error Extract tikaUnpack + readZipEntry, rename tikaExtractPreview to tikaLargestPreview, and rename ErrNoImageFromRawFile to the generic ErrNoEmbeddedImage. No behaviour change; sets up reuse for audio cover art.
dschmidt
changed the title
dschmidt
force-pushed
the
feat/thumbnails-audio-tika
branch
2 times, most recently
from
August 19, 2026 17:51
b1cbc52 to
23fd802
Compare
Contributor
Author
|
If we agree on the Tika approach, this supersedes #3208 :) |
When a Tika server is configured, audio cover art is extracted through it by default, selecting the tagged front cover (ID3 APIC 'Cover (front)') via /unpack/all sidecar metadata, else the first embedded image. TikaDecoder picks by mime type: audio/* front cover, everything else the largest preview. THUMBNAILS_PREPROCESSOR_AUDIO_PROCESSOR=builtin forces the in-process extractor; it is deprecated and slated for removal in the next major.
dschmidt
force-pushed
the
feat/thumbnails-audio-tika
branch
from
August 19, 2026 17:55
23fd802 to
1592ab2
Compare
dschmidt
commented
Aug 20, 2026
| type Preprocessor struct { | ||
| Tika Tika `yaml:"tika"` | ||
| // AudioProcessor selects the audio cover-art extractor. | ||
| AudioProcessor string `yaml:"audio_processor" env:"THUMBNAILS_PREPROCESSOR_AUDIO_PROCESSOR" desc:"The processor for extracting cover art from audio files. When a Tika server is configured it is used by default. Set this to 'builtin' to force the in-process extractor even when a Tika server is configured. The 'builtin' processor is deprecated and will be removed in a future major release." introductionVersion:"%%NEXT%%" deprecationVersion:"%%NEXT%%" removalVersion:"%%NEXT_PRODUCTION_VERSION%%" deprecationInfo:"THUMBNAILS_PREPROCESSOR_AUDIO_PROCESSOR only selects the deprecated in-process audio extractor; once it is removed audio cover art always uses Tika when configured." deprecationReplacement:""` |
Contributor
Author
There was a problem hiding this comment.
Not sure we need this - one could argue, Tika is opt-in and when you choose to opt in, you get audio extraction via tika.
Thoughts?
dschmidt
force-pushed
the
feat/thumbnails-raw-embedded-preview
branch
from
August 30, 2026 11:47
4ba384f to
0d0a6a6
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #3332, which lands the shared Tika unpack helper and the generic
ErrNoEmbeddedImage.Route audio cover art through Tika, so all embedded-media extraction can eventually run through Tika (and
dhowden/tagbe dropped).When a Tika server is configured, cover art is extracted via Tika by default: it selects the tagged front cover (ID3 APIC "Cover (front)") via
/unpack/allsidecar metadata, else the first embedded image, matching the deterministic selection from #3208. Verified byte-identical to the in-process reader on real files (front-cover and untyped). Without a Tika server it stays in-process.THUMBNAILS_PREPROCESSOR_AUDIO_PROCESSOR=builtinforces the in-process extractor even when a Tika server is configured; thatbuiltinpath is deprecated and slated for removal in the next major.TikaDecoderpicks by mime type:audio/*front cover, everything else the largest renderable preview.