diff --git a/docs/openedx_content/decisions/0017-thumbnails-for-images.rst b/docs/openedx_content/decisions/0017-thumbnails-for-images.rst new file mode 100644 index 000000000..9b418404f --- /dev/null +++ b/docs/openedx_content/decisions/0017-thumbnails-for-images.rst @@ -0,0 +1,174 @@ +.. _openedx-content-adr-0017: + +17. Thumbnails for Images +========================= + +Status +------ + +Draft. Depends on :ref:`openedx-content-adr-0012`, :ref:`openedx-content-adr-0013` and :ref:`openedx-content-adr-0016`. + +Context +------- + +When a course author uploads an image to a course's "Files", stored as an :class:`Asset` (a shared static asset: a :class:`PublishableEntity` whose :class:`AssetVersion` points to a single :class:`Media`) per :ref:`openedx-content-adr-0013`, Studio automatically generates a smaller thumbnail image. This lets the Files page show a preview of each image without loading the full file. + +:ref:`openedx-content-adr-0013` (decision 6) drops the thumbnails generated by the legacy contentstore when migrating a course's Files to Assets, and leaves thumbnails to a separate ADR. This is that ADR. + +The Files page is not the only place that needs smaller versions of an image. The legacy contentstore is also used to generate two sizes of each course's card image for the dashboard and the about page (``CourseOverviewImageConfig``, 375×200 and 750×400 by default), and content library "cards" should show image previews too. Other places, like oversized images in HTML content, would also benefit from being resized before being sent to the browser. + +Three properties of the existing design shape this decision: + +- :class:`PublishableEntityVersion` objects, which back every :class:`AssetVersion` (as well as every :class:`ComponentVersion`), are created once and never updated. +- A thumbnail is not authored content: it is a derived, disposable artifact that can always be regenerated from the original file. The legacy contentstore already treats it this way: it is generated whenever an asset is uploaded or a course is imported, but never written into a course's export package. +- :class:`Media` is content-addressed: its file is stored under a path derived from the hash of its bytes, inside the ``media_file_namespace`` of its :class:`LearningPackage` (:ref:`openedx-content-adr-0012`, decision 6). Two :class:`Media` rows with the same bytes in the same namespace share a single stored file, even when they belong to different learning packages. + +Decisions +--------- + +1. Thumbnails are derived files, not content +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A thumbnail is never a :class:`PublishableEntity`, a :class:`PublishableEntityVersion` or a :class:`Media` row, and it is never associated with an :class:`AssetVersion` or a :class:`ComponentVersion`. Creating a thumbnail never creates a new version of anything. The source image continues to be versioned normally, like any other :class:`Asset` content, but the thumbnail does not participate in that versioning, in draft/publish, or in export and backup. + +2. Thumbnail generation is a generic API in openedx-core +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The code that computes thumbnail paths and generates thumbnails lives in a new app in openedx-core, outside of the learning package infrastructure. It exposes a generic API that takes a source :class:`Media` and the name of a variant (e.g. ``get_thumbnail(media, variant="files")``). The app does not define any variants of its own: it looks up the variant's dimensions in the setting from decision 4, where the platform and plugins define them. + +Because the API only accepts variant names, a caller cannot request arbitrary dimensions. A plugin that passed a value taken from a request directly to the API would get an error for any name not listed in that setting, instead of generating a new size. + +3. A thumbnail's storage path is a pure function of the source bytes and the resize parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Thumbnails are stored as plain files, under a path computed from the source :class:`Media` and the parameters used to generate them: + +``thumbnails/{media_file_namespace}/{source-media-hash}-{width}x{height}-{algorithm-version}.{ext}`` + +This mirrors the ``content/{media_file_namespace}/{hash_digest}`` path that :class:`Media` files already use (:ref:`openedx-content-adr-0012`, decision 6), under a separate ``thumbnails/`` prefix. + +There is no database model that maps a source image to its thumbnails. Given a source :class:`Media` and a size, the path is computed, and the thumbnail either already exists there or does not. + +- The **source hash** is the content hash that :class:`Media` already uses to store its own file, so the same image bytes always map to the same thumbnail, regardless of which :class:`Asset`, :class:`Component` or learning package they belong to. +- The **width and height** are the actual dimensions requested. Changing the configured size of a thumbnail therefore produces new files, instead of leaving stale ones under the same name. +- The **algorithm version** is bumped whenever the resizing code or its output format changes in a way that changes the result, so that existing thumbnails are never overwritten with different content. + +4. Thumbnail sizes are named, configured variants +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each consumer defines the sizes it needs as named variants (for example, one for the Files page, one for content library cards, and the two course card image sizes). Requests ask for a variant by name, and the variant's configured dimensions are used to compute the path from decision 3. + +This bounds how many thumbnails can exist for each image, and prevents a request from forcing the generation of an unbounded number of sizes. Because the API from decision 2 only accepts the variants listed in the setting described below, this limit does not depend on each consumer following this convention. + +Variants are defined in a single Django setting, read by the openedx-core app from decision 2. The platform provides the default variants in code, which each deployment can override, and plugins and other apps that use the API add their own to the same setting. Variants are not configurable at runtime (e.g. through the Django admin), because: + +- **Variant names are a contract with the frontend.** Each UI requests the variant it was built for, so adding or removing a variant only makes sense together with the code that uses it. +- **Dimensions are a UI decision.** The size of a thumbnail follows the layout that displays it, and should change together with that layout's code. +- **Changing a variant's dimensions is expensive.** It leaves every existing thumbnail of that variant unused (see Consequences) and causes all of them to be generated again. + +The name and format of the setting, and which properties a variant has beyond its dimensions (e.g. whether it is generated eagerly, see decision 5), are left to the implementation. + +5. Thumbnails are generated eagerly when possible, and lazily as a fallback +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Thumbnails for the variants configured to be generated eagerly are generated at the points where the legacy contentstore already generates them: when an author uploads an image, and when a course is imported. Other variants are only generated lazily, the first time they are requested. + +When a thumbnail is requested and its file does not exist yet, it is generated at that moment from the source :class:`Media`, stored at its path, and served. Because the path is a pure function of its inputs, two concurrent requests for the same missing thumbnail produce the same file at the same path. + +Because lazy generation can be triggered by anyone who can view the source image, images whose declared dimensions exceed a safe limit are not decoded, and are treated as images that fail to generate. + +The lazy fallback covers every case where no eager generation happened, without any special handling: courses migrated from contentstore, whose thumbnails are not migrated; reruns, copies and syncs into other learning packages; past versions of an :class:`Asset`; and new variants added after the image was uploaded. + +6. Thumbnails are requested through their source, not by storage path +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The storage path from decision 3 is an internal key, not a public URL. A thumbnail is requested through the same URL that already serves its source file, with the variant name added as a query parameter: + +- A shared :class:`Asset`, at its learning-package-level URL (:ref:`openedx-content-adr-0013`, decision 5), e.g. ``.../assets/published/images/fig1.png?thumbnail=files``. +- A file attached to a :class:`Component`, at the Component's file URL (:ref:`openedx-content-adr-0005`), e.g. ``.../{component_key}/published/static/fig1.png?thumbnail=library_card``. + +The view that serves the source file handles the request exactly as it would for the file itself, and only changes what it serves at the end: + +- **The source is identified by the URL.** Both shared Assets and files attached to Components are covered, including the fallback from a Component's file URL to a shared Asset (:ref:`openedx-content-adr-0013`, decision 4). +- **The version is chosen by the URL.** ``draft`` (used by Studio, including the Files page), ``published`` (used by the LMS), and ``?v=N`` for historical versions follow the same rules as for the file. The thumbnail is computed from the :class:`Media` of whichever version is resolved. +- **The same permission checks are applied.** For a shared Asset, its permissions mode (:ref:`openedx-content-adr-0013`, decision 9); for a file attached to a Component, the Component's permissions. A thumbnail is never more visible than its source: a user who cannot access an image cannot access any of its thumbnails. + +Once the source :class:`Media` is resolved and access is granted, the view calls the API from decision 2 with that :class:`Media` and the requested variant, and serves the resulting thumbnail file the same way it serves any other file. Placeholders and deleted Assets, which have no current :class:`Media`, return the same response as a request for the file itself. + +7. The contentstore compatibility shim delegates to this system +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +During the first phase of :ref:`openedx-content-adr-0016`, Studio keeps calling the legacy contentstore API, which a compatibility shim implements on top of Assets. That API includes thumbnail functions, which the shim implements with this system: + +- **Generating a thumbnail** (``generate_thumbnail()``, called on upload and import) triggers the eager generation from decision 5. +- **The thumbnail location** returned for each file (``thumbnail_location``, which the Files page uses to show the preview) is the URL of the source file with the ``files`` variant, as described in decision 6. +- **Deleting a thumbnail** (when a file is deleted) does nothing, because thumbnail files are never deleted (see Consequences). +- **Legacy thumbnail URLs** (``asset-v1:...+type@thumbnail+block@...``) are not :class:`Asset`\ s. The shim either maps them to the URL of their source file with the ``files`` variant, or returns a 404. + +This keeps image previews on the Files page of migrated courses without changes to the frontend. + +Consequences +------------ + +- No schema changes are required in ``openedx_content``, and no new database models are required anywhere: :class:`Asset` and :class:`AssetVersion` are used as defined in :ref:`openedx-content-adr-0013`. +- Thumbnails are deduplicated within a ``media_file_namespace``: the same image bytes used by different :class:`Asset`\ s, by files attached to :class:`Component`\ s, or by different learning packages (e.g. a course and its reruns) share the same thumbnail files. +- Copying content between learning packages (reruns, copy/paste, library sync) requires no thumbnail-specific handling. +- Migrating a course from contentstore requires no thumbnail-specific step: thumbnails are generated on first request. +- This system must be implemented in time for the first phase of :ref:`openedx-content-adr-0016`. Otherwise, the Files page of migrated courses would show no image previews. +- The thumbnail for a past version of an :class:`Asset` can be retrieved the same way as the current one, starting from that version's :class:`Media`. +- Supporting a new size only requires configuring a new variant. +- Thumbnails are not included in a course's export package, and are regenerated after import. This preserves the behavior of the legacy contentstore. +- Since there is no database record of which thumbnails exist, serving a thumbnail requires checking whether its file exists in storage. With remote storage (e.g. S3), this check adds latency to every thumbnail request. Eager generation (decision 5) means the file almost always exists, so a request rarely has to wait for a thumbnail to be generated, but it still pays for the check. If this becomes a problem, the check can be avoided later (e.g. by caching which thumbnails exist, or by redirecting to the expected URL) without changing this design. +- Thumbnail files are never deleted. When a configured size or the algorithm version changes, the previous files are left behind. ``openedx_content`` has no garbage-collection mechanism for :class:`Media` files either. +- There is no inventory of thumbnails: finding which thumbnails exist for a :class:`Media` requires listing the storage under its hash. No such inventory is needed for now, because nothing in this design has to enumerate thumbnails. +- Thumbnails are not covered by the per-learning-package storage accounting and cleanup that :class:`Media` rows support. A thumbnail is shared by every learning package that uses the same image bytes, so it cannot be attributed to a single one. Because every thumbnail path starts with ``thumbnails/{media_file_namespace}/``, thumbnails can still be measured or removed per organization by storage prefix, and any removed thumbnail is regenerated on its next request. + +Rejected Alternatives +--------------------- + +A platform-side model mapping each source image to its thumbnail +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An earlier version of this ADR proposed an ``ImageMediaThumbnail`` model, keyed on ``(image_media, variant)``, with a foreign key to the :class:`Media` row holding the thumbnail. This was rejected because: + +- Every input of a thumbnail (the source bytes and the resize parameters) is immutable, so a mutable row that is "repointed" when a thumbnail is regenerated adds state without adding information. +- :class:`Media` rows belong to a single learning package, so every rerun or copy into another learning package would need new rows, even though the files themselves are shared. + +Storing the thumbnail inside the source's versioned data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An :class:`Asset` holds exactly one file per version (:ref:`openedx-content-adr-0013`), so a thumbnail cannot be grouped with its source in the same :class:`AssetVersion`. Storing it as a separate version, or making versions mutable for derived files, was rejected because :class:`PublishableEntityVersion` is immutable by design: backup and restore, export/import and version history all depend on a version being a fixed snapshot. Adding a derived file must never create a new version of authored content. + +Keeping thumbnails in the legacy MongoDB-backed contentstore +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Leaving thumbnails behind in the legacy contentstore would keep a MongoDB dependency alive for one narrow piece of functionality, working against the retirement of contentstore planned in :ref:`openedx-content-adr-0016`. + +A dedicated endpoint for thumbnails +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An earlier version of this ADR proposed a separate platform endpoint for thumbnails (e.g. ``/api/contentstore/v2/.../thumbnail``). It was rejected because that endpoint would have to duplicate the logic that resolves a file's source, chooses its version and checks its permissions. + +Allowing arbitrary dimensions in thumbnail requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Letting callers request any width and height would be more flexible, but combined with lazy generation it would let a single client force the generation and storage of an unbounded number of files. + +Configuring variants at runtime through a model +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Variants could be stored in a model editable through the Django admin. This would allow changing sizes without a deployment. It was rejected because variant names and dimensions are tied to the frontend code that requests and displays them, and because a change made from the admin would silently cause every thumbnail of that variant to be generated again. + +Generating thumbnails only at upload and import time +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Generating thumbnails only eagerly would avoid any image processing during requests, but every case not covered by an upload or import (migrated courses, copies into other learning packages, new variants, past versions) would then need its own explicit generation step. The lazy fallback (decision 5) handles all of them with a single mechanism. + +Open Questions +-------------- + +- **Deduplicating thumbnails across the whole site.** Using a single storage location for all thumbnails, instead of one per ``media_file_namespace``, would deduplicate more, but would make it harder to remove all of an organization's files. This ADR starts with per-namespace storage. +- **Image metadata.** An ``ImageMedia`` model, joined 1:1 with :class:`Media` and holding metadata derived from the image bytes (e.g. dimensions), is not needed by this design, but could complement it (e.g. to avoid upscaling small images, or to show dimensions on the Files page). +- **Images that fail to generate.** If a :class:`Media` declares an ``image/*`` type but cannot be decoded, no thumbnail file is ever created, and every request tries to generate it again. A first version can accept this, since such images should be rare. If it becomes a problem, failures could be remembered for a while (e.g. a cache entry with an expiry, or a marker file in storage) so they are not retried on every request. +- **Name of the openedx-core app.** Decision 2 places the code in a new openedx-core app, but does not fix its name. +- **Thumbnails of XBlocks, not just images.** This ADR is scoped to thumbnails of image files. A related, but architecturally distinct, need is a visual preview of an XBlock component itself (e.g. a Problem or an HTML interactive), generated by rendering the component rather than resizing an existing image — useful for content library "card" views that cover any component type, not just ones with an attached image. That is out of scope here, but worth exploring as a future extension.