CourseContent model - associates a CourseRun with a LearningPackage [FC-0138]

.importlinter

@@ -58,14 +58,19 @@ layers=
     # The "backup_restore" applet handles the new export and import mechanism.
     openedx_content.applets.backup_restore
 
+    # The "courses" applet maps course runs to the Learning Packages that hold
+    # their content. It sits above the content applets so that it may use any of them.
+    openedx_content.applets.courses
+
     # The "components" applet is responsible for storing versioned Components,
     # which in Open edX Studio terminology maps to things like individual
     # Problems, Videos, and blocks of HTML text. This is also the type we would
     # associate with a single "leaf" XBlock–one that is not a container type and
     # has no child elements.
     # The "containers" app is built on top of publishing, and is a peer to
     # "components" but they do not depend on each other.
-    openedx_content.applets.components | openedx_content.applets.containers
+    # The "assets" app is a peer to "components".
+    openedx_content.applets.components | openedx_content.applets.containers | openedx_content.applets.assets
 
     # The "media" applet stores the simplest pieces of binary and text data,
     # without versioning information. These belong to a single Learning Package.

docs/openedx_content/decisions/0011-do-not-implement-copy-on-write.rst

@@ -6,7 +6,7 @@
 Status
 ------
 
-Proposed.
+Accepted.
 
 Context
 -------

docs/openedx_content/decisions/0012-course-learning-packages.rst

@@ -6,7 +6,7 @@
 Status
 ------
 
-Proposed.
+Accepted.
 
 Context
 -------

docs/openedx_content/decisions/0013-shared-static-assets.rst

@@ -6,7 +6,7 @@
 Status
 ------
 
-Proposed.
+Accepted.
 
 Context
 -------
@@ -34,14 +34,14 @@ Because Assets cannot hold more than one file, when several course files have so
 * By attaching all the related files to the same XBlock Component; or
 * By organizing the related Assets (one file per Asset) into a ``Collection``
 
-2. Assets are uniquely keyed by filename
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+2. Assets are uniquely keyed by filename (path)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Each Asset holds only a single file, e.g. ``solar-system.svg``, and within a given learning package (i.e. within a given course or library), each Asset's filename must be unique. (This is in contrast to asset files attached to XBlock Components, which allow multiple asset files per Component.) The filename is stored directly as a column on the ``Asset`` model, not on ``AssetVersion`` nor the ``Media`` row that any given ``AssetVersion`` points to.
+Each Asset holds only a single file, e.g. ``solar-system.svg``, and within a given learning package (i.e. within a given course or library), each Asset's filename must be unique. (This is in contrast to asset files attached to XBlock Components, which allow multiple asset files per Component.) The filename is stored directly as a ``path`` column on the ``Asset`` model, not on ``AssetVersion`` nor the ``Media`` row that any given ``AssetVersion`` points to.
 
-Filenames should be case-sensitive, such that ``fig1.png`` and ``Fig1.png`` are both allowed. Since MySQL's default collation is case-insensitive, the ``filename`` column must explicitly use a case-sensitive collation (e.g. using ``case_sensitive_char_field``, as ``ref_field`` does), including for its uniqueness constraint. For backwards compatibility, the filename is also allowed to contain ``/``, such as ``images/image1.png``. By convention, an Asset's ``entity_ref`` is ``openedx-asset:{filename}``, but nothing (including asset URLs) should rely on that; look Assets up by ``filename`` using its explicit column.
+Filenames should be case-sensitive, such that ``fig1.png`` and ``Fig1.png`` are both allowed. Since MySQL's default collation is case-insensitive, the ``path`` column which stores it must explicitly use a case-sensitive collation (e.g. using ``case_sensitive_char_field``, as ``ref_field`` does), including for its uniqueness constraint. For backwards compatibility, the filename/path is also allowed to contain ``/``, such as ``images/image1.png``. By convention, an Asset's ``entity_ref`` is ``openedx-asset:{path}``, but nothing (including asset URLs) should rely on that; look Assets up by ``path`` using its explicit column.
 
-If a file is uploaded with the same filename as an Asset that has been (soft) deleted, the deleted Asset is un-deleted: the upload becomes a new draft version of the existing Asset, rather than creating a new Asset. Its version history and any existing references to it are retained. If a specific permissions mode (``locked`` or ``private``, and separately ``allow_scripts`` or not) was specified when uploading, it will have that value; otherwise it will take on the most restrictive values between the defaults and the previous values (i.e. if the previously-deleted asset was locked, the new asset will be locked; the ``allow_scripts`` flag will be reset to ``False``).
+If a file is uploaded with the same filename/path as an Asset that has been (soft) deleted, the deleted Asset is un-deleted: the upload becomes a new draft version of the existing Asset, rather than creating a new Asset. Its version history and any existing references to it are retained. If a specific permissions mode (``locked`` or ``staff``, and separately ``allow_scripts`` or not) was specified when uploading, it will have that value; otherwise it will take on the most restrictive values between the defaults and the previous values (i.e. if the previously-deleted asset was locked, the new asset will be locked; the ``allow_scripts`` flag will be reset to ``False``).
 
 3. Assets support relative links
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -104,7 +104,7 @@ When migrating assets from MongoDB/contentstore to ``openedx_content``, each ass
 
 * **Thumbnails** generated by contentstore are dropped and not migrated. Thumbnails will be handled separately, as described in an upcoming ADR.
 * **Video transcripts** stored in contentstore (e.g. ``subs_*.srt.sjson``) become Assets like any other file. Video blocks reference transcripts by name rather than via ``/static/`` URLs, and during this migration they are still in modulestore with no :class:`Component`, so no dependency tracking is set up for transcript Assets at this stage. Dependencies between video components and their transcript Assets can be added when course content itself is migrated to ``openedx_content``.
-* **python_lib.zip** automatically gets ``private`` mode (decision 9), so that it remains inaccessible to learners as it is today.
+* **python_lib.zip** automatically gets ``staff`` mode (decision 9), so that it remains inaccessible to learners as it is today.
 * **HTML files** get ``allow_scripts=True`` (decision 10) if the course currently relies on them being rendered with scripts enabled, so that they keep working.
 
 Each course's Files are migrated in a single step, which also creates the run's learning package and ``CourseContent`` row (:ref:`openedx-content-adr-0012`, decision 3). A course is never in a mixed state: before the migration, all of its Files are in contentstore, and afterwards, contentstore is no longer consulted for that course.
@@ -114,25 +114,25 @@ Each course's Files are migrated in a single step, which also creates the run's
 
 All Assets (in fact, all PublishableEntities) have a mutable ``title`` which can be used to store a human-readable name for the entity, such as "Illustration of the Moon's Orbit (SVG)". The title is versioned (it's defined by :class:`PublishableEntityVersion`), so changing the title creates a new version.
 
-The title of an asset can be changed at any time, but the filename cannot be changed. (Changing the filename would break references/links to the file from the course content.)
+The title of an asset can be changed at any time, but the filename/path cannot be changed. (Changing the filename would break references/links to the file from the course content.)
 
 8. Assets are not part of the learner-facing outline
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Assets are not children of the course container or any of its descendants. Putting them there would mean every asset upload creates a new version of an outline container.
 
-9. Asset permissions mode: "public", "locked", "private"
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+9. Asset permissions mode: "public", "locked", "staff"
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The ``Asset`` model will have a permissions column with an enum that allows values of ``public`` (default), ``locked``, and ``private``. The permissions mode is on the unversioned model (not part of versioning), because often authors will wish to lock down all versions of an asset, not just lock the current version while still allowing access to previous versions.
+The ``Asset`` model will have a permissions column with an enum that allows values of ``public`` (default), ``locked``, and ``staff``. The permissions mode is on the unversioned model (not part of versioning), because often authors will wish to lock down all versions of an asset, not just lock the current version while still allowing access to previous versions.
 
 * ``public`` is the default for now, as it was in contentstore. There are no permissions checks on such assets, and anyone who knows the filename/URL can access it. This was chosen as the default more for performance reasons (skipping expensive permissions checks) rather than product reasons, and we may change to ``locked`` as default in the future.
 * ``locked`` mirrors the existing contentstore flag for course files. It means that only enrolled users can access the given file.
-* ``private`` means that only course staff can access the asset, e.g. for storing grading rubric PDFs. This is a new feature compared to contentstore, although it will be used to prevent learners from downloading ``python_lib.zip``, a special shared asset used by the problem XBlock that must not be accessible to learners (contentstore achieves this with a hard-coded rule).
+* ``staff`` means that only course staff can access the asset, e.g. for storing grading rubric PDFs. This is a new feature compared to contentstore, although it will be used to prevent learners from downloading ``python_lib.zip``, a special shared asset used by the problem XBlock that must not be accessible to learners (contentstore achieves this with a hard-coded rule).
 
 In **libraries**, these permissions checks have little effect, but indicate what permissions an asset should have when copied into the course.
 
-This information must be preserved round-trip on course exports/import. The attributes like ``locked``, ``private``, ``allow_scripts``, and ``title`` need to go into ``policies/assets.json``. Private shared assets other than ``python_lib.zip`` will have to be exported to a separate folder outside of ``static/`` and will be silently ignored if imported into an older version of Open edX that doesn't support private assets.
+This information must be preserved round-trip on course exports/import. The attributes like ``locked``, ``staff``, ``allow_scripts``, and ``title`` need to go into ``policies/assets.json``. Staff-only shared assets other than ``python_lib.zip`` will have to be exported to a separate folder outside of ``static/`` and will be silently ignored if imported into an older version of Open edX that doesn't support private ``staff`` assets.
 
 10. HTML Assets can be allowed to run scripts
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -156,9 +156,9 @@ Consequences
 
 **Replaced files are not deleted**, and old versions of the replaced file will still exist; this follows from the fact that the files are now versioned.
 
-**Assets cannot be renamed.** Because the filename *is* the identifier, new files can be uploaded, but renaming an Asset would break any existing usages that refer to it by its filename identifier.
+**Assets cannot be renamed.** Because the filename/path *is* the identifier, new files can be uploaded, but renaming an Asset would break any existing usages that refer to it by its filename identifier.
 
-**There will be two different ways to use files in course content**: by attaching them directly to XBlock Components, or by uploading them as shared assets. Content libraries already support the former (attached to Components), and courses support the latter. Note that the ``locked`` and ``private`` modes will only be supported for shared assets, as any files attached to a ``Component`` that are meant to be accessible to learners will share the same permissions as the ``Component`` they're attached to.
+**There will be two different ways to use files in course content**: by attaching them directly to XBlock Components, or by uploading them as shared assets. Content libraries already support the former (attached to Components), and courses support the latter. Note that the ``locked`` and ``staff`` modes will only be supported for shared assets, as any files attached to a ``Component`` that are meant to be accessible to learners will share the same permissions as the ``Component`` they're attached to.
 
 **Linking library components into courses will be simplified**, once we support XBlocks with files attached to their ``Component``, because part of the complexity in copying library components into courses involves analyzing their attached files and merging them into the course's shared Course Files. If we can instead just copy the ``Component``, including all its attached files, directly into the course's Learning Package, no analysis nor merging into shared files is necessary.
 

docs/openedx_content/decisions/0014-referencing-assets.rst

@@ -6,7 +6,7 @@
 Status
 ------
 
-Proposed.
+Accepted, but not implemented.
 
 Context
 -------
@@ -90,7 +90,7 @@ But, when an XBlock Component with references to shared assets is pasted/copied
 Each referenced Asset is handled as follows:
 
 1. If the destination Learning Package already has an Asset that is a downstream copy of the same upstream Asset (because an author manually imported/copied it to the destination earlier), that existing copy is reused.
-2. Otherwise, if the shared Asset is ``private`` or ``allow_scripts`` is True, an error occurs (private assets should not be implicitly copied, and component-local assets cannot be private nor allow scripts).
+2. Otherwise, if the shared Asset is ``staff`` or ``allow_scripts`` is True, an error occurs (staff-only assets should not be implicitly copied, and component-local assets cannot be staff-only nor allow scripts).
 3. Otherwise, the shared Asset gets converted to a component-local asset. (Note: this treats both ``locked`` and ``public`` assets the same, converting them to component-local and changing their permissions to match the destination component.)
 
 For example: In a library is a shared asset (an image, ``example.jpg``) and an HTML Component which references that image (``<html>...<img src="/static/example.jpg">...</html>``). When the HTML Component is copied from the library into a course (creating an upstream link), the ``example.jpg`` asset will be converted to a component-local asset attached to the HTML Component in the course, as ``/static/example.jpg``. Importantly, if the image asset (``example.jpg``) in the library is later edited and the change is published, the system must indicate that there is a newer version of the HTML Component available to sync to the course (even if the HTML component is unchanged, just because it has a dependency on the shared image asset). This will require updating the library-to-course sync code to account for ``dependencies_hash_digest`` and not just compare the ``version_num`` of each component.

docs/openedx_content/decisions/0015-shadow-components.rst

@@ -6,7 +6,7 @@
 Status
 ------
 
-Proposed.
+Accepted, but not implemented.
 
 Context
 -------

src/openedx_content/admin.py

@@ -3,9 +3,11 @@
 """
 # pylint: disable=wildcard-import
 
+from .applets.assets.admin import *
 from .applets.backup_restore.admin import *
 from .applets.collections.admin import *
 from .applets.components.admin import *
 from .applets.containers.admin import *
+from .applets.courses.admin import *
 from .applets.media.admin import *
 from .applets.publishing.admin import *

src/openedx_content/api.py

@@ -17,10 +17,12 @@
 #     `DraftChangeLogEventData` vs `DraftChangeLogRecord` is clearer if the former is `signals.DraftChangeLogEventData`)
 from . import signals
 # The rest of the public API (other than models):
+from .applets.assets.api import *
 from .applets.backup_restore.api import *
 from .applets.collections.api import *
 from .applets.components.api import *
 from .applets.containers.api import *
+from .applets.courses.api import *
 from .applets.media.api import *
 from .applets.publishing.api import *
 from .applets.sections.api import *

src/openedx_content/applets/assets/admin.py

@@ -0,0 +1,58 @@
+"""
+Django admin for ``assets`` models
+"""
+
+from django.contrib import admin
+from django.template.defaultfilters import filesizeformat
+
+from openedx_django_lib.admin_utils import ReadOnlyModelAdmin
+
+from .models import Asset, AssetVersion
+
+
+@admin.register(Asset)
+class AssetAdmin(ReadOnlyModelAdmin):
+    """
+    Django admin for Asset model
+    """
+
+    list_display = ("id", "path", "learning_package", "permissions", "allow_scripts", "created", "created_by")
+    list_filter = ("learning_package",)
+    search_fields = ("path",)
+
+    def get_queryset(self, request):
+        queryset = super().get_queryset(request)
+        return queryset.select_related(
+            "learning_package",
+            "publishable_entity",
+            "publishable_entity__created_by",
+        )
+
+
+@admin.register(AssetVersion)
+class AssetVersionAdmin(ReadOnlyModelAdmin):
+    """
+    Django admin for AssetVersion model
+    """
+
+    list_display = ("asset", "version_num", "title", "media_type", "format_size", "created", "created_by")
+    list_filter = ("asset__learning_package",)
+    search_fields = ("asset__path", "publishable_entity_version__title")
+
+    def get_queryset(self, request):
+        queryset = super().get_queryset(request)
+        return queryset.select_related(
+            "asset",
+            "publishable_entity_version",
+            "publishable_entity_version__created_by",
+            "media",
+            "media__media_type",
+        )
+
+    @admin.display(description="Media type")
+    def media_type(self, av_obj: AssetVersion) -> str:
+        return str(av_obj.media.media_type)
+
+    @admin.display(description="Size")
+    def format_size(self, av_obj: AssetVersion) -> str:
+        return filesizeformat(av_obj.media.size)