ADR for reusable "File components" and how course assets are referenced/served [FC-0138] - #822
bradenmacdonald wants to merge 12 commits into
Conversation
|
Thanks for the pull request, @bradenmacdonald! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
|
|
||
| An upload is technically most like a folder, and our preliminary thinking about this idea also used the terms "AssetSet" or "Folder" for what we are naming an "Upload" in this ADR. "Upload" is chosen because it matches the "Uploads" part of the "Files & Uploads" page in the Studio UI, implies a generally singular thing without ruling out multiple files, doesn't conflict with any other names used in the platform, and is hopefully fairly self-evident to users, unlike a technical term like "AssetSet". | ||
|
|
||
| 2. Uploads group related files |
There was a problem hiding this comment.
A wrinkle with this is that files nested in folders exist today in Files and Uploads. You can't create it through the UI, but you can with import/export, as well as with at least one plugin (something eduNEXT made maybe?).
There was a problem hiding this comment.
I'll mention that, but I'm not sure it's a big wrinkle? The existing "components have assets" functionality already allows "subfolders" since each asset path can have / as needed, right? i.e. ComponentVersionMedia.path allows / within the path, and I think we're even using that in library components, or for the OLX or something.
| Context | ||
| ------- | ||
|
|
||
| Today, a course's "Files & Uploads" live in the legacy MongoDB contentstore, as a flat, course-wide namespace of paths that OLX references as ``/static/{path}``. As a first step in migrating all content away from MongoDB, we need to define how to store such files in ``openedx_content`` (within a :class:`LearningPackage`). |
There was a problem hiding this comment.
FWIW, while there are references to "Files and Uploads" in code, the current, unthemed UX actually just says "Files". The "Uploads" part was removed years ago, apparently.
There was a problem hiding this comment.
Huh, I hadn't really noticed that either. Well, OK. I still kind of like the "Upload" name but I'm completely open to going with "File(s)(Set)" or anything else that someone wants to propose.
There was a problem hiding this comment.
Regarding the name, I've gone with "File component(s)" for now.
6fe58fa to
ee4a005
Compare
08aa7b9 to
0bbe03b
Compare
|
For those following this so far, I have made major revisions to everything, primarily renaming it from "Upload" to "File component" and adding a "Consequences" section. |
0ebd33f to
d5a8ae0
Compare
| Naturally, the ``FileComponentMetadata`` table will enforce that ``legacy_path`` is unique per learning package. | ||
|
|
||
| 2. File components can be referenced using new ``oex-asset:`` URL | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
There was a problem hiding this comment.
I just pushed this new ADR 14.
Decision 1 is the only part that is relevant to our short-term project of "move static assets into openedx_content and serve them using the contentstore API". The rest is trying to paint a picture of the long-term functionality we'll get when we also migrate courseware itself to openedx_content.
The reason I'm sketching this all out now is so we can decide if we want all this, or if we'll instead go with the dumb, simple approach that does the bare minimum for contentstore compatibility but drops versioning, tracking, linking, etc.
|
OK, I still have some TODOs and things to clean up, but I'd like to get more input before focusing on the fine details. This is ready for review, @ChrisChV @ormsbee @kdmccormick. CC @xitij2000 and @Kelketek on ADR 14 in case you have any thoughts on this after working on the XBlock Asset Picker UI. |
| When copying a Component with references to shared File components to a new Learning Package, each referenced File component is handled as follows: | ||
|
|
||
| - First, if the destination Learning Package already has a File component that is a downstream copy of the same upstream File component (see decision 6), that existing copy is reused, even if its ``component_code`` or content differs. The copy is not updated as a side effect of pasting, since that would change every other component that uses it; if the source uses a newer upstream version, the author can sync the File component as usual. Matching on upstream prevents repeated pastes or imports of the same library content from creating a new copy of the File component each time. | ||
| - Otherwise, if the destination Learning Package already has a File component with identical ``component_code`` and media asset file hash(es), it is reused and no File components need to be copied. Identical bytes don't prove that it's the same asset, but reusing a File component with the same code and content is harmless. |
There was a problem hiding this comment.
I'm a bit worried about this one. If this occurs, the File Component in the destination Learning Package might already be in use by another component. If an author updates the asset, it will be updated in both the existing component and the copied component. However, the author is unaware of this; as far as they are concerned, they are only changing the asset in the existing component.
This is an ADR for representing Course Files in openedx_content.
Builds on #812 and #818 (merged).
View the rendered ADRs here:
AI disclosure: Claude helped me with research, writing, and formatting the RST, but I wrote most of the content myself.