Refactor to authoring package

.importlinter

@@ -23,25 +23,30 @@ name = Lib / Core / Contrib Layering
 type = layers
 layers=
     openedx_learning.contrib
-    openedx_learning.core
+    openedx_learning.apps
     openedx_learning.lib
 
-# This is layering within our Core apps. Every new Core app should be added to
+# This is layering within our Authoring apps. Every new app should be added to
 # this list when it it created.
 [importlinter:contract:core_apps_layering]
-name = Core App Dependency Layering
+name = Authoring App Dependency Layering
 type = layers
 layers=
+    # The public authoring API is at the top–none of the apps should call to it.
+    openedx_learning.api.authoring
+
     # The "components" app is responsible for storing versioned Components,
     # which is Open edX Studio terminology maps to things like individual
     # Problems, Videos, and blocks of HTML text. This is also the type we would
     # associate with a single "leaf" XBlock–one that is not a container type and
     # has no child elements.
-    openedx_learning.core.components
+    openedx_learning.apps.authoring.components
+
     # The "contents" app stores the simplest pieces of binary and text data,
     # without versioning information. These belong to a single Learning Package.
-    openedx_learning.core.contents
+    openedx_learning.apps.authoring.contents
+
     # The lowest layer is "publishing", which holds the basic primitives needed
     # to create Learning Packages and manage the draft and publish states for
     # various types of content.
-    openedx_learning.core.publishing
+    openedx_learning.apps.authoring.publishing

README.rst

@@ -1,16 +1,13 @@
-openedx-learning
-=============================
+Open edX Learning Core (and Tagging)
+====================================
 
 |pypi-badge| |ci-badge| |codecov-badge| |doc-badge| |pyversions-badge|
 |license-badge|
 
-This is experimentation/prototyping and not in any way production ready!
-------------------------------------------------------------------------
-
 Overview
 --------
 
-The Open edX Learning repository holds Django apps that represent core learning concepts and data models that have been extracted from edx-platform.
+The ``openedx_learning`` package holds Django apps that represent core learning concepts and data models that have been extracted from edx-platform. At the moment, this repo also contains the ``openedx_tagging`` package, but this will likely be moved out in the future.
 
 Motivation
 ----------
@@ -26,18 +23,16 @@ Parts
 ~~~~~
 
 * ``openedx_learning.lib`` is for shared utilities, and may include things like custom field types, plugin registration code, etc.
-* ``openedx_learning.core`` contains our Core Django apps, where foundational data structures and APIs will live.
+* ``openedx_learning.apps`` contains our Learning Core Django apps, where foundational data structures and APIs will live. The first of these is ``authoring``, which holds apps related to the editing and publishing of learning content.
 * ``openedx_tagging.core`` contains the core Tagging app, which provides data structures and apis for tagging Open edX objects.
 
-App Dependencies
-~~~~~~~~~~~~~~~~
+Learning Core Package Dependencies
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Anything can import from ``openedx_learning.lib``.
+Learning Core code should never import from ``edx-platform``.
 
-Core apps can import from each other, but cannot import from other apps outside of core. For those apps:
+We want to be very strict about dependency management internally as well. Please read the `.importlinter config file <.importlinter>`_ file and the `Python API Conventions ADR <docs/decisions/0016-python-public-api-conventions>`_ for more details.
 
-* ``learning_publishing`` has no dependencies. All the other apps depend on it.
-* ``learning_composition`` and ``learning_navigation`` both depend on ``learning_partitioning``
 
 Model Conventions
 ~~~~~~~~~~~~~~~~~
@@ -46,7 +41,7 @@ We have a few different identifier types in the schema, and we try to avoid ``_i
 
 * ``id`` is the auto-generated, internal row ID and primary key. This never changes. Data models should make foreign keys to this field, as per Django convention.
 * ``uuid`` is a randomly generated UUID4. This is the stable way to refer to a row/resource from an external service. This never changes. This is separate from ``id`` mostly because there are performance penalties when using UUIDs as primary keys with MySQL.
-* ``key`` is intended to be a case-sensitive, alphanumeric key, which holds some meaning to library clients. This is usually stable, but can be changed, depending on the business logic of the client. The apps in this repo should make no assumptions about it being stable. It can be used as a suffix.
+* ``key`` is intended to be a case-sensitive, alphanumeric key, which holds some meaning to library clients. This is usually stable, but can be changed, depending on the business logic of the client. The apps in this repo should make no assumptions about it being stable. It can be used as a suffix. Since ``key`` is a reserved name on certain database systems, the database field is ``_key``.
 * ``num`` is like ``key``, but for use when it's strictly numeric. It can also be used as a suffix.
 
 
@@ -61,7 +56,7 @@ The structure of this repo follows [OEP-0049](https://open-edx-proposals.readthe
 Code Overview
 -------------
 
-The ``openedx_learning.apps`` package contains all our Django applications. All apps are named with a ``learning_`` prefix to better avoid name conflicts, because Django's app namespace is flat. Apps will adhere to `OEP-0049: Django App Patterns <https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0049-django-app-patterns.html>`_.
+The ``openedx_learning.apps`` package contains all our Django applications.
 
 Development Workflow
 --------------------
@@ -75,7 +70,7 @@ One Time Setup
   cd openedx-learning
 
   # Set up a virtualenv using virtualenvwrapper with the same name as the repo and activate it
-  mkvirtualenv -p python3.8 openedx-learning
+  mkvirtualenv -p python3.11 openedx-learning
 
 
 Every time you develop something in this repo

docs/api_reference.rst

@@ -0,0 +1,8 @@
+API Reference
+=============
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   public_apis/*

docs/conf.py

@@ -511,6 +511,9 @@ def on_init(app):  # pylint: disable=unused-argument
 
     Read the Docs won't run tox or custom shell commands, so we need this to
     avoid checking in the generated reStructuredText files.
+
+    Note: there is no auto-building of API docs for the entire repo. We
+    explicitly build reference docs for the public API in /docs/public_apis/
     """
     docs_path = os.path.abspath(os.path.dirname(__file__))
     root_path = os.path.abspath(os.path.join(docs_path, '..'))
@@ -519,9 +522,6 @@ def on_init(app):  # pylint: disable=unused-argument
         # If we are, assemble the path manually
         bin_path = os.path.abspath(os.path.join(sys.prefix, 'bin'))
         apidoc_path = os.path.join(bin_path, apidoc_path)
-    check_call([apidoc_path, '-o', docs_path, os.path.join(root_path, 'openedx_learning'),
-                os.path.join(root_path, 'openedx_learning/migrations')])
-
 
 def setup(app):
     """

docs/decisions/0008-tagging-tree-data-arch.rst

@@ -27,7 +27,7 @@ All taxonomies are trees
 
 We could use a tree structure for all taxonomies: flat taxonomies would have only 1 level of tags under the root, while nested taxonomies can be deeper.
 
-To implement this, we'd link each taxonomy to a root tag, with the user-visible tags underneath. 
+To implement this, we'd link each taxonomy to a root tag, with the user-visible tags underneath.
 
 It was simpler instead to link the tag to the taxonomy, which removes the need for the unseen root tag.
 

docs/decisions/0009-tagging-administrators.rst

@@ -34,7 +34,7 @@ Rejected Alternatives
 Django users & groups
 ~~~~~~~~~~~~~~~~~~~~~
 
-This is a standard way to grant access in Django apps, but it is not used in Open edX. 
+This is a standard way to grant access in Django apps, but it is not used in Open edX.
 
 .. _get_organizations: https://github.com/openedx/edx-platform/blob/4dc35c73ffa6d6a1dcb6e9ea1baa5bed40721125/cms/djangoapps/contentstore/views/course.py#L1958
 .. _CourseCreator: https://github.com/openedx/edx-platform/blob/4dc35c73ffa6d6a1dcb6e9ea1baa5bed40721125/cms/djangoapps/course_creators/models.py#L27

docs/decisions/0010-taxonomy-enable-context.rst

@@ -31,7 +31,8 @@ When is a taxonomy field shown to course authors in a given course?
 | False       | False                       | True                     | False                         |
 +-------------+-----------------------------+--------------------------+-------------------------------+
 
-.. _Course
+.. _Course:
+
 Course
 ~~~~~~
 
@@ -43,7 +44,8 @@ Disabling taxonomies for a course will remove/hide the taxonomy fields from the
 
 Future versions may add more granularity to these settings, to be determined by user needs.
 
-.. _Instance
+.. _Instance:
+
 Instance
 ~~~~~~~~
 
@@ -52,7 +54,8 @@ Taxonomy contains a boolean ``enabled`` field.
 A Taxonomy can be disabled for all contexts by setting ``enabled = False``.
 If ``enabled = True``, then the :ref:`Organization` and :ref:`Course` contexts determine whether a taxonomy will be shown to course authors.
 
-.. _Organization
+.. _Organization:
+
 Organization
 ~~~~~~~~~~~~
 

docs/decisions/0012-system-taxonomy-creation.rst

@@ -46,15 +46,15 @@ In this way, the system-defined taxonomies are fully dynamic and can represent
 tags based on Languages, Users, or Organizations that may exist in large numbers
 or be constantly created.
 
-At present, there isn't a good way to *list* all of the [potential] tags that
+At present, there isn't a good way to *list* all of the potential tags that
 exist in a system-defined Taxonomy. We may add an API for that in the future,
 for example to list all of the available languages. However for other cases like
 users it doesn't make sense to even try to list all of the available tags. So
 for now, the assumption is that the UI will not even try to display a list of
 available tags for system-defined taxonomies. After all, system-defined tags are
 usually applied automatically, rather than a user manually selecting from a
 list. If there is a need to show a list of tags to the user, use the API that
-lists the actually applied tags - i.e. the values of the ``ObjectTag``s
+lists the actually applied tags - i.e. the values of the ``ObjectTags``
 currently applied to objects using the taxonomy.
 
 Tags hard-coded by fixtures/migrations

docs/decisions/0014-single-taxonomy-view-api.rst

@@ -135,12 +135,12 @@ Return a simple list of tags, regardless of whether it is root or leaf.
 Add the children to the root pagination
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Ex. If the ``page_size`` is 100, when fetching the first root tag, which has 10 children tags, 
+Ex. If the ``page_size`` is 100, when fetching the first root tag, which has 10 children tags,
 11 tags are counted for the total and there would be reamin 89 tags to be obtained.
 
 **Cons:**
 
-- If there is a branch with a number of tags that exceeds ``page_size``, 
+- If there is a branch with a number of tags that exceeds ``page_size``,
   it would only return that branch.
 - All branches are variable in size, therefore a variable number of root tags
   would be returned. This would cause interfaces between taxonomies to be inconsistent
@@ -150,7 +150,7 @@ Ex. If the ``page_size`` is 100, when fetching the first root tag, which has 10
 Search on frontend
 ~~~~~~~~~~~~~~~~~~
 
-We constrain the number of tags allowed in a taxonomy for MVP, so that the API 
+We constrain the number of tags allowed in a taxonomy for MVP, so that the API
 can return all the tags in one page. So we can perform the tag search on the frontend.
 
 **Cons:**

docs/index.rst

@@ -3,10 +3,10 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-openedx-learning
-====================
+Open edX Learning Core
+======================
 
-An experiment.
+The boring, foundational bits of a learning platform that are hard to get right at scale. Currently being developed for content storage.
 
 Contents:
 
@@ -17,11 +17,10 @@ Contents:
    getting_started
    testing
    internationalization
-   modules
    changelog
+   api_reference
    decisions
 
-
 Indices and tables
 ==================
 

docs/public_apis/authoring_api.rst

@@ -0,0 +1,7 @@
+Authoring API
+=============
+
+.. automodule:: openedx_learning.api.authoring
+   :members:
+   :imported-members:
+   :show-inheritance:

docs/public_apis/authoring_models.rst

@@ -0,0 +1,7 @@
+Authoring Models
+================
+
+.. automodule:: openedx_learning.api.authoring_models
+   :members:
+   :imported-members:
+   :show-inheritance:

olx_importer/management/commands/load_components.py

@@ -28,9 +28,9 @@
 from django.db import transaction
 
 # Model references to remove
-from openedx_learning.core.components import api as components_api
-from openedx_learning.core.contents import api as contents_api
-from openedx_learning.core.publishing import api as publishing_api
+from openedx_learning.apps.authoring.components import api as components_api
+from openedx_learning.apps.authoring.contents import api as contents_api
+from openedx_learning.apps.authoring.publishing import api as publishing_api
 
 SUPPORTED_TYPES = ["problem", "video", "html"]
 logger = logging.getLogger(__name__)

openedx_learning/__init__.py

@@ -1,4 +1,4 @@
 """
 Open edX Learning ("Learning Core").
 """
-__version__ = "0.9.4"
+__version__ = "0.10.0"

openedx_learning/api/authoring.py

@@ -0,0 +1,14 @@
+"""
+This is the public API for content authoring in Learning Core.
+
+This is the single ``api`` module that code outside of the
+``openedx_learning.apps.authoring.*`` package should import from. It will
+re-export the public functions from all api.py modules of all authoring apps. It
+may also implement its own convenience APIs that wrap calls to multiple app
+APIs.
+"""
+# These wildcard imports are okay because these api modules declare __all__.
+# pylint: disable=wildcard-import
+from ..apps.authoring.components.api import *
+from ..apps.authoring.contents.api import *
+from ..apps.authoring.publishing.api import *

openedx_learning/api/authoring_models.py

@@ -0,0 +1,13 @@
+"""
+This is where we expose a small handful of models and model mixins that we want
+callers to extend or make foreign keys to. Callers importing this module should
+never instantiate any of the models themselves–there are API functions in
+authoring.py to create and modify data models in a way that keeps those models
+consistent.
+"""
+# These wildcard imports are okay because these modules declare __all__.
+# pylint: disable=wildcard-import
+from ..apps.authoring.components.models import *
+from ..apps.authoring.contents.models import *
+from ..apps.authoring.publishing.model_mixins import *
+from ..apps.authoring.publishing.models import *

openedx_learning/core/components/api.py → openedx_learning/apps/authoring/components/api.py

@@ -21,6 +21,24 @@
 from ..publishing import api as publishing_api
 from .models import Component, ComponentType, ComponentVersion, ComponentVersionContent
 
+# The public API that will be re-exported by openedx_learning.apps.authoring.api
+# is listed in the __all__ entries below. Internal helper functions that are
+# private to this module should start with an underscore. If a function does not
+# start with an underscore AND it is not in __all__, that function is considered
+# to be callable only by other apps in the authoring package.
+__all__ = [
+    "get_or_create_component_type",
+    "create_component",
+    "create_component_version",
+    "create_next_version",
+    "create_component_and_version",
+    "get_component",
+    "get_component_by_key",
+    "component_exists_by_key",
+    "get_components",
+    "create_component_version_content",
+]
+
 
 def get_or_create_component_type(namespace: str, name: str) -> ComponentType:
     """
@@ -308,6 +326,9 @@ def look_up_component_version_content(
 
     Can raise a django.core.exceptions.ObjectDoesNotExist error if there is no
     matching ComponentVersionContent.
+
+    This API call was only used in our proof-of-concept assets media server, and
+    I don't know if we wantto make it a part of the public interface.
     """
     queries = (
         Q(component_version__component__learning_package__key=learning_package_key)

openedx_learning/core/components/apps.py → openedx_learning/apps/authoring/components/apps.py

@@ -9,8 +9,8 @@ class ComponentsConfig(AppConfig):
     Configuration for the Components Django application.
     """
 
-    name = "openedx_learning.core.components"
-    verbose_name = "Learning Core: Components"
+    name = "openedx_learning.apps.authoring.components"
+    verbose_name = "Learning Core > Authoring > Components"
     default_auto_field = "django.db.models.BigAutoField"
     label = "oel_components"
 

openedx_learning/core/components/models.py → openedx_learning/apps/authoring/components/models.py

@@ -19,12 +19,19 @@
 
 from django.db import models
 
-from ...lib.fields import case_sensitive_char_field, immutable_uuid_field, key_field
-from ...lib.managers import WithRelationsManager
+from ....lib.fields import case_sensitive_char_field, immutable_uuid_field, key_field
+from ....lib.managers import WithRelationsManager
 from ..contents.models import Content
 from ..publishing.model_mixins import PublishableEntityMixin, PublishableEntityVersionMixin
 from ..publishing.models import LearningPackage
 
+__all__ = [
+    "ComponentType",
+    "Component",
+    "ComponentVersion",
+    "ComponentVersionContent",
+]
+
 
 class ComponentType(models.Model):
     """