[Draft] edx-platform Architectural Roadmap

[kdmccormick]

About

This is a place to coordinate long-term technical work within the edx-platform repository for the benefit of:

• Developers of edx-platform core
• Developers of edx-platform extensions
• Site Operators

This work will be completed by a variety of means. It will include both volunteer contributions as well as funded contributions. Some items will be accomplished as part of other feature work, others by their own focused projects. The purpose of this roadmap is not to make the work happen, but rather align all edx-platform developers around a shared vision which can collectively accomplish over time.

Out-of-scope: We will not track tasks that are entirely outside of edx-platform. We will not track improvements for Authors or Learners here. The Platform Roadmap is the right place for those.

Where does this come from? Originally, this Google Doc. Going forward, the roadmap will be managed by edx-platform Maintainers with input from edx-platform Core Contributors.

What's the status? We are still building this out. Stay tuned.

Roadmap Epics

MAKE IT SMALLER (maintenance burden, build size)

• Complete the MFE conversion process for edx-platform #31620
• Extract non-Core apps into Hooks-based platform plugins #36184
• Enforce restrictions on new apps and new dependencies (prior work: no-new-apps ADR)
• Whittle down requirements list (continuing Braden's work on kernel.in)
• Migrate ModuleStore off of MongoDB platform-roadmap#3

MAKE IT FASTER (for learners, educators)

• Once we’ve switched over to Learning Core data models for courseware content access, investigate our memory/CPU usage and optimize to reduce the costs of running the platform.
• (There have got to be more items we can add here -- will page Dave when he's back from PTO :)

MAKE IT EASIER TO CONFIGURE (for devs, ops, admins)

• [DEPR]: Paver #34467
• [DEPR] Devstack public-engineering#247
• [DEPR]: Dockerfiles and Docker images public-engineering#263
• Let's talk about OEP-45 (Configuring & Operating Open edX) openedx-proposals#587
• Greenfield prod-ready Django settings without YAML #35146
• Refactor existing settings files without changing their behavior?
• Invert or remove Waffle flags that Tutor already enables
• Switch to Redis by default for caching.
• Support bare-metal edx-platform
• Simplify logging config (also see A draft for making a decision on managing logging configurations  #28603)

MAKE IT EASIER TO INTEGRATE (for "external" devs)

• Document HTTP API support levels, map them to audiences (Operator vs Instructor vs Learner), and map them to product features
• Define which Python APIs are supported
• Define best practices for how to build HTTP APIs (starting point)
• Clear documentation for how to integrate with external auth providers vs internal.
• Improve/document AuthZ including documenting the roles that exist in our platform.
• Either model the catalog in edx-platform or make edx-platform oblivious to it (starting point ADR)
• API Improvements #32609

MAKE IT MAKE SENSE (for "internal" devs)

• Consolidate to a single edx-platform docs structure.
• Repo Structure: Define the structure of edx-platform as it stands today, define the structure of the repository that we want (incl. LMS vs CMS concerns), make a plan to get there for existing code, document where new code should go, and use importlinter to help enforce all of this
• Type hint edx-platform's major packges, add more type hints to edx-platform, and then make type checking opt-out instead of opt-in
• Turn in-repo apps that are edx-django-plugins into normal Django apps.
• Resolve various Auth Challenges, including switching all HTTP APIs to use DRF with standard Auth classes
• Improve edx-platform docs and discoverability #35014
• Create ADR for avoiding use of plugin patterns where they are not needed #35898

MAKE MODULESTORE GO AWAY (various benefits)

• Libraries Relaunch - Component Reuse and Improved UI platform-roadmap#327
• [epic] edx-platform XBlock Extraction #34827
• XBlock 2.0
• Limit ModuleStore usage to just CMS and Courseware (implementing Dave's ADR)
• Limit Modulestore usage to just CMS
• Finish purging remnants of XModule system, including renaming to make the system more comprehensible
• Try to reimplement Course Blocks API on top of Learner Core (allowing removal of block transformers)
• Deprecate XBlock children above the Unit
• Deprecate XBlock children (period) -- requires replacing dynamic blocks with Learning Core systems
• Migrate courses from ModuleStore to Learning Core

MAKE IT MORE RESILIENT (needs work)

• Don't allow error pages to raise errors #35216

[feanil]

Notes from discussion at Axim:

• It would be useful to make sure we make dependencies clear when we flesh this out into tickets.
• If it's possible to put in any sizing info, that would also help. Are there any quick wins?

[kdmccormick]

Notes from edx-platform maintenance meeting:

• Benefit to add: Making it more resillient (for operators)
  • Example problems:
    • Migrations that are not atomic that cannot be reversed
    • course-discovery requests taking down the LMS
    • genreally, less likely to fall down for mysterious reasons
  • Will page Dave once he's back on PTO
• Project management?
  • Turn top-level groups into labels so we can prioritize the list
  • Sizing labels
  • Not a board, since this isn't a cohesive project