Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions source/site_ops/concepts/databases.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
.. _Open edX Databases:

Open edX Platform Database and Search Technologies
##################################################

.. tags:: site operator, concept

The Open edX platform uses several databases and search backends, each
serving different purposes.

Relational (SQL)
****************

**MySQL**
The long-standing primary relational database for most services (LMS,
CMS/Studio), storing users, courses, enrollments, and grades.

**PostgreSQL**
Newly supported as of the Verawood release, with migrations updated to
make the Open edX platform compatible across both MySQL and PostgreSQL.
Changes include dynamic SQL generation based on the database engine,
cross-database field type handling, and custom psycopg2 adapter
registration for Open edX's OpaqueKey types like ``CourseLocator``. A
community-developed Tutor plugin (`tutor-contrib-postgresql <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_)
supports PostgreSQL-based Tutor deployments. See :ref:`use postgresql`.

**SQLite**
Used in local development and testing environments.

NoSQL / Document
****************

**MongoDB**
Used to store course content structure in Studio. There is ongoing work to
migrate away from MongoDB toward MySQL via the Learning Core
(`openedx-learning <https://github.com/openedx/openedx-core>`_) initiative.

Caching / In-Memory
*******************

**Redis**
Used for caching, session storage, and Celery task queuing. Superseded
Memcached in modern deployments.

**Memcached**
Legacy caching layer, largely replaced by Redis.

Search
******

**Meilisearch**
The current primary search engine for content library and course
authoring search (introduced in Quince), used by the Authoring MFE for
course and library content discovery, as well as the Catalog MFE for course searching.

**Typesense**
A supported alternative search backend for site operators, attractive in
part because it supports raft-based clustering for high availability — a
limitation that has been noted with Meilisearch. See :ref:`Use Typesense search backend`.

.. note:: Typesense is currently (as of Verawood) only supported for LMS searching, _not_ Studio/libraries.

**Elasticsearch / OpenSearch**
Used for older LMS-level search (course catalog, user search). OpenSearch
is the current preferred variant, with ongoing community discussion about
whether to deprecate it in favor of lighter-weight alternatives.

Message Broker
**************

**Redis** (also serves this role) **or RabbitMQ**
Used as a Celery broker for background task processing.

.. seealso::

:ref:`Use Typesense search backend`

:ref:`use postgresql`

**Maintenance chart**

+--------------+-------------------------------+----------------+--------------------------------+
| Review Date | Working Group Reviewer | Release |Test situation |
+--------------+-------------------------------+----------------+--------------------------------+
| 2025-04-10 | sarina | Verawood | Pass |
+--------------+-------------------------------+----------------+--------------------------------+
1 change: 1 addition & 0 deletions source/site_ops/how-tos/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Site Operators: How-tos
.. tags:: site operator, how-to

.. toctree::
:maxdepth: 1
:glob:

*
Expand Down
137 changes: 137 additions & 0 deletions source/site_ops/how-tos/use_postgresql.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
.. _use postgresql:

Use PostgreSQL as the Relational Database Backend
#################################################

.. note::

PostgreSQL support was introduced in the Verawood release. The Tutor plugin
that enables this feature (`tutor-contrib-postgresql <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_)
is under active development. Migration from an existing MySQL installation is
not yet supported. See `Current PostgreSQL Limitations`_ for details.

PostgreSQL is a powerful open source relational database system with a strong
reputation for reliability, standards compliance, and extensibility. It is a
common choice for production web applications and is widely supported by
managed cloud database services such as Amazon RDS, Google Cloud SQL, and
Azure Database for PostgreSQL.

Historically, the Open edX platform used MySQL as its only supported relational
database. As of the Verawood release, PostgreSQL is also supported as a
relational database backend for the Open edX platform, thanks to community
contributions that updated the platform's Django migrations and field adapters
to work correctly across both database engines.

Prerequisites
*************

- A Tutor-based Open edX deployment (local or Kubernetes)
- Tutor Verawood or later
- A fresh Open edX installation (see `Current PostgreSQL Limitations`_)

Installation
************

PostgreSQL support is provided via the community-maintained Tutor plugin
``tutor-contrib-postgresql``, authored by `@qasimgulzar
<https://github.com/qasimgulzar>`_.

.. warning::

This plugin is under active development and has not yet reached a stable
release. Use in production environments is at your own risk. Review the
`plugin repository <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_
for the latest status before proceeding.

Step 1: Install the plugin
==========================

.. code-block:: bash

pip install git+https://github.com/qasimgulzar/tutor-contrib-postgresql

Step 2: Enable the plugin
=========================

.. code-block:: bash

tutor plugins enable postgresql

Step 3: Build the Open edX image
=================================

The Open edX image must be rebuilt to include the necessary PostgreSQL client
libraries and Python packages (``psycopg2``).

.. code-block:: bash

tutor images build openedx

Step 4: Launch the platform
============================

.. code-block:: bash

tutor local launch

This will initialize a fresh Open edX database in PostgreSQL and run all
Django migrations against it.

.. _Current PostgreSQL Limitations:

Current Limitations
*******************

Fresh installations only
========================

At this time, PostgreSQL support is available for fresh Open edX installations
only. There is no supported or documented migration path from an existing MySQL
database to PostgreSQL. Operators with an existing MySQL-backed deployment who
wish to switch to PostgreSQL will need to wait for migration tooling and
documentation to be developed.

If you are interested in contributing to or tracking the development of
MySQL-to-PostgreSQL migration support, see the `tutor-contrib-postgresql
repository <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_ and the
`original platform pull request
<https://github.com/openedx/openedx-platform/pull/35762>`_.

Known rough edges
=================

PostgreSQL support was introduced with the following known limitations that may
be addressed in subsequent releases.

- Opaque key adapter coverage for v2 content libraries (beta libraries in
Studio) may be incomplete. Runtime errors related to ``LearningContextKey``
or ``UsageKey`` serialization, if encountered, should be reported to the
plugin maintainer.
- The ``tutor-contrib-postgresql`` plugin does not yet have a stable release.
Install directly from the GitHub repository as shown above.

Getting Help
************

PostgreSQL support for the Open edX platform is a community-driven effort. If
you encounter issues or have questions:

- Open an issue on the `tutor-contrib-postgresql repository
<https://github.com/qasimgulzar/tutor-contrib-postgresql/issues>`_
- Post on the `Open edX discussion forum <https://discuss.openedx.org>`_


.. seealso::

:ref:`Open edX Databases`

`Tutor PostgreSQL plugin <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_


**Maintenance chart**

+--------------+-------------------------------+----------------+--------------------------------+
| Review Date | Working Group Reviewer | Release |Test situation |
+--------------+-------------------------------+----------------+--------------------------------+
| 2025-04-10 | sarina | Verawood | Pass |
+--------------+-------------------------------+----------------+--------------------------------+