[FC-0118] docs: ADR for normalizing nested json apis - #38305
Conversation
openedx-webhooks
added
the
open-source-contribution
|
Thanks for the pull request, @taimoor-ahmed-1! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
taimoor-ahmed-1
changed the title
| GET /api/courses/v1/blocks/<usage_id>/?depth=1&requested_fields=id,type,display_name | ||
| GET /api/course_structure/v1/?view=minimal | ||
|
|
||
| **Response shape (minimal vs full):** | ||
|
|
||
| .. code-block:: json | ||
|
|
||
| // minimal (?view=minimal or ?fields=id,type,display_name,children) |
There was a problem hiding this comment.
Above you have depth and requested_fields as the params, but here you have fields=...? Both seem fine but we should be consistent.
| **Prefer IDs + follow-up over embedding:** | ||
|
|
||
| .. code-block:: text | ||
|
|
||
| GET /api/courses/v1/blocks/ → returns block IDs and types | ||
| GET /api/courses/v1/blocks/<id>/ → returns full block when needed |
There was a problem hiding this comment.
This is good for programmatic used but can be really costly for MFEs for example where they want to be able to request everything they need with a minimal number of requests. Is there some nuance we can add here about when it would be okay to use the full view?
There was a problem hiding this comment.
Added an exception clause
|
|
||
| * Cons / Costs | ||
|
|
||
| * Must maintain multiple representations; requires careful schema/versioning discipline. |
There was a problem hiding this comment.
Can you elaborate on this? Why would we maintain multiple representations?
There was a problem hiding this comment.
There will be separate serializer code paths, separate tests, separate OpenAPI schema entries for each variant, all of which must stay in sync as the data model evolves.
| References | ||
| ========== | ||
|
|
||
| * “Hard-to-Parse Deeply Nested JSON” recommendation in the Open edX REST API standardization notes. |
7792d0f to
5b316e2
Compare
Abdul-Muqadim-Arbisoft
left a comment
Abdul-Muqadim-Arbisoft
left a comment
There was a problem hiding this comment.
@taimoor-ahmed-1 this ADR should explicitly reference the pagination ADR (#38300). @bradenmacdonald already raised the tree-pagination question over there and even floated combining the two. His comment outlines two patterns: either return the full structural representation (IDs, types, parent/child relationships) and paginate node content separately, or return the tree to a fixed maximum depth with explicit child-fetch URLs for subtrees beyond that depth.
Since we are moving with keeping the ADRs separate, we need to clarify here how ?fields=... and minimal views interact with paginated lists at tree-shaped endpoints? For example, if a node has 1000+ children and the client requests them via field selection, do those children paginate, get capped at a max depth, or get returned as a flat ID list that's paginated separately? Without an explicit answer in one of the two ADRs, I think we'll end up giving conflicting guidance for the same endpoints, Course Blocks being the obvious one.
For reference: #38300
|
Waiting for the suggestion @Abdul-Muqadim-Arbisoft suggested, then will re-review this. |
@Abdul-Muqadim-Arbisoft I have made changes to the ADR, can you please review? |
|
@taimoor-ahmed-1 can you update the commits to pass commit lint, then I think this is good to merge. |
2e2d8f1 to
f1919a7
Compare
9a23ecc
into
openedx:docs/ADRs-axim_api_improvements
* docs: add ADR for standardizing serializer usage (#38139) * docs: explicitly mention API versioning incase of backwards incompatible change (#38188) Co-authored-by: Muhammad Faraz Maqsood <faraz.maqsood@A006-01130.local> * docs: add ADR for standardizing permissions usage (#38187) Currently, authorization logic is implemented inconsistently across views, serializers, and custom access checks. This ADR will define a consistent approach using DRF permission classes, object-level permissions, and queryset scoping where appropriate. Co-authored-by: Taimoor Ahmed <taimoor.ahmed@A006-01711.local> * docs: minor change in ADR language * fix: add s back * docs: migrate restful & legacy django api endpoints to standard drf viewsets (#38191) * docs: Add ADR for ensuring GET requests are idempotent Add edx-platform/docs/decisions/0030-ensure-get-requests-are-idempotent.rst as an accepted ADR. Define policy that GET endpoints must be strictly read-only, with side effects moved to explicit write endpoints or async event pipelines. Include edx-platform relevance, anti-pattern vs preferred code examples, and rollout guidance for testing and migration. * docs: add ADR for standardizing API documentation and schema coverage - Propose adoption of drf-spectacular across Open edX services - Require @extend_schema decorators for all API endpoints - Document request/response schemas, status codes, and error conditions * docs: remove incorrect ADR number * docs: address api-doc-tools deprecation in ADR per review feedback - Add context explaining what api-doc-tools is and its relationship to drf-yasg - Document deprecation and archival of api-doc-tools as a consequence - Add migration guide mapping api-doc-tools decorators and URL helpers to their drf-spectacular equivalents - Add rejected alternative for updating api-doc-tools internals - Add rollout step for final archival cutover Closes review comment by @feanil * docs: expand ADR-0027 with api-doc-tools deprecation and drf-yasg incompatibility analysis Address review feedback on FC-0118 ADR 0027: - Add context paragraph explaining what api-doc-tools is (drf-yasg shim, decorators it provides, schema view, OpenAPI 2.0 output) - Document deprecation of api-doc-tools and drf-yasg as a consequence, including transition-window behavior - Add detailed 8-point incompatibility analysis explaining why drf-yasg cannot be replaced with drf-spectacular inside api-doc-tools (recorded in the ADR itself for future reference) - Add migration plan for existing api-doc-tools consumers with concrete decorator/import/setting mapping - Update Rollout Plan to track api-doc-tools removal - Add references to drf-spectacular migration guide, drf-yasg upstream status, and api-doc-tools repository * chore: fix edx-mantained to edX-platform * docs: add ADR for standardizing pagination across APIs (#38300) * docs: add ADR for api versioning strategy (#38304) * docs: add ADR for standardizing filtering/sorting parameters (#38303) * docs: add ADR-0029 standardized error responses decision (#38246) * docs: add ADR for merging similar endpoints (#38262) * docs: ADR for normalizing nested json apis (#38305) * docs: add separate example for input & output serializers * docs: ADR for documenting and consolidating internal MFE APIs (#38309) * docs: ADR for documenting and consolidating internal MFE APIs Define a plan to document all undocumented internal LMS APIs consumed by MFEs into stable, OpenAPI-described contracts. Introduces a consolidated config endpoint pattern with optional course/user context, authentication boundaries, and a rollout plan following OEP-21 DEPR process. * docs: add ADR for canonical MFE configuration endpoint Record that /api/frontend_site_config/v1/ is the canonical endpoint for MFE/front-end runtime configuration (frontend-base SiteConfig, OEP-65) and that /api/mfe_config/v1 is legacy, on the DEPR path tracked in #37255 and added, and that user-context data (roles, permissions) belongs on resource-oriented endpoints rather than on a configuration payload. Documentation/schema coverage is deferred to the API Documentation & Schema Coverage ADR (#38189). Partially supersedes ADR 0001 (MFE Config API). Part of FC-0118 Open edX REST API standardization (#38137). Refs #38280 * docs: add ADR for standardizing authentication patterns (#38301) * docs: add ADR for standardizing authentication patterns * docs: resolve confusion & update the ADR based on OEP-0042 * docs: support multiple valid auth schemes & deprecate BearerAuthentication * docs: change wording for decisions a bit. * docs: add real examples in accordance with our updated decisions * docs: sync ADR with edx-drf-extensions issue 284 openedx/edx-drf-extensions#284 * docs: make doc more explicit & address comments * docs: move Bearer auth depr plan out of ADR Move BearerAuthentication depr plan out of this doc So that it resides in single place i.e. to its deprecation ticket. * docs: add a pointer file in oauth_dispatch for this ADR * docs: make decision more clear * docs: make authentication_classes usage more clearer * docs: adress the comment related to session authentication * chore: correct file number w.r.t order of the ADRs --------- Co-authored-by: Muhammad Faraz Maqsood <faraz.maqsood@A006-01130.local> Co-authored-by: Taimoor Ahmed <68893403+taimoor-ahmed-1@users.noreply.github.com> Co-authored-by: Taimoor Ahmed <taimoor.ahmed@A006-01711.local> Co-authored-by: Robert Raposa <rraposa@edx.org> Co-authored-by: Abdul Muqadim <abdul.muqadim@A006-01811.local> Co-authored-by: Abdul Muqadim <abdul.muqadim@192.168.1.7> Co-authored-by: Abdul-Muqadim-Arbisoft <139064778+Abdul-Muqadim-Arbisoft@users.noreply.github.com>
7 participants
Description
Adds ADR-0036 documenting the strategy for reducing deeply nested JSON payloads across Open edX APIs.
Some endpoints (course block trees, OLX structure, progress views) return large, deeply nested responses that are expensive to parse for both human clients and automated agents. This ADR establishes:
?view=minimal / ?fields=... query params as the standard convention for requesting a stripped-down representation of complex resources
Prefer IDs + follow-up endpoints over embedding entire sub-trees inline (e.g. return block IDs in a list response, fetch full block data via a detail endpoint)
Document all response variants (minimal vs. full) in OpenAPI schemas
related Issue: #38281