Skip to content

Clear featured-list caches when featured learning paths are edited - #3757

Open
mbertrand wants to merge 12 commits into
mainfrom
featured-list-cache-clearing
Open

Clear featured-list caches when featured learning paths are edited#3757
mbertrand wants to merge 12 commits into
mainfrom
featured-list-cache-clearing

Conversation

@mbertrand

@mbertrand mbertrand commented Aug 11, 2026

Copy link
Copy Markdown
Member

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/11979

Description (What does it do?)

Channel managers couldn't see featured-list edits for up to 24h (Redis view cache) plus 30min (Fastly page cache). Now:

  • Editing a unit channel's featured learning path through the API (path update/delete, item add/reorder/remove, or the bulk add-to-list action) clears the featured_resources Redis cache, hard-purges /c/unit/<name> in Fastly, and soft-purges the homepage, once the change commits.
  • The clear runs synchronously in the request rather than via celery, so it can't sit behind ETL/indexing work on the default queue and is guaranteed done by the time the save returns. It's best-effort: a Redis or Fastly failure just logs and doesn't break the save, and each Fastly purge is independent of the others.
  • Frontend learning-path mutations also invalidate the featured React Query cache, so editors see the change without a full reload.
  • call_fastly_purge_api gains a soft=True kwarg (sends Fastly-Soft-Purge: 1).

How can this be tested?

  1. Seed featured lists if you haven't already:

    docker compose run --rm web python manage.py populate_featured_lists
  2. In a private/incognito window (anonymous), load a unit channel page — e.g. http://open.odl.local:8062/c/unit/mitx — and note the courses in the featured carousel. Load the homepage too. This primes the anonymous Redis cache for /api/v1/featured/.

  3. In a window where you're logged in as an admin user, go to http://open.odl.local:8062/learningpaths, open "MITx Featured Resources", and remove a course from the list (or add one via a course card's add-to-list dialog).

    docker compose logs -f web | grep -i "Skipping Fastly"

    As soon as the edit saves you should see:

    Skipping Fastly purge for /c/unit/mitx (dev environment)
    Skipping Fastly purge for / (dev environment)
    

    The Fastly purges are skipped locally (no FASTLY_API_KEY); on RC/prod they become real PURGE requests against /c/unit/<name> (hard) and / (soft).

  4. Reload the incognito channel page and homepage: the featured carousel reflects the edit. Without this PR, the anonymous featured API response stays cached for REDIS_VIEW_CACHE_DURATION (24h default in deployed envs) and the pages keep rendering the old list.

Additional Context

  • Fastly URL purges are exact-match, so query-string variants of the channel page (?topic=...) stay cached up to s-maxage (30 min). The base URL the editor refreshes is hard-purged. Per-channel surrogate keys are the upgrade path if that ever matters.
  • Edits via Django admin/shell don't trigger invalidation — the hooks are on the viewsets deliberately, since model signals would fire on every ETL run.

mbertrand and others added 8 commits August 11, 2026 13:58
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The browser query client's 30-minute staleTime (matched to the CDN TTL)
means the editor's own session never refetches the featured list after
they edit a learning path — the backend purge only helps on a full page
reload. Invalidate the featured query key in the mutations that map to
the backend's cache-clearing write hooks so the editor sees their change
immediately on client-side navigation too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hoist the tasks import to module level (tasks was already imported at
module scope, so the function-local import bought nothing), shrink the
bulk-action id set (featured_list_id__in coerces strings), and delete
the tautological retry-config test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rize

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

OpenAPI Changes

No changes detected

View full changelog

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).

mbertrand and others added 3 commits August 12, 2026 08:47
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cache-clear task landed on the default Celery queue, which also runs
ETL backpopulates and bulk indexing, so a purge could sit behind minutes
of queued work -- defeating the point of the feature (editor sees changes
within seconds). The purge work is cheap (tuned Redis pattern delete plus
two Fastly calls with 5s timeouts) and only triggered by rare human saves,
so run it best-effort in the request via transaction.on_commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mbertrand
mbertrand marked this pull request as ready for review August 12, 2026 15:16
Copilot AI balanced review requested due to automatic review settings August 12, 2026 15:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds cache invalidation when featured learning paths are edited.

Changes:

  • Adds Fastly soft-purge support and backend invalidation hooks.
  • Invalidates featured React Query caches after mutations.
  • Adds backend/frontend tests and null-safe item counts.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
main/utils.py Adds soft-purge headers.
main/utils_test.py Tests soft-purge behavior.
learning_resources/views.py Adds featured-cache invalidation hooks.
learning_resources/views_learningpath_test.py Tests backend invalidation paths.
frontends/ol-components/src/components/LearningResourceCard/LearningResourceListCard.tsx Handles missing item counts.
frontends/api/src/hooks/learningResources/index.ts Invalidates featured queries after bulk edits.
frontends/api/src/hooks/learningResources/index.test.ts Tests bulk-edit invalidation.
frontends/api/src/hooks/learningPaths/index.ts Invalidates featured queries after path mutations.
frontends/api/src/hooks/learningPaths/index.test.ts Tests path-mutation invalidation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread learning_resources/views.py Outdated
Comment on lines +504 to +505
Runs synchronously in the request (not via Celery) so the purge is done
by the time the editor's save returns, regardless of worker backlog.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentional — the celery version sat on the default queue behind ETL backpopulates and bulk indexing, so a purge could wait minutes, which defeats the point of the issue (editor confirms their change within seconds). The work on the request path is small: a tuned Redis pattern delete plus a couple of Fastly purges that normally return in tens of ms. Worst case with Fastly down is ~5s per purge before the save returns (still succeeds, just logs). The PR description was stale from the earlier approach — updated it to match.

A failed channel-page purge no longer aborts the homepage purge (or other
channel purges); each failure is logged and the loop continues.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (1)

learning_resources/views.py:495

  • If Redis raises here, clear_featured_caches exits before any Fastly purge runs; the outer callback catches the exception, but all channel and homepage objects remain stale. Handle/log the Redis failure locally so the independent best-effort Fastly purges still execute.
    clear_views_cache(key_prefix="featured_resources")

@mbertrand mbertrand added the Needs Review An open Pull Request that is ready for review label Aug 12, 2026
@shanbady shanbady self-assigned this Aug 12, 2026

@shanbady shanbady left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. LGTM 👍

@shanbady shanbady assigned mbertrand and unassigned shanbady Aug 12, 2026
@shanbady shanbady added Waiting on author and removed Needs Review An open Pull Request that is ready for review labels Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants