Skip to content

fix(worker): N+1 query in ProcessFlakesTask - #1013

Closed
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/worker-n-plus-1-flakes-gOnDiX
Closed

fix(worker): N+1 query in ProcessFlakesTask#1013
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/worker-n-plus-1-flakes-gOnDiX

Conversation

@sentry

@sentry sentry Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

This PR addresses an N+1 query performance issue within the ProcessFlakesTask in app.tasks.flakes.ProcessFlakeTask. Root Cause: Previously, process_flakes_for_commit would iterate through uploads and call process_single_upload for each. Inside process_single_upload, get_testruns(upload) would issue a separate database query for Testrun objects for every single upload. This resulted in an N+1 query pattern where N is the number of uploads for a given commit. Additionally, handle_pass was calling flake.save() individually for each flake that reached its expiration criteria (30 recent passes), leading to excessive database writes within the processing loop. Changes Made: 1. Batch Testrun Fetching: A new function get_testruns_for_uploads was introduced to fetch all Testrun objects for a list of uploads in a single, optimized database query. process_flakes_for_commit now calls this function once and passes the relevant subset of testruns to process_single_upload. 2. Bulk Flake Expiration Updates: The handle_pass function no longer calls flake.save(). Instead, expiring flakes are collected into a list (expiring_flakes). After all uploads have been processed, process_flakes_for_commit performs a single Flake.objects.bulk_update operation for all collected expiring flakes. These changes significantly reduce the number of database queries and writes, improving the performance of the ProcessFlakesTask. ### Legal Boilerplate Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Fixes WORKER-YPV

@thomasrockhu-codecov

Copy link
Copy Markdown
Contributor

Closing as duplicate — Sentry Seer generated multiple PRs for the same issue. Keeping the newest/best candidate for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant