fix(worker): prevent Cobertura processing timeouts and improve performance - #1891
fix(worker): prevent Cobertura processing timeouts and improve performance#1891sentry[bot] wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (60.71%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. 📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (60.71%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1891 +/- ##
==========================================
- Coverage 91.83% 91.81% -0.02%
==========================================
Files 1332 1332
Lines 51830 51847 +17
Branches 1647 1647
==========================================
+ Hits 47598 47604 +6
- Misses 3911 3922 +11
Partials 321 321
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
This PR addresses
ChordError: Dependency ... raised SoftTimeLimitExceeded()errors originating from theapp.tasks.upload.UploadProcessorCelery task.Root Cause:
TheThe
CoberturaProcessor.processmethod was taking an excessive amount of time (~8.86s in one observed case) to parse large Cobertura XML reports, pushing theUploadProcessortask close to its soft time limit. When theSoftTimeLimitExceededsignal fired during thefinallyblock's Redis cleanup (state.clear_in_progress_uploads), it caused the task to be marked asFAILURE, leading to aChordErrorin the downstream callback, even if the main report processing had completed successfully.Solution Implemented:
finallyblock handling: Wrapped thestate.clear_in_progress_uploadscall inservices/processing/processing.pywith atry/except SoftTimeLimitExceeded. This prevents late-arriving timeout signals from causing aChordErrorfor an otherwise completed task, ensuring the chord callback receives a successful result.ReportTooLargeErrorexception and a configurableparsers.cobertura.max_classeslimit (defaulting to 50,000 classes) inservices/report/languages/cobertura.py. Reports exceeding this limit now fail fast with a clear error (UNSUPPORTED_FILE_FORMAT) instead of consuming excessive resources and timing out.CoberturaProcessor.from_xmlto merge the collection of path-fixing data into the initial iteration over<class>elements. This eliminates a redundant second full XML scan, significantly improving performance for large reports.ReportTooLargeErrorinReportService.build_report_from_raw_content(services/report/__init__.py), mapping it toUploadErrorCode.UNSUPPORTED_FILE_FORMATfor consistent error reporting.Legal Boilerplate
Look, I get it. The entity doing business as "Codecov" is owned by Harness, Inc. In 2026 Harness acquired Codecov and as a result Harness is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Harness can use, modify, copy, and redistribute my contributions, under Harness's choice of terms.
Fixes WORKER-Z5H