Skip to content

[SYSTEMDS-3931] Out-of-core right indexing operations#2351

Closed
janniklinde wants to merge 5 commits into
apache:mainfrom
janniklinde:OOCRightIndexingPR2
Closed

[SYSTEMDS-3931] Out-of-core right indexing operations#2351
janniklinde wants to merge 5 commits into
apache:mainfrom
janniklinde:OOCRightIndexingPR2

Conversation

@janniklinde

@janniklinde janniklinde commented Nov 10, 2025

Copy link
Copy Markdown
Contributor

This patch introduces out-of-core right indexing support. Note that because we forward aligned blocks, issues could occur with OOCEvictionManager (evictions might happen based on the old stream, affecting the new output stream with the same block references). Similar issues could occur when having stacked CachingStreams.

@codecov

codecov Bot commented Nov 16, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.87296% with 71 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.31%. Comparing base (c7300f3) to head (52ebd31).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
...ntime/instructions/ooc/IndexingOOCInstruction.java 73.46% 21 Missing and 18 partials ⚠️
...instructions/ooc/MatrixIndexingOOCInstruction.java 82.22% 13 Missing and 11 partials ⚠️
.../runtime/instructions/ooc/UnaryOOCInstruction.java 0.00% 3 Missing ⚠️
...java/org/apache/sysds/runtime/util/IndexRange.java 0.00% 2 Missing and 1 partial ⚠️
.../sysds/runtime/instructions/ooc/CachingStream.java 88.88% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2351      +/-   ##
============================================
+ Coverage     72.29%   72.31%   +0.01%     
- Complexity    46829    46879      +50     
============================================
  Files          1508     1510       +2     
  Lines        177638   177927     +289     
  Branches      34880    34937      +57     
============================================
+ Hits         128430   128668     +238     
- Misses        39511    39531      +20     
- Partials       9697     9728      +31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mboehm7

mboehm7 commented Nov 16, 2025

Copy link
Copy Markdown
Contributor

LGTM - thanks for the patch @janniklinde. During the merge I only resolved the merge conflicts and removed a few warnings.

@mboehm7 mboehm7 closed this in b0ef875 Nov 16, 2025
@github-project-automation github-project-automation Bot moved this from In Progress to Done in SystemDS PR Queue Nov 16, 2025
aperov9 pushed a commit to aperov9/systemds that referenced this pull request Nov 17, 2025
@janniklinde janniklinde deleted the OOCRightIndexingPR2 branch February 20, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants