Skip to content

Compare large arrays in memory-bounded chunks#918

Open
Silvren wants to merge 1 commit into
Open-EO:masterfrom
Silvren:agent/memory-bounded-array-comparison
Open

Compare large arrays in memory-bounded chunks#918
Silvren wants to merge 1 commit into
Open-EO:masterfrom
Silvren:agent/memory-bounded-array-comparison

Conversation

@Silvren

@Silvren Silvren commented Jul 13, 2026

Copy link
Copy Markdown

Summary

  • lazily open NetCDF job result datasets instead of eagerly loading them
  • compare compatible numerical arrays above a 64 MiB threshold in bounded rectangular chunks
  • stream mismatch counts and absolute-difference statistics without full-size intermediate arrays
  • preserve rtol, atol, NaN, complex-number, and pixel_tolerance behavior
  • keep the existing detailed comparison path for smaller arrays

Testing

  • python -m pytest tests/testing -q (86 passed)
  • pre-commit run --files CHANGELOG.md openeo/testing/results.py tests/testing/test_results.py

Fixes #871

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

testing: support large arrays

1 participant