Skip to content

Avoid copying input data in CMIP6 CESM2 air pressure level fixes#2454

Closed
bouweandela wants to merge 13 commits into
mainfrom
in-memory-fix-file
Closed

Avoid copying input data in CMIP6 CESM2 air pressure level fixes#2454
bouweandela wants to merge 13 commits into
mainfrom
in-memory-fix-file

Conversation

@bouweandela

@bouweandela bouweandela commented Jun 10, 2024

Copy link
Copy Markdown
Member

Description

Use ncdata to fix files in-memory instead of copies on disk.

Related to #2129 and #674


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.


To help with the number pull requests:

@bouweandela bouweandela mentioned this pull request Jun 10, 2024
62 tasks
@valeriupredoi

Copy link
Copy Markdown
Contributor

very cool @bouweandela 👏 Always nice to see data transfer optimizations, hope you don't mind me turning on the GA see if any issues for older Pythons - I took a look at the failed tests - couple of them due to mocks not called (easy fix) but one fails due to what looks to be a reversal of the lev bounds, can prob fix that for you here, if you don't have time 🍺

@codecov

codecov Bot commented Oct 17, 2024

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.78%. Comparing base (e9fccbd) to head (b95ee63).
⚠️ Report is 254 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2454   +/-   ##
=======================================
  Coverage   94.77%   94.78%           
=======================================
  Files         251      251           
  Lines       14266    14292   +26     
=======================================
+ Hits        13520    13546   +26     
  Misses        746      746           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Enable lock sharing between ncdata and Iris

Suppress load warnings

Restore lat/lon coord units after load

All fixes in fix_file
@bouweandela

bouweandela commented Oct 23, 2024

Copy link
Copy Markdown
Member Author

TODO: move to new fix preprocessor function instead of changing fix_file?

What about chunking?

@github-actions

Copy link
Copy Markdown
Contributor

In order to maintain a backlog of relevant pull requests, we automatically label them as stale after 180 days of inactivity.

If this pull request is still important to you, please comment below to remove the stale label. Otherwise, this pull request will be automatically closed in 60 days. If this pull request only suffers from a lack of reviewers, please tag the @ESMValGroup/technical-lead-development-team so they can help you find a suitable reviewer.

@github-actions github-actions Bot added the Stale label Jun 27, 2025
@github-actions

Copy link
Copy Markdown
Contributor

This stale pull request has been automatically closed due to a lack of community activity.

If you still care about this pull request, please re-open it. If you do not have sufficient permissions for this, tag the @ESMValGroup/technical-lead-development-team in a comment below including a short message that this pull request is still relevant to you. They will re-open it on your behalf.

@github-actions github-actions Bot closed this Aug 27, 2025
@bouweandela bouweandela deleted the in-memory-fix-file branch March 13, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

2 participants