Fix ERA5 native6 fix to handle single monthly-averaged NetCDF files.

[rbeucher]

Description

Modifies the get_frequency function in ERA5 native6 fix to handle case where monthly-averaged data are stored in single NetCDF file (one file per month) and have thus a time dimension length of 1.

Closes #2511

Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[rbeucher]

Hi @schlunma, @valeriupredoi,

Could someone please take a moment to review this for me? Thanks!

R

[codecov]

Codecov Report

❌ Patch coverage is 91.30435% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.42%. Comparing base (cbb23dc) to head (0b6af74).
⚠️ Report is 98 commits behind head on main.

Files with missing lines	Patch %	Lines
esmvalcore/cmor/_fixes/native6/era5.py	91.30%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2512      +/-   ##
==========================================
- Coverage   95.46%   95.42%   -0.04%     
==========================================
  Files         260      260              
  Lines       15526    15513      -13     
==========================================
- Hits        14822    14804      -18     
- Misses        704      709       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[valeriupredoi]

looking good @rbeucher - and biy did you have to wait for a review - sorry 🍺 @schlunma @sloosvel maybe we can squeeze this one in 2.12?

[schlunma]

This currently breaks support for daily ERA5 data that has been added in #2178.

[github-actions]

In order to maintain a backlog of relevant pull requests, we automatically label them as stale after 180 days of inactivity.

If this pull request is still important to you, please comment below to remove the stale label. Otherwise, this pull request will be automatically closed in 60 days. If this pull request only suffers from a lack of reviewers, please tag the @ESMValGroup/technical-lead-development-team so they can help you find a suitable reviewer.

[flicj191]

I will look into this again to fix the single monthly files

[flicj191]

@schlunma @valeriupredoi @ESMValGroup/esmvaltool-developmentteam
Can we get this re-checked please?
FYI @rbeucher

[bouweandela]

We discussed this feature in the associated issue, and there I suggested #2511 (comment). Does that not work?

[flicj191]

We discussed this feature in the associated issue, and there I suggested #2511 (comment). Does that not work?

Ah I missed that sorry, I'll test it out and found that it still runs the get_frequency each time for class AllVars here:

ESMValCore/esmvalcore/cmor/_fixes/native6/era5.py

Line 570 in 9ef036a

self._fix_monthly_time_coord(cube)

ESMValCore/esmvalcore/cmor/_fixes/native6/era5.py

Lines 585 to 587 in 9ef036a

	def _fix_monthly_time_coord(cube):
	"""Set the monthly time coordinates to the middle of the month."""
	if get_frequency(cube) == "monthly":

I can change this part in this PR

[flicj191]

@bouweandela I've updated, let me know what you think. thanks!

[bouweandela]

Looks fine now. I wonder if we could completely get rid of the get_frequency function?

It would be nice to add a unit test, could you do that?

[flicj191]

Removing get_frequency I had to parse the frequency to fix_hourly_time_coordinate and fix_accumulated_units which I think works, similar to _fix_monthly_time_coord but the tests are failing, not sure what i'm missing? the fixes aren't applied? @bouweandela

[flicj191]

I have reverted the changes of removing the get_frequency function, more testing was showing that it wasn't working if frequency wasn't defined first in the dataset which doesn't seem to be common practise so would still be useful to have.

[bouweandela]

@flicj191 In #2855 I've changed the test so it is more realistic and you can see that a frequency keyword argument is now always supplied.

Pull request has changed since review and @schlunma is currently on leave.