feat(obs4ref): add CEDA obs_for_ref_v2 reference datasets

[lewisjared]

Summary

Adds 33 obs4REF reference files across 12 CEDA obs_for_ref_v2 datasets, drops 8 superseded entries. NetCDF uploaded to ref-obs4ref-public, resolving at https://obs4ref.climate-ref.org/obs4REF/.... Registry 58 -> 83.

Added / upgraded

• LORA-1-0 (mrro)
• WECANN-1-0 (gpp/hfls/hfss)
• ESACCI-CLOUD-AVHRR-AMPM-3-0 (12 radiation/cloud vars)
• GFED-5-0 (burntFractionAll)
• SAGE-CCI-OMPS (o3, zonal)
• FLUXNET2015-1-0 (gpp, site)
• HWSD-2-0 (cSoil)
• CALIPSO-ICECLOUD-1-0 (cli)
• WOA-23 under NOAA-NCEO-OCL (no3/o2/po4/si/so/thetao/ohc/ohcJm2)
• RAPID-2023-1a (msftmz)
• CCI-CryoClim-FSC-1 (snc)
• Hoffman-1-0 (fgco2/nbp)

Removed

• WECANN-1-0 20250516 (3 files) -> v20250902
• NOAA-NCEI/WOA2023 20250516 (5 files) -> NOAA-NCEO-OCL/WOA-23
• Old objects stay in the bucket as orphans; prune separately.

Provenance

• Each entry carries a # source: comment with its CEDA dap.ceda.ac.uk download URL.
• Documentary only (pooch skips comments); fetches still resolve against the obs4ref base URL, not CEDA.
• The comment is the only place the CEDA path survives — the registry path uses the DRS grid label (gn/gm/gr/gnz/site), not the CEDA resolution string.

Verification

• All 33 files md5-match the CEDA listing and scripts/create-registry.py output.
• import climate_ref loads 83 entries, 0 per-file URL overrides.
• Every uploaded key returns HTTP 200 at the public base URL, including each new grid/frequency shape (site, gnz, gr, fx, monC).

Follow-up

• Several diagnostics need updating to consume the newer datasets (superseded WECANN, NOAA-NCEI/WOA2023, old RAPID source). Tracked separately.

Summary by CodeRabbit

• New Features

  • Expanded the Obs4REF reference registry with 33 additional files across 12 observational dataset categories (including cloud/radiation, burnt area, ozone, fluxes, soil carbon, ice cloud, ocean biogeochemistry, AMOC transport, snow cover, and ocean carbon fluxes).

• Bug Fixes

  • Refreshed multiple registry entries with updated versions and metadata, including replacements for older WECANN and WOA2023 records.

• Tests

  • Updated unit test expectations to reflect the larger Obs4REF registry.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e545bd5e-0736-4681-abe8-6ca509ccd6e2

📥 Commits

Reviewing files that changed from the base of the PR and between 7439171 and 600eff2.

📒 Files selected for processing (1)

• packages/climate-ref-core/tests/unit/test_dataset_registry/test_dataset_registry.py

✅ Files skipped from review due to trivial changes (1)

• packages/climate-ref-core/tests/unit/test_dataset_registry/test_dataset_registry.py

---

📝 Walkthrough

Walkthrough

This PR updates the obs4REF dataset registry reference file with newer dataset entries, updated MD5 hashes, and added source comments, adjusts the related unit test count, and adds a changelog entry describing the registry changes.

Changes

Obs4REF reference registry update

Layer / File(s)	Summary
Updated dataset entries and provenance comments packages/climate-ref/src/climate_ref/dataset_registry/obs4ref_reference.txt	Replaces ARCCSS LORA-1-1, ColumbiaU WECANN, and NOAA WOA2023 entries with newer versioned datasets, updates DWD and other dataset paths and MD5 hashes, adds new FMI, Fluxnet2015, IIASA-FAO, DoESS-UCI GFED, NR CCI-CryoClim-FSC, and UCI-ORNL Hoffman entries, and adds # source: comment lines throughout.
Updated registry file count expectation packages/climate-ref-core/tests/unit/test_dataset_registry/test_dataset_registry.py	Changes the unit test constant for the expected Obs4REF file count from 58 to 83.
Changelog entry for registry update changelog/778.feature.md	Adds a changelog entry describing the addition of 33 new reference-data files across 12 observational dataset categories and the replacement of older WECANN and WOA2023 entries.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: adding obs4REF reference datasets from CEDA.
Description check	✅ Passed	The description is detailed and covers the change, verification, provenance, and removals, but it does not follow the template's checklist format exactly.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/obs4ref-ceda-datasets

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
core	92.35% <ø> (ø)
providers	86.56% <ø> (+0.25%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.