Skip to content

Add code to generate coordinates datasets#8

Merged
jdha merged 8 commits into
mainfrom
test_dataset
Jun 2, 2021
Merged

Add code to generate coordinates datasets#8
jdha merged 8 commits into
mainfrom
test_dataset

Conversation

@malmans2
Copy link
Copy Markdown
Member

@malmans2 malmans2 commented May 17, 2021

  • Closes Datasets for developing #6
  • Tests added
  • Passes pre-commit run --all-files
  • Changes are documented in whats-new.rst
  • New functions/methods are listed in api.rst
  • Project, label, and assignee tabs are populated

I added a function to generate rectilinear grids. It could be handy in general, so it's public under utils.
E.g.,

from pydomcfg.utils import generate_nemo_rectilinear_grid
ds = generate_nemo_rectilinear_grid(range(1, 10), range(1, 20))

I also created a couple of simple bathymetry datasets (flat and island), we can come up with something more interesting when we actually need datasets to test. There's also a placeholder test, just to show how we could make use of this datasets.
Test datasets are not exposed to users in the documentation, but can be accessed with:

from pydomcfg.tests.datasets import ds_flat, ds_island

Hopefully I got NEMO's Arakawa right, but double-check!

@malmans2 malmans2 added the enhancement New feature or request label May 17, 2021
@codecov
Copy link
Copy Markdown

codecov Bot commented May 17, 2021

Codecov Report

Merging #8 (6912ac9) into main (bd61d57) will increase coverage by 32.50%.
The diff coverage is 97.14%.

Impacted file tree graph

@@             Coverage Diff             @@
##             main       #8       +/-   ##
===========================================
+ Coverage   60.00%   92.50%   +32.50%     
===========================================
  Files           1        2        +1     
  Lines           5       40       +35     
===========================================
+ Hits            3       37       +34     
- Misses          2        3        +1     
Flag Coverage Δ
unittests 92.50% <97.14%> (+32.50%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pydomcfg/utils.py 96.96% <96.96%> (ø)
pydomcfg/__init__.py 71.42% <100.00%> (+11.42%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd61d57...6912ac9. Read the comment docs.

@malmans2
Copy link
Copy Markdown
Member Author

@pyNEMO/pydomcfg I'm going to move the function to create the horizontal Cartesian grid under the tests directory. Then other functions could use that one to add different bathy (e.g., channel, ...).

The inputs should be 1D array-like so we are not constrained to regular grids. Currently the function inputs are deltas (just because I thought that's what really matters), and the first point is at (0, 0). Would you guys prefer to switch to x/y coordinates for input? I.e., input lat lon rather than e1 e2?

Also, this is the dataset structure:

<xarray.Dataset>
Dimensions:     (x: 5, y: 10)
Coordinates:
    e1t         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    e1v         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    e1f         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    e1u         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    glamt       (y, x) float64 0.0 1e+03 2e+03 3e+03 ... 1e+03 2e+03 3e+03 4e+03
    glamv       (y, x) float64 0.0 1e+03 2e+03 3e+03 ... 1e+03 2e+03 3e+03 4e+03
    nav_lon     (y, x) float64 0.0 1e+03 2e+03 3e+03 ... 1e+03 2e+03 3e+03 4e+03
    glamf       (y, x) float64 500.0 1.5e+03 2.5e+03 ... 2.5e+03 3.5e+03 4.5e+03
    glamu       (y, x) float64 500.0 1.5e+03 2.5e+03 ... 2.5e+03 3.5e+03 4.5e+03
    e2t         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    e2u         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    e2f         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    e2v         (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    gphit       (y, x) float64 0.0 0.0 0.0 0.0 0.0 ... 9e+03 9e+03 9e+03 9e+03
    gphiu       (y, x) float64 0.0 0.0 0.0 0.0 0.0 ... 9e+03 9e+03 9e+03 9e+03
    nav_lat     (y, x) float64 0.0 0.0 0.0 0.0 0.0 ... 9e+03 9e+03 9e+03 9e+03
    gphif       (y, x) float64 500.0 500.0 500.0 ... 9.5e+03 9.5e+03 9.5e+03
    gphiv       (y, x) float64 500.0 500.0 500.0 ... 9.5e+03 9.5e+03 9.5e+03
Dimensions without coordinates: x, y
Data variables:
    Bathymetry  (y, x) float64 1e+03 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
    mask        (y, x) int64 1 1 1 1 1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1 1 1 1 1

Anything missing/redundant? Can we get rid of nav_* and/or mask, or are they gonna be needed?

@malmans2
Copy link
Copy Markdown
Member Author

malmans2 commented Jun 1, 2021

I slightly changed the original structure.
There's a generate_cartesian_grid under utils (eventually will become part of domhgr when we are done with zgr). The arguments are now face coordinates rather than grid spacing:

def generate_cartesian_grid(x_f, y_f) -> Dataset:

I've also introduced a class to generate test datasets:

class Bathymetry:

Example usage:

from pydomcfg.tests.bathymetry import Bathymetry
square = Bathymetry(range(10), range(10))
shallow = square.flat(10)
deep = square.flat(1.0e3)

If it looks OK let's merge this so @oceandie has a test dataset to get started and @jdha can add methods to Bathymetry (e.g., channel, slope, half_bowl, ...)

@oceandie
Copy link
Copy Markdown
Contributor

oceandie commented Jun 2, 2021

I slightly changed the original structure.
There's a generate_cartesian_grid under utils (eventually will become part of domhgr when we are done with zgr). The arguments are now face coordinates rather than grid spacing:

def generate_cartesian_grid(x_f, y_f) -> Dataset:

I like the idea, just a minor cooment: maybe in the future I would just add some more comments/documentation - it is not immediately clear what you mean by face and center (it depends from where you start) and to track this nomenclature with the Arakawa-C structure.

I've also introduced a class to generate test datasets:

class Bathymetry:

Example usage:

from pydomcfg.tests.bathymetry import Bathymetry
square = Bathymetry(range(10), range(10))
shallow = square.flat(10)
deep = square.flat(1.0e3)

If it looks OK let's merge this so @oceandie has a test dataset to get started and @jdha can add methods to Bathymetry (e.g., channel, slope, half_bowl, ...)

@jdha
Copy link
Copy Markdown
Contributor

jdha commented Jun 2, 2021

minor semantics, but would it not be cleaner for
generate_cartesian_grid(x_f, y_f)
to take the nemo-like form
generate_cartesian_grid(ppe1_m, ppe2_m, jpiglo, jpjglo)
this would also allow the bathy tests to be constructed using single parameters rather than vectors (and possibly even using a namelist?)

@malmans2
Copy link
Copy Markdown
Member Author

malmans2 commented Jun 2, 2021

Makes sense! How about if ppe{1,2}_m is a vector jp{i,j}glo must be None, otherwise jp{i,j}glo must be an integer? So we cover all cartesian grids and we align with NEMO namelists?
So we would revert to users providing grid spacing rather than coordinates (@oceandie thoughts?). We could probably support both grid spacing/coordinates as input, but I think this would just over-complicate things.
I guess we could also add ppg{lam,phi}0, by default set to (0, 0).

@oceandie
Copy link
Copy Markdown
Contributor

oceandie commented Jun 2, 2021

Makes sense! How about if ppe{1,2}_m is a vector jp{i,j}glo must be None, otherwise jp{i,j}glo must be an integer? So we cover all cartesian grids and we align with NEMO namelists?
So we would revert to users providing grid spacing rather than coordinates (@oceandie thoughts?). We could probably support both grid spacing/coordinates as input, but I think this would just over-complicate things.
I guess we could also add ppg{lam,phi}0, by default set to (0, 0).

I agree with both your suggestions - than in the end this will be just for testing our zgr developments ....

@jdha
Copy link
Copy Markdown
Contributor

jdha commented Jun 2, 2021

I agree with both your suggestions - than in the end this will be just for testing our zgr developments ....

It won't be just for testing zgr dev as the unit tests will probably continue to use this (+ it'll be integrated into hgr later). Happy to go with @malmans2 hybrid suggestion.

@oceandie
Copy link
Copy Markdown
Contributor

oceandie commented Jun 2, 2021

I agree with both your suggestions - than in the end this will be just for testing our zgr developments ....

It won't be just for testing zgr dev as the unit tests will probably continue to use this (+ it'll be integrated into hgr later). Happy to go with @malmans2 hybrid suggestion.

I have some doubts regarding the later integration into hgr - I think if we'll go towards a general curvilinear grid approach (where the cartesian grid is just a special case), the structure will be a bit different - but happy to go now and think about this later on

@malmans2
Copy link
Copy Markdown
Member Author

malmans2 commented Jun 2, 2021

OK, I think it's in good shape now. Please have another look and feel free to merge if it's good to go!

@jdha jdha merged commit 833d4f1 into main Jun 2, 2021
@jdha jdha deleted the test_dataset branch June 2, 2021 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Datasets for developing

3 participants