Derived variable needs different input dataset for CMIP5/CMIP6

[ledm]

The AMOC derived variable needs to use different data depending on whether we're looking at CMIP5 or CMIP6 data. This is because the input variable that was used in CMIP5 (msftmyz) has been superseded by a slightly different variable in CMIP6 (msftyz).

Is there a way to derive a variable such that it checks for the presence/absence of the two variables msftmyz or msftyz then uses the best one? It shouldn't fail if it doesn't find one dataset, provided that it finds the other.

(The difference between the two is that msftmyz is defined in the meridional direction and msftyz is defined in the Y direction.)

I like to keep you on your toes! Enjoy Bavaria this week!

[mattiarighi]

I think @axel-lauer had a similar issue some time ago, but I can't remember the details.

[zklaus]

@debe-kevin also just stumbled upon this.

There is a facility in the cmor checker for variables that have changed name from CMIP5 to CMIP6, namely the mapping in

ESMValCore/esmvalcore/cmor/table.py

Line 73 in 20296fc

_CMIP_5to6_varname = {

@ledm I am not completely clear on the move: there seem to be two variables that can be considered to be the new versions of msftmyz in CMIP6: msftmz and msftyz. My intuition is that actually msftmz is closer to msftmyz in spirit. What do you think?

[schlunma]

Is this fixed by #230 @valeriupredoi? Did not try it, just guessing from reading the source code 😬

[ledm]

I've been looking at what data is available and I've seen that the follow variable have CMIP6 model data in the historical r1* ensemble member:

• msftmyz : GISS-E2-1-G, GISS-E2-1-G
• msftmz : CanESM5, CESM2, CESM2-WACCM, SAM0-UNICON
• msftyz : CNRM-CM6-1, CNRM-CM6-1, CNRM-ESM2-1, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, EC-Earth3-Veg, IPSL-CM6A-LR, HadGEM3-GC31-LL, UKESM1-0-LL

It looks like the CMIP6 model have done a pick-n-mix with their favourite streamfunction variable. Perhaps it would be better to use a preferential cascade to determine the AMOC (ie, first try to use msftmyz, then try msftmz then try msftyz?)

A valid scientific question arises: Is there a difference between the meridional overturning circulation and the Y-direction overturning circulation? @tillku @shaynemcgregor, any thoughts?

[zklaus]

msftmyz is wrong. This variable has been retired in december 2017 in favor of the other two. These models simply need fixes. More precisely looking at the given models, the files claim to use data spec version 01.00.23, but since version 01.00.19 the variable does not exist.

The difference between the other two is described in the data request. Quoting from the data request for msftyz:

Function of Y, Z, basin. For a model with a cartesian latxlon grid, this is the same as the "Ocean Meridional Overturning Mass Streamfunction" (msftmz) and should in this case be omitted.

and for msftmz

Function of latitude, Z, basin. For a model with a cartesian latxlon grid, this is the same as the "Ocean Y Overturning Mass Streamfunction" (msftyz), which should in this case be omitted. For other models, this transport should be approximated as the transport across zig-zag paths corresponding to latitudes with spacing between latitudes appropriate to the model's resolution.

[ledm]

Okay, so now we know which variable to look at - how do we implement this in a derived variable?

At the moment, I've tried:

    # Required variables
    required = [{'short_name': 'msftmyz', 'project': 'CMIP5'},
                {'short_name': 'msftyz', 'project': 'CMIP6'}]

but that doesn't work as it still looks for both fields.

Do we need to do a re-write to the derivation module?

[schlunma]

Okay, so now we know which variable to look at - how do we implement this in a derived variable?

At the moment, I've tried:

    # Required variables
    required = [{'short_name': 'msftmyz', 'project': 'CMIP5'},
                {'short_name': 'msftyz', 'project': 'CMIP6'}]

but that doesn't work as it still looks for both fields.

Do we need to do a re-write to the derivation module?

In #230 required is going to be a function dependent on project, i.e. it will be possible to do

@staticmethod
def required(project):
    if project == 'CMIP5':
        ....
    else:
        ....

[zklaus]

Actually there is a special facility for names that changed from CMIP5 to CMIP6:

ESMValCore/esmvalcore/cmor/table.py

Line 73 in c6d3cce

_CMIP_5to6_varname = {

So I think we should add msftmyz -> msftmz to that.

Then in the recipe you will have three variables with additional_datasets as appropriate.

In other words, imho there is no derivation needed here and it is principally impossible to accurately derive msftyz from msftmz or vice versa. If one wanted to do that, they would need to go back to the 3d field and special per-grid/model knowledge to define a good path.

[ledm]

@zklaus, does that facility work for derived variables as well? It hasn't work for me in this case yet.

[zklaus]

What do you mean? Neither msftmyz in CMIP5 nor the two CMIP6 variables are derived, are they?

[zklaus]

Ah sorry, of course you are speaking about the amoc derived variable. Forget what I said. Let me look into that (and by that I mean pestering @schlunma, of course).

[zklaus]

@ledm you can have a look at the derivation of sispeed for an example.
Basically you then use the CMIP5 name in the required attribute and for the CMIP6 variables it will use the new name instead. This should solve the issue for the msftmz models, which is what I would start with.

Then to tackle those providing only msftyz maybe we can add a derivation of msftmz. Whether to do this inaccurately from msftyz (probably still good enough) or do something more involved to get it from a 3d field can be discussed. So all in all the required block for amoc then should look like

required = [{'short_name': 'msftmyz', 'derive': True}]

Sound good?

[ledm]

Sorry, I just don't get it. How is sispeed similar? There doesn't appear to be a cmip5/cmip6 naming difference there.

Also, it looks like PR #230 should resolve this issue right, @schlunma? If so, it might be better to wait until that PR is accepted...