Determine activities and institutes for CMIP6 loading CV table

[sloosvel]

Fixes #37

[zklaus]

Works nicely!

[jvegreg]

Can the people with access to DKRZ to check that the changes in the config-developer are ok for them? I have checked BADC and it works smoothly, allowing us to mix data from CMIP and HighResMIP in the same recipe

[valeriupredoi]

this is very useful, totally missed it until @zklaus pointed me to it offline 🍺

[mattiarighi]

@bouweandela can we merge this?
@schlunma needs it quite urgently for the automatisation of the results browser.

Thanks!

[mattiarighi]

@schlunma just merge this once tested.

[schlunma]

Works well for CMIP6 data sets appearing in esmvalcore/cmor/tables/cmip6/Tables/CMIP6_CV.json. However, there are models (e.g. BCC-CSM2-MR) missing in there, for which the tool will throw the following error now:

KeyError: "Dataset key institute must be specified for {...}"

Do we use an old version of the CMIP6 tables? @zklaus @jvegasbsc

[schlunma]

Also, once this is merged the users might need to update their CMIP6 rootpath in the config-user.yml file. We also need to update the symlink on DKRZ pointing to the CMIP6 archive to one directory above the current one.

[jvegreg]

Do we use an old version of the CMIP6 tables? @zklaus @jvegasbsc

Yes, we have #158 to solve this. BCC-CSM2-MR is in that pull request

[mattiarighi]

@schlunma can you update and test again after #158?

[schlunma]

CESM2-WACCM is missing, even in the new version of the tables. Other than that the PR works fine. I tested CMIP, ScenarioMIP and C4MIP data.

[mattiarighi]

Is there any easy workaround @jvegasbsc?
Otherwise I would suggest to merge this and wait for the next release of the tables.

[zklaus]

Otherwise I would suggest to merge this and wait for the next release of the tables.

I thing that's the right way. That source_id was added on 2019-07-29, a couple more have been added since, but there is currently no release of the tables that contains this information.

[jvegreg]

Is there any easy workaround @jvegasbsc?
The easiest one is to add the institute in the recipe. I think is not a good idea to keep an extra list of those missing in the tool, as it will be a nightmare to maintain