Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 0 additions & 128 deletions doc/recipe/preprocessor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -300,134 +300,6 @@ and cell measure (``areacella``), but do not use ``areacella`` for dataset
timerange: '1990/2000'
scripts: null


.. _`Fx variables as cell measures or ancillary variables`:

Legacy method of specifying supplementary variables
---------------------------------------------------

.. deprecated:: 2.8.0
The legacy method of specifying supplementary variables is deprecated and will
be removed in version 2.10.0.
To upgrade, remove all occurrences of ``fx_variables`` from your recipes and
rely on automatically defining the supplementary variables based on the
requirement of the preprocessor functions or specify them using the methods
described above.
To keep using the legacy behaviour until v2.10.0, set
``use_legacy_supplementaries: true`` in the :ref:`user configuration file` or
run the tool with the flag ``--use-legacy-supplementaries=True``.

Prior to version 2.8.0 of the tool, the supplementary variables could not be
defined at the variable or dataset level in the recipe, but could only be
defined in the preprocessor function that uses them using the ``fx_variables``
argument.
This does not work well because in practice different datasets store their
supplementary variables under different facets.
For example, one dataset might only provide the ``areacella`` variable under the
``1pctCO2`` experiment while another one might only provide it for the
``historical`` experiment.
This forced the user to define a preprocessor per dataset, which was
inconvenient.

============================================================== =====================
Preprocessor Default fx variables
============================================================== =====================
:ref:`area_statistics<area_statistics>` ``areacella``, ``areacello``
:ref:`mask_landsea<land/sea/ice masking>` ``sftlf``, ``sftof``
:ref:`mask_landseaice<ice masking>` ``sftgif``
:ref:`volume_statistics<volume_statistics>` ``volcello``
:ref:`weighting_landsea_fraction<land/sea fraction weighting>` ``sftlf``, ``sftof``
============================================================== =====================

If the option ``fx_variables`` is not explicitly specified for these
preprocessors, the default fx variables in the second column are automatically
used. If given, the ``fx_variables`` argument specifies the fx variables that
the user wishes to input to the corresponding preprocessor function. The user
may specify these by simply adding the names of the variables, e.g.,

.. code-block:: yaml

fx_variables:
areacello:
volcello:

or by additionally specifying further keys that are used to define the fx
datasets, e.g.,

.. code-block:: yaml

fx_variables:
areacello:
mip: Ofx
exp: piControl
volcello:
mip: Omon

This might be useful to select fx files from a specific ``mip`` table or from a
specific ``exp`` in case not all experiments provide the fx variable.

Alternatively, the ``fx_variables`` argument can also be specified as a list:

.. code-block:: yaml

fx_variables: ['areacello', 'volcello']

or as a list of dictionaries:

.. code-block:: yaml

fx_variables: [{'short_name': 'areacello', 'mip': 'Ofx', 'exp': 'piControl'}, {'short_name': 'volcello', 'mip': 'Omon'}]

The recipe parser will automatically find the data files that are associated
with these variables and pass them to the function for loading and processing.

If ``mip`` is not given, ESMValCore will search for the fx variable in all
available tables of the specified project.

.. warning::
Some fx variables exist in more than one table (e.g., ``volcello`` exists in
CMIP6's ``Odec``, ``Ofx``, ``Omon``, and ``Oyr`` tables; ``sftgif`` exists
in CMIP6's ``fx``, ``IyrAnt`` and ``IyrGre``, and ``LImon`` tables). If (for
a given dataset) fx files are found in more than one table, ``mip`` needs to
be specified, otherwise an error is raised.

.. note::
To explicitly **not** use any fx variables in a preprocessor, use
``fx_variables: null``. While some of the preprocessors mentioned above do
work without fx variables (e.g., ``area_statistics`` or ``mask_landsea``
with datasets that have regular latitude/longitude grids), using this option
is **not** recommended.

Internally, the required ``fx_variables`` are automatically loaded by the
preprocessor step ``add_fx_variables`` which also checks them against CMOR
standards and adds them either as ``cell_measure`` (see `CF conventions on cell
measures
<https://cfconventions.org/cf-conventions/cf-conventions.html#cell-measures>`_
and :class:`iris.coords.CellMeasure`) or ``ancillary_variable`` (see `CF
conventions on ancillary variables
<https://cfconventions.org/cf-conventions/cf-conventions.html#ancillary-data>`_
and :class:`iris.coords.AncillaryVariable`) inside the cube data. This ensures
that the defined preprocessor chain is applied to both ``variables`` and
``fx_variables``.

Note that when calling steps that require ``fx_variables`` inside diagnostic
scripts, the variables are expected to contain the required ``cell_measures`` or
``Fx variables as cell measures or ancillary variables``. If missing, they can be added using the following functions:

.. code-block::

from esmvalcore.preprocessor import (add_cell_measure, add_ancillary_variable)

cube_with_area_measure = add_cell_measure(cube, area_cube, 'area')

cube_with_volume_measure = add_cell_measure(cube, volume_cube, 'volume)

cube_with_ancillary_sftlf = add_ancillary_variable(cube, sftlf_cube)

cube_with_ancillary_sftgif = add_ancillary_variable(cube, sftgif_cube)

Details on the arguments needed for each step can be found in the following sections.

.. _Vertical interpolation:

Vertical interpolation
Expand Down
Loading