Skip to content

Configure fixes instead of programming them #755

Description

@bouweandela

At the moment fixes are programmed. This has two drawbacks:

  1. The threshold to contributing a new fix is high, as it requires programming skills and the time and ability to implement unit tests. This leads to a low number of contributions for fixes (people fix stuff on their own computer but do not share) and the fixes that are contributed, often do not make it past the pull request stage.
  2. It does not scale well, as applying the same fix to multiple datasets leads to code duplication. Even if we would manage to scan the entire CMIP6 archive regularly for problems, implementing fixes for those problems would be difficult to automate as it would require code generation.

Fixes could be made configurable instead. This would have the advantage that once a fix function is implemented, it can be easily re-used across many different variables and datasets. Because we use yaml files for all our configuration, that would probably be the best format to use.

The idea for configurable fixes was presented to us by @agstephens from the C3S 34e project. An example (work in progress) of this idea is in the data characterization and data fixing libraries developed in that project.

Metadata

Metadata

Assignees

Labels

cmorRelated to the CMOR standardfix for datasetRelated to dataset-specific fix files

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions