Improve user friendliness of data finding (missing data)

[bouweandela]

At the moment, missing data is not reported in a very user friendly way. The following improvements would be good to have

• report all missing files at once instead of stopping at the first file not found
• do not stop if input files are missing for tasks that are not selected for processing with the --diagnostics command line option
• make the locations where the tool looked for data more clear, now they are in main_log_debug.txt (see Enhance error when input files are not found #98)
• make it more clear where (which variable / diagnostic script of which diagnostic) in the recipe the missing files are defined

[bouweandela]

Related to #31

[stefsmeets]

The missing data error has been a huge showstopper for me recently. It is too difficult to figure out what is missing, where it is looking, and what it is actually looking for. Probably all the information is there, but it is dispersed over a few places and difficult to parse.

I think part of the problem that makes this difficult is that the data are defined by both the dataset and the variable. It would be useful if ESMValTool could spit out the dataset entry that is causing problems.

I think the best way to address this is to improve the Python error message. This can then be fed into the logging so we do not repeat ourselves.