netcdf siconc_SImon_FGOALS-g3_historical_r1i1p1f1_gn_185001-201412 {
dimensions:
time = UNLIMITED ; // (1980 currently)
j = 218 ;
i = 360 ;
bnds = 2 ;
strlen = 7 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 0001-01-01 00:00:00" ;
time:calendar = "365_day" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double time_bnds(time, bnds) ;
int j(j) ;
j:units = "1" ;
j:long_name = "cell index along second dimension" ;
int i(i) ;
i:units = "1" ;
i:long_name = "cell index along first dimension" ;
double latitude(j, i) ;
latitude:standard_name = "latitude" ;
latitude:long_name = "latitude" ;
latitude:units = "degrees_north" ;
latitude:missing_value = 1.e+20 ;
latitude:_FillValue = 1.e+20 ;
double longitude(j, i) ;
longitude:standard_name = "longitude" ;
longitude:long_name = "longitude" ;
longitude:units = "degrees_east" ;
longitude:missing_value = 1.e+20 ;
longitude:_FillValue = 1.e+20 ;
.......
longitude =
246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260,
261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274,
275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288,
289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302,
303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316,
317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330,
331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344,
345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358,
359, 360, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
1.00000001504747e+30, 1.00000001504747e+30, 1.00000001504747e+30,
1.00000001504747e+30, 1.00000001504747e+30, 1.00000001504747e+30,
1.00000001504747e+30, 1.00000001504747e+30, 1.00000001504747e+30,
1.00000001504747e+30, 1.00000001504747e+30, 1.00000001504747e+30,
1.00000001504747e+30,
.....
The code then tries to subtract 360 multiple times from 1e30 and is stuck in a near-infinite loop. It does not give an error message - it simply hangs. The previous version of the code without this update processes these files OK.
Or is it better to fix these values model-by-model? This is what i have been doing so far. But even if we do this, perhaps it would be better to have some sort of error check here, so that the code doesn't just hang.
One final comment - it looks like the code reads in all versions of the files, not just the latest version. So even if we ask modelling centres to fix this error, unless you actually delete the old version (which you can't assume in code that should be portable etc) the code will still read it in and get stuck. In the case of the NorESM2-LM file mentioned above, the modelling centre has published a new version without the spurious longitude values v20191108, but esmvaltool reads in both versions, and hangs when it tries to process the old one.
This update to
check.py:(de103fc#diff-4bd932e41bce403396718de759e9c312) (further changed by this update fc95457)
attempts to set longitude values to within 0 to 360 range by repeatedly adding or subtracting 360 from the values. Some files, such as this:
CMIP/CAS/FGOALS-g3/historical/r1i1p1f1/SImon/siconc/gn/v20200402/siconc_SImon_FGOALS-g3_historical_r1i1p1f1_gn_185001-201412.ncor this:
DAMIP/NCC/NorESM2-LM/hist-GHG/r1i1p1f1/SImon/siconc/gn/v20190815/siconc_SImon_NorESM2-LM_hist-GHG_r1i1p1f1_gn_185001-185912.nchave some longitudes and latitudes flagged with a high number like 1e30:
The code then tries to subtract 360 multiple times from 1e30 and is stuck in a near-infinite loop. It does not give an error message - it simply hangs. The previous version of the code without this update processes these files OK.
Would one option would be to change:
to
?
Or is it better to fix these values model-by-model? This is what i have been doing so far. But even if we do this, perhaps it would be better to have some sort of error check here, so that the code doesn't just hang.
One final comment - it looks like the code reads in all versions of the files, not just the latest version. So even if we ask modelling centres to fix this error, unless you actually delete the old version (which you can't assume in code that should be portable etc) the code will still read it in and get stuck. In the case of the NorESM2-LM file mentioned above, the modelling centre has published a new version without the spurious longitude values v20191108, but esmvaltool reads in both versions, and hangs when it tries to process the old one.