Attempt concatenation when cubes have overlapping data#280
Conversation
|
I was the one that added the documentation, I will do my penitence adding tests for this feature |
|
hahaha no worries @jvegasbsc am almost finished writing tests 🍺 |
|
It is already done |
|
I have to add #259 to make the tests pass, as I was using 0:00 hours in the time |
|
sweeet! I was just about to commit, but not no more, cheers Javi 🍺 |
|
@jvegasbsc I fixed your fix, man 😁 |
mathause
left a comment
mathause
left a comment
There was a problem hiding this comment.
I am a bit torn - it is nice functionality but quite magical - i.e. unexpected/ unwanted things can happen silently. It would be nice if the breakpoint can [optionally] be set by a keyword argument.
-
Example 1:
- {dataset: MIROC5, project: CMIP5, mip: Amon, exp: [historical, rcp85], ensemble: r1i1p1, start_year: 1950, end_year=2099}
This works beautifully - the historical data goes to 2012, rcp85 starts in 2006 & it concatenates the way I want - historical to 2005 and rcp85 from 2006. -
Example 2:
- {dataset: CMCC-CESM, project: CMIP5, mip: Amon, exp: [historical, rcp85], ensemble: r1i1p1, start_year: 1950, end_year=2099}
Here, historical goes to 2005, but rcp85 starts in 2000. So I get historical until 1999 and rcp85 starts in 2000 & I don't even realize this happens.
|
@mathause cheers muchly for testing!! good to hear you are getting the (mostly) desired output, yeah, I'll have a look at the changes next week, off to pub now 🍺 |
|
cheers for suggestions @mathause - put them in. Given you tested (cheers a lot for that!), are you happy to go? 🍺 |
|
Could not get to work right now, have to come back on monday... |
|
OK @bouweandela debug messages added - now it should be very clear what data is being used and how 🍺 Can we go ahead pls 🎖️ |
Co-Authored-By: Bouwe Andela <bouweandela@users.noreply.github.com>
Co-Authored-By: Bouwe Andela <bouweandela@users.noreply.github.com>
|
@valeriupredoi I made some minor changes, can you check if you're still happy with the result? |
bouweandela
left a comment
bouweandela
left a comment
There was a problem hiding this comment.
@mattiarighi Could you please test?
valeriupredoi
left a comment
valeriupredoi
left a comment
There was a problem hiding this comment.
good stuff, cheers @bouweandela - let's merge this then 🍺
| f"ends before %s", cubes[0], cubes[1]) | ||
| logger.debug(f"Cube %s contains all needed data " | ||
| f"so using it fully", cubes[1]) | ||
| if data_end_1 <= data_end_2: |
|
@mattiarighi you wanna test pls mate? 🍺 |
6 participants
Better concatenation for two cubes that have time overlapping data and identical metadata - solves https://github.com/ESMValGroup/ESMValTool/issues/1329 and the olde issue with overlapping CMIP data (ie 1850-2000 and 1850-2300 both files in the same repo)
BTW whoever implemented the
exp: listfunctionality *should have tested properly or added info to documentation, at the moment it's completely useless 🍺Tasks
If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.
Closes #349 and possibly (untested) #54