Preserve deferred loading of netCDF aux coords.

[rhattersley]

Fix for #1287.

(Replaces #1290.)

[rhattersley]

@ocefpaf

the problem is just delayed. Right?

Yes. 😒

It fixes the issue with load_cube being slower than load_raw, but the deferred loading doesn't survive the indexing operation. Looks like I'll have to go the whole hog and use biggus instead...

[ocefpaf]

Here is the test:

http://nbviewer.ipython.org/gist/ocefpaf/e8f60b7eb040b1aee0e7

Is there a way to slice without creating the auxiliary coordinate? A more elegant solution would be for aux_factory.py to create the coordinate just for the sliced data. (Sorry I cannot be more helpful, but some parts of iris code base are way beyond my limited python knowledge.)

[rhattersley]

Is there a way to slice without creating the auxiliary coordinate?

Biggus 😁

[rhattersley]

Hopefully #1293 will get travis working.

[rhattersley]

@ocefpaf (and @rsignell-usgs) ... the latest commit should get us pretty close.

[ocefpaf]

Yes, things are back to normal (with the exception of the travis failures above 😄). Thanks a lot!

http://nbviewer.ipython.org/gist/ocefpaf/e8f60b7eb040b1aee0e7

PS: Just read what @pelson wrote on biggus to try to understand what is happing in the code. Amazing stuff!

[rhattersley]

I've rebased to incorporate #1293 with the aim of getting the tests to pass.

[rhattersley]

Travis says 👍 😄

[esc24]

Super. I'm on it.

[esc24]

It would be good to add the dtype to the repr.

[esc24]

On reflection - we shouldn't be changing the repr in a bug fix release. Similarly, we shouldn't be changing the API of LazyArray. I'd be more comfortable putting this onto master. If you really want in 1.7.x, then we could make the dtype a keyword arg, but that doesn't make a lot of sense to me.

[pelson]

Whilst I agree that these changes are excellent, they are not in the spirit of a patch release (v1.7.2). That is not to say that this is not something we should do (I desperately want to see this problem resolved) - it is just that the concept of having a LTS point release is a little broken IMHO...

[esc24]

Not sure about this. How about instantiating a new one and setting _array manually?

[esc24]

I can sympathise with @pelson's view that this is a big change to put into 1.7.x. I can justify it to myself in terms as a bug fix to the LTS release as I see it fixing a recently added, unintended side-effect. However, the change to a public API concerns me.

[rsignell-usgs]

Don't rush on my account I have conda module for Wakari and I'm happy for
now

On Thursday, August 28, 2014, Ed Campbell notifications@github.com wrote:

I can sympathise with @pelson https://github.com/pelson's view that
this is a big change to put into 1.7.x. I can justify it to myself in terms
as a bug fix to the LTS release as I see it fixing a recently added,
unintended side-effect. However, the change to a public API concerns me.

—
Reply to this email directly or view it on GitHub
#1291 (comment).

Dr. Richard P. Signell (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598

[rhattersley]

this is a big change to put into 1.7.x

Agreed - it's too big. From a semantic version point of view, it doesn't fit the criteria for a patch release.

I've removed some unnecessary code (related to pickle/copy) which helps, but there is still the issue of the new dtype argument to LazyArray. Possibly the ability of an AuxCoord to accept a biggus Array is also a problem.

I'm less and less convinced that the goal of deferred slicing is compatible with a patch release.

[rhattersley]

I'm less and less convinced that the goal of deferred slicing is compatible with a patch release.

@rsignell-usgs - would it cause you a problem if this was targetted at v1.8 instead?

[rsignell-usgs]

@rhattersley No problem for v1.8. I can use the conda module built from this pull request until then.

[pelson]

would it cause you a problem if this was targetted at v1.8 instead?

Would it cause a problem to release a v1.8 early? 😉

[rsignell-usgs]

👍

[ocefpaf]

@rhattersley I tried to using the latest iris checkout and notice that things are still slow. Did this get merged somewhere else? Or is this the only PR currently implementing this?

[rhattersley]

is this the only PR currently implementing this?

Yes, sorry. 😒 It needs re-targeting back at master. It slipped off my urgent to-do list because @rsignell-usgs had his own conda build.

[ocefpaf]

I know (I built it 😉) but things are getting a little awkward as iris evolves and we are still relying on an old branch just for that feature.

[rhattersley]

things are getting a little awkward

Right then - I'll have to see if I can make progress on it then! (Unless you're about to volunteer ... 😉)

[ocefpaf]

I can try to copy what you did here on a fresh master checkout and create a new PR. But I am afraid that my "git mergetool" is not so good to rebase this PR . (I know because I tried 😬 and got lost with the changes.) Is that OK for you? If so I will start right away.

[rhattersley]

Is that OK for you?

Yes - that's fine. Feel free to do whatever is easiest!

[ocefpaf]

Working on it...

[ocefpaf]

Done #1502.

[rhattersley]

Now handled by #1502.