use imageio for input/output tasks?

[tobyhodges]

Opening this issue so that there is an obvious place to find this discussion in future.

In #238, @mkcor wrote:

Getting to the gist of this PR, I'm following up on scikit-image/scikit-image#5036 (comment): The vision for scikit-image is to focus on actual image processing, delegating other functionalities (e.g., input/output, drawing, ...) to dedicated libraries. For reading and writing image data, this dedicated library is now available and stable, it is imageio.

Therefore, I'm thinking that, before releasing this lesson as 'stable,' we may want to consider making this change. If you agree with it, I'll propagate it to all subsequent episodes (episode 03 representing the biggest chunk). From an educational perspective, I can see how we may prefer sticking with only one library instead of two. I would argue that the proposed change doesn't add a dependency (you can readily import imageio once you have installed scikit-image); so the setup wouldn't be affected.

I could add a sentence to introduce the imageio library, if you deem it appropriate; the existing sentence seems almost sufficient and works well with the proposed change: "Let's load our image data from disk using the imread function from the imageio.v3 library and display it using ..."

Marianne tagged the other scikit-image maintainers, @stefanv @jni @alexdesiqueira @grlee77 @lagru, in that pull request so I am doing the same here.

Here is a list of pros and cons I can identify for making the change:

➕ reflects preferences of the scikit-image project
➕ reflects best practice for the scikit-image user community
➕ seems to require only very minor changes to function & argument names, no syntax changes
➕ those new names are no less intuitive than the skimage.io equivalents
➕ new dependency requires no additional setup steps

➖ introduces a new library that needs to be explained in the lesson
➖ the need to specifically import v3 implies the existence of (at least) a v2 and v1, which may need to be explained - why is it necessary to import imageio.v3 as iio and not "just" imageio as iio as I suspect many new users would try to do?
➖ learners have to develop a mental model of when it is appropriate to use imageio vs skimage

Overall, I am in favour of making this change. However, in addition to the adjustments @mkcor has already proposed in the #238, I think that we need more context for the new library. A few sentences explaining:

• what imageio is (and isn't) for,
• why you need to import v3 specifically,
• and what the typical workflow might look like when switching between the two libraries
  • e.g. is it correct to think of it as "load image using a function from imageio, process it using functionality from skimage, then save any resulting images with another function from imageio"?

I'll leave this (and #238) open for a few days, to give @K-Meech @bobturneruk @quist00 and @uschille a chance to join the discussion.

[mkcor]

why you need to import v3 specifically

Admittedly I wasn't aware of Image IO v3 until I submitted #238. I had always done import imageio as io until very recently (for example, in my tutorial at Data Umbrella on October 25th [1]). And, since I would call function volread to read 'volumetric' data, I hadn't come across this deprecation warning for imread:

DeprecationWarning: Starting with ImageIO v3 the behavior of this function will switch to that of iio.v3.imread. To keep the current behavior (and make this warning disappear) use `import imageio.v2 as imageio` or call `imageio.v2.imread` directly.

Also, it turns out I was on maternity leave when the imageIO warnings (due to v2 -> v3 migration) were reported [2]!

My understanding from the migration guide [3] is that the current better practice is to import imageio.v3 as iio but I find it unfortunate from a teaching viewpoint, indeed. Maybe @almarklein or @FirefoxMetzger could recommend a workaround?

[1] https://www.youtube.com/watch?v=NqdhuU1fX5A
[2] scikit-image/scikit-image#6341
[3] https://imageio.readthedocs.io/en/stable/reference/userapi.html#migration-from-v2

[FirefoxMetzger]

My understanding from the migration guide [3] is that the current better practice is to import imageio.v3 as iio but I find it unfortunate from a teaching viewpoint, indeed. Maybe @almarklein or @FirefoxMetzger could recommend a workaround?

@mkcor I must admit that I am lacking a bit of context here. Could you elaborate on how/what is unfortunate about this import?

[mkcor]

Hi @FirefoxMetzger!

Thanks for your response.

I mean 'unfortunate' in the sense that, in addition to introducing imageio (which, let's agree on this for now, is fine), the teacher will have to explain that imageio has a v2 API and a v3 API, and we happen to use the v3 API, and why so... Because it's the latest, I guess it would be a sufficient reason: Can we afford to explain that it supports nD natively?

It's all super interesting and, personally, as a teacher, I tend to make digressions... But, generally speaking, we want to avoid any unnecessary cognitive load in a lesson (which is going to be officially part of The Carpentries curriculum).

[mkcor]

PS: @FirefoxMetzger I just thought of the following: Can we expect the deprecation warning to be gone... soon?

@tobyhodges if, by miracle, that timing could coincide with the 'release' of the lesson ('stable' status), we could simply use import imageio as iio and iio.imread(...), hence getting rid of one of the downsides for the proposed change.

[stefanv]

I don't see the point in deprecating skimage.io. It makes for a very simple interface that users are already familiar with, and has such a low maintenance burden. We can get rid of the plugin system, if that feels necessary.

[FirefoxMetzger]

@mkcor Right. I would like to explain this here from two angles: first from the dev perspective (interesting for you/us so we have the necessary context) and then from the user perspective (interesting for students learning about image processing).

From the dev perspective, the vX formalism kills two birds with one stone.

Firstly, it gives us more wiggle room when making backward-incompatible improvements. We could easily add a imageio.v4 with some experimental changes and release it without breaking any production code that uses imageio.v3. That is very useful because we can get user feedback early since there are people using the library in ways that we/I would have never imagined.

Secondly, it gives a clear migration path for when you do remove things after a deprecation cycle. People can immediately use ìmageio>=3.X and postpone migration until they have time (via import imageio.v2 as imageio). At the same time, they can continue to update ImageIO to get security updates, bug fixes, or new backend features. This beats the alternative approach of starting a new repo/library for a version-major change where you then have to import imageio3 (which looks suspiciously like import imageio.v3 as iio except that the latter is more explicit). The new repo approach always risks losing/splitting your community, because they stay with the old repo and don't migrate with you.

Now, from the user's side, the main benefit is that you are forced to explicitly version your dependency. If you are a scientist that writes a script to analyze some data, chances are you are not going to track all the versions of all the dependencies you are using. You should, but that doesn't mean it happens in practice - unfortunately. If another person (or future you) wants to use your script 5 years down the line and sees import imageio.v3 then that will correspond to installing imageio<4.0.

The other benefits for the user mirror the benefits on the dev side. You can try out the future version of the API early (import imageio.v4 as iio) and enjoy some of the new improvements even if some of your codebase still contains legacy code.

Can we afford to explain that it supports nD natively?

The direct answer is yes, one of the exciting things in v3 is that you can more easily deal with ndimages, which allows you to imread/imwrite all the things. You also get a few other improvements like improps which gives you standardized metadata.

I'm still not sure I get the underlying question though. You've also asked for a workaround in the previous post, so I'm guessing that you are looking for a short way to explain why we use import imageio.v3 as iio instead of import imageio as iio?

[FirefoxMetzger]

Right, I have just read through the entire issue and I think I get it now. Does the following help?

PS: @FirefoxMetzger I just thought of the following: Can we expect the deprecation warning to be gone... soon?

My goal was to release it this month, but then I got delayed due to an ongoing job hunt (I finally graduated from my PhD 🚀). I might be able to release it towards the end of the month, but I am reluctant to do so, given the proximity to the Christmas holidays. People rush out final features before the end of the year and I don't want to make that period more complicated by applying deprecations. I guess a good new estimate is mid-January.

introduces a new library that needs to be explained in the lesson

The way we have traditionally explained this in class is that ImageIO allows you to read images using iio.imread and to write images using iio.imwrite. That is of course high-level and you could add much more details, but it is a very good start that solves 80% of what people typically do with the library.

the need to specifically import v3 implies the existence of (at least) a v2 and v1, which may need to be explained - why is it necessary to import imageio.v3 as iio and not "just" imageio as iio as I suspect many new users would try to do?

The same reason why you do import matplotlib.pyplot as plt. You select the high-level API you want to use to interact with ImageIO. (And also document which version-major you use, so that other people can replicate your work in the future. Even if you didn't keep a requirements.txt 👼)

learners have to develop a mental model of when it is appropriate to use imageio vs skimage

Yes, I would have classified it as @tobyhodges suggested. ImageIO is for reading/writing (specifically for encoding and decoding image and video data from various sources). Skimage is for processing (specifically transforming image data into something useful).

[mkcor]

I don't see the point in deprecating skimage.io. It makes for a very simple interface that users are already familiar with, and has such a low maintenance burden. We can get rid of the plugin system, if that feels necessary.

@stefanv thanks for chiming in. I think that, whether we deprecate skimage.io or not, introducing imageio in this lesson has educational value. As @FirefoxMetzger puts it:

ImageIO is for reading/writing (specifically for encoding and decoding image and video data from various sources). Skimage is for processing (specifically transforming image data into something useful).

[stefanv]

I suppose the question I ask when teaching is: what is the added benefit to the learner. For beginners, anything more than io.imread is noise. For more sophisticated users, having additional capabilities become useful.

Along those lines, it would make sense to make it clear in the skimage docs that imageio is used under the hood, and is recommended for more sophisticated data handling (I think we have some of that language in place, but could expand it further).

[mkcor]

@FirefoxMetzger thanks for your thorough replies.

From the dev perspective, the vX formalism kills two birds with one stone.

I'm absolutely sold! In particular, I'm a big fan of the following:

Secondly, it gives a clear migration path for when you do remove things after a deprecation cycle.

[snip]

Now, from the user's side, the main benefit is that you are forced to explicitly version your dependency.

I'm sold as well. The only issues being raised here are from... the teacher's side (to oversimply), in a specific setting: I've never meant to question the development choices or pathways made at imageio!

Thanks for the write-up anyway: This way I could catch up on this migration episode I had missed; plus, your project is an inspiration... (thinking of https://discuss.scientific-python.org/t/a-pragmatic-pathway-towards-skimage2/530) 👀

The same reason why you do import matplotlib.pyplot as plt.

Good point!

[mkcor]

I suppose the question I ask when teaching is: what is the added benefit to the learner. For beginners, anything more than io.imread is noise.

Yes, that's what I meant with "we want to avoid any unnecessary cognitive load in a lesson." And here learners are beginners, I'm afraid (they are not absolute beginners in Python though: https://datacarpentry.org/image-processing/prereqs/index.html).

@FirefoxMetzger that's why I asked "Can we afford to explain that it supports nD natively?" because it adds yet another conceptual box for the learner to process.

For more sophisticated users, having additional capabilities become useful.

@tobyhodges do we consider 'more sophisticated users' among the learners?

[mkcor]

My goal was to release it this month, but then I got delayed due to an ongoing job hunt (I finally graduated from my PhD 🚀).

@FirefoxMetzger congratulations!! 👏

I might be able to release it towards the end of the month, but I am reluctant to do so, given the proximity to the Christmas holidays. People rush out final features before the end of the year and I don't want to make that period more complicated by applying deprecations. I guess a good new estimate is mid-January.

Good to know. Thank you so much!

[FirefoxMetzger]

For beginners, anything more than io.imread is noise.

So we are essentially debating

from skimage import io

img = io.imread(...)

versus

import imageio.v3 as iio

img = iio.imread(...)

This is my personal opinion only, but I think the difference is rather small; perhaps either option can work just fine?

On a tangent, we have been teaching ImageIO for the past 2 years now, and so far I didn't get any questions about why we import imageio.v3 and not import imageio. My guess is that most of our students just accepted that this is how you import the library without questioning it.