Skip to content

Tutorial for parameter sweep with napari#67

Closed
ruhmamehek wants to merge 3 commits intonapari:mainfrom
ruhmamehek:master
Closed

Tutorial for parameter sweep with napari#67
ruhmamehek wants to merge 3 commits intonapari:mainfrom
ruhmamehek:master

Conversation

@ruhmamehek
Copy link

Added tutorial with examples for using dask and magicgui to perform parameter sweep with napari, as suggested in issue #43

I have added three examples for the same:

  • Lazy parameter sweep for a 2D periodic function using dask
  • Parameter sweep using napari to visualize thresholding of an image using adaptive, Sauvola, and Niblack techniques from the scikit-image library for a range of values of block_size and window_size
  • Adding apply_threshold method as a magicgui widget to napari for visualizing the thresholded image for the technique selected from the drop-down menu, and the value of block_size or window_size passed via the slider.

Also added relevant assets to demonstrate all examples to the assets/tutorials folder.

Added tutorial with examples for using dask and magicgui to perform parameter sweep with napari. Added relevant assets to the assets/tutorials folder.
Copy link
Contributor

@tlambert03 tlambert03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the great tutorial! I've left quite a bit of feedback, but please don't take that as anything less than great enthusiasm for your contribution :) ... I'd just like to tweak some of the wording and simplify some of the examples a bit where possible without changing the end result.

I think the major points for the code come down to this:

  • the second example isn't lazy. You don't actually say it is anywhere, but with the emphasis on lazy evaluation in the intro, and the presence of dask arrays in the second example, a reader might easily assume that it is still lazy. So I would either point that out explicitly... or just go the extra distance and make it lazy! 😄
  • the magicgui example is great, but can be further simplified quite a bit since napari/napari#981 added most of that functionality directly to napari.

please let me know if you have any questions and I look forward to reviewing the next version!

@@ -0,0 +1,229 @@
# Parameter sweep in napari using Dask and magicgui

A parametric sweep allows for a parameter to be swept through a range of user-defined values.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A parametric sweep allows for a parameter to be swept through a range of user-defined values.
A parametric sweep allows for a parameter to be continuously varied through a range of user-defined values.

(try not to use the same word in the definition of that word)

# Parameter sweep in napari using Dask and magicgui

A parametric sweep allows for a parameter to be swept through a range of user-defined values.
Using napari to perform parameter sweep, allows the user to call a method with different parameter values, by moving the slider across the axis.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Using napari to perform parameter sweep, allows the user to call a method with different parameter values, by moving the slider across the axis.
By using `napari` to perform a parameter sweep, the user may call a method with different parameter values (by moving a slider in napari), while observing the effect in the viewer.

A parametric sweep allows for a parameter to be swept through a range of user-defined values.
Using napari to perform parameter sweep, allows the user to call a method with different parameter values, by moving the slider across the axis.

This tutorial is designed to help users visualise parameter sweep using napari, dask and magicgui. We start with the example of _lazy parameter sweep_ for a 2D parametric function and then proceed to real-world examples.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This tutorial is designed to help users visualise parameter sweep using napari, dask and magicgui. We start with the example of _lazy parameter sweep_ for a 2D parametric function and then proceed to real-world examples.
This tutorial is designed to help users visualise parameter sweeps using napari, [Dask](https://dask.org/) and [magicgui](https://magicgui.readthedocs.io/). We start with the example of _lazy parameter sweep_ for a 2D parametric function and then proceed to real-world examples.


This tutorial is designed to help users visualise parameter sweep using napari, dask and magicgui. We start with the example of _lazy parameter sweep_ for a 2D parametric function and then proceed to real-world examples.

The python library, Dask, allows us to perform our tasks _lazily_. This means that rather than immediately performing tasks that the function has to perform, it records the tasks and computes them as and when required.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The python library, Dask, allows us to perform our tasks _lazily_. This means that rather than immediately performing tasks that the function has to perform, it records the tasks and computes them as and when required.
The python library, Dask, allows us to perform our tasks [_lazily_](https://en.wikipedia.org/wiki/Lazy_evaluation). This means that rather than immediately performing the tasks that a function has to perform, it records the tasks and computes them only when required.

from itertools import product
```

We then define a 2D periodic function by generating 200 values between -5π and +5π, and take their sine, and cosine, as illustrated below:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
We then define a 2D periodic function by generating 200 values between -5π and +5π, and take their sine, and cosine, as illustrated below:
We then define a 2D periodic function by generating 200 values between -5π and +5π, and take their sine and cosine, as illustrated below:


## 3. Parameter Sweep with napari using the magicgui widget

We can use the magicgui widget along with napari to better visualize the different thresholding techniques. Here, the thresholded image is computed only for the layer and thresholding technique selected from the drop-down menus, for the value of `block_size` or `window_size` passed via the slider to the `apply_threshold` method.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
We can use the magicgui widget along with napari to better visualize the different thresholding techniques. Here, the thresholded image is computed only for the layer and thresholding technique selected from the drop-down menus, for the value of `block_size` or `window_size` passed via the slider to the `apply_threshold` method.
To auto-generate a user interface that lets us explore different thresholding techniques, we can use [magicgui](https://magicgui.readthedocs.io/en/latest/). Here, the thresholded image is computed only for the layer and thresholding technique selected from the drop-down menus; and the value of `block_size` or `window_size` for the current threshold method is controlled by a slider.

Comment on lines +145 to +168
Next, we define the methods `get_layers` and `show_layer_result` to obtain the layers in the napari viewer, as well as add any new layers to display the result of thresholding:
```python
def get_layers(gui, layer_type):
try:
viewer = gui.parent().qt_viewer.viewer
return tuple(l for l in viewer.layers if isinstance(l, layer_type))
except AttributeError:
return ()

def show_layer_result(gui, result, return_type) -> None:
if result is None:
return
try:
viewer = gui.parent().qt_viewer.viewer
except AttributeError:
return
try:
viewer.layers[gui.result_name].data = result
except KeyError:
adder = getattr(viewer, f"add_{return_type.__name__.lower()}")
adder(data=result, name=gui.result_name)

register_type(layers.Layer, choices=get_layers, return_callback=show_layer_result)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned in the (recently updated) magicgui tutorial on parameter sweeps, this whole code block will be unnecessary for users once the next version of napari is released (or, if you're installing napari from the github master instead of pip). While it's kinda useful in a magicgui tutorial to see how to register custom types, for a napari-focused crowd/tutorial I think we should hide these implementation details. We are close to a 0.3.0 release which will make this all obsolete. Note though, in order to get the result to show up in the viewer, you'll need to provide a return type annotation... see below.


We then create a viewer with napari, and add the `data.page()` image layer to try out parameter sweep with different thresholding techniques. We use the `magicgui` decorator for the method `apply_threshold` to turn it into a magicgui.

`auto_call = True` indicates magicgui to call `apply_threshold` whenever the value of a parameter changes. The QDoubleSlider is used to pass the size parameter to the function, and we offer the choices to threshold an image layer using the adaptive, Sauvola and Niblack techniques:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`auto_call = True` indicates magicgui to call `apply_threshold` whenever the value of a parameter changes. The QDoubleSlider is used to pass the size parameter to the function, and we offer the choices to threshold an image layer using the adaptive, Sauvola and Niblack techniques:
`auto_call = True` tells magicgui to call the `apply_threshold` function whenever the user changes one of the parameter in the GUI. The `QDoubleSlider` is used to ensure that the size parameter slider is interpreted as a `float` type, and we offer the `choices` to threshold an image layer using the adaptive, Sauvola and Niblack techniques:

Comment on lines +176 to +194
with gui_qt():
# creating a viewer and adding the data.page() image layer
viewer = Viewer()
viewer.add_image(data.page(), name="page")

# passing size as (2* floor(size/2)+1), as block-size and window size have to be odd natural numbers
@magicgui(
auto_call=True,
size={"widget_type": QDoubleSlider, "maximum": 99, "fixedWidth": 400},
technique={"choices": ["adaptive", "niblack", "sauvola"]},
)
def apply_threshold(layer: layers.Image, size=1, technique="adaptive") -> None:
if layer:
if technique=="adaptive":
return layer.data > threshold_local(layer.data, (2* floor(size/2)+1))
elif technique=="sauvola":
return layer.data > threshold_sauvola(layer.data, (2* floor(size/2)+1))
elif technique=="niblack":
return layer.data > threshold_niblack(layer.data, (2* floor(size/2)+1))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the entire example (including the register_type bit above and the show_result bit below) can be reduced to this:

from magicgui import magicgui
from magicgui._qt import QDoubleSlider
from napari import Viewer, gui_qt, layers
from skimage import data, filters
from math import floor

with gui_qt():
    # creating a viewer and adding the data.page() image layer
    viewer = Viewer()
    viewer.add_image(data.page(), name="page")

    # passing size as (2* floor(size/2)+1), as block-size and window size
    # have to be odd natural numbers
    @magicgui(
        auto_call=True,
        size={"widget_type": QDoubleSlider, "maximum": 99, "fixedWidth": 400},
        technique={"choices": ["local", "niblack", "sauvola"]},
    )
    def apply_threshold(layer: layers.Image, size=50, technique="local") -> layers.Image:
        if layer:
            func = getattr(filters, "threshold_" + technique)
            return layer.data > func(layer.data, (2 * floor(size / 2) + 1))

    gui = apply_threshold.Gui()
    gui.parentChanged.connect(gui.refresh_choices)
    viewer.layers.events.changed.connect(lambda x: gui.refresh_choices("layer"))
    viewer.window.add_dock_widget(gui)

Things to point out here:

  • we don't need to register anything for the napari.layers.Layer type on the current master branch of napari (or any future versions), so let's keep that out of this tutorial
  • I made the default size=50 so that the initial image isn't black
  • when you have a lot of functions with similar names and identical APIs like these threshold functions, you can reduce a lot of code using the func = getattr(filters, ...) line here... this also makes it very easy to add a new threshold type simply by adding a new item to the choices list.
  • note that I added a return annotation of -> layers.Image to the apply_threshold function ... that tells magicgui to add an Image layer to the viewer... so we can actually get rid of the show_result callback below.

Comment on lines +206 to +216
The `show_result` method is a callback function to add the thresholded image to the layers. This updates the resultant layer whenever the `apply_threshold` method is called:
```python

def show_result(result):
try:
viewer.layers["thresholded"].data = result
except KeyError:
viewer.add_image(data=result, name="thresholded")

apply_threshold.called.connect(show_result)
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `show_result` method is a callback function to add the thresholded image to the layers. This updates the resultant layer whenever the `apply_threshold` method is called:
```python
def show_result(result):
try:
viewer.layers["thresholded"].data = result
except KeyError:
viewer.add_image(data=result, name="thresholded")
apply_threshold.called.connect(show_result)
```

this can be removed if you add the -> layers.Image: type annotation to the function

- Modified thresholding example, to make it lazy
- Updated magicgui example
@ruhmamehek
Copy link
Author

Hi!

Thank you very much for your suggestions. @BhavyaC16 and I have made the following changes:

  • Made the thresholding example lazy, and also modified the code to remove duplication
  • Updated the magicgui example
  • Provided links to external docs wherever appropriate
  • Tried to limit our text in markdown to 80-100 characters per line
  • Made the suggested changes in wording
  • Updated GIFs demonstrating the examples

Looking forward to your review and suggestions on this. :D

@melissawm
Copy link
Member

Hi folks! Can we do another pass here? If this content is ready, I can move it over to napari/napari (unless @ruhmamehek wants to do it 😄 )

@psobolewskiPhD
Copy link
Member

Ack, boy did this one fall through the cracks which is a massive shame because this is a pretty awesome tutorial both for napari and dask.delayed!
I think if the code examples all still work it's definitely worth porting over to https://github.com/napari/docs

@psobolewskiPhD
Copy link
Member

I've made a task issue in napari/docs to revisit this.
napari/docs#329
Closing, since this won't be merged here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants