Merge pull request #1011 from qin-yu/bioimageio-docs

Add docs for BioImage.IO: finding & sharing models

Author: carsen-stringer

cellpose/export.py

@@ -16,7 +16,7 @@
 DEFAULT_MODEL_NAME="My Cool Cellpose"
 DEFAULT_MODEL_DOCUMENTATION="A cool Cellpose model trained for my cool dataset."
 DEFAULT_MODEL_AUTHORS='[{"name": "Qin Yu", "affiliation": "EMBL", "github_user": "qin-yu", "orcid": "0000-0002-4652-0795"}]'
-DEFAULT_MODEL_CITE='[{"text": "For more details of the model itself, see the manuscript", "doi": "10.1101/2024.02.19.580954", "url": null}]'
+DEFAULT_MODEL_CITE='[{"text": "For more details of the model itself, see the manuscript", "doi": "10.1242/dev.202800", "url": null}]'
 DEFAULT_MODEL_TAGS="cellpose 3d 2d"
 DEFAULT_MODEL_LICENSE="MIT"
 DEFAULT_MODEL_REPO="https://github.com/kreshuklab/go-nuclear"
@@ -308,7 +308,7 @@ def parse_args():
     parser.add_argument("--model_name", required=True, type=str, help="Model name, e.g., My Cool Cellpose")
     parser.add_argument("--model_documentation", required=True, type=str, help="Model documentation, e.g., A cool Cellpose model trained for my cool dataset.")
     parser.add_argument("--model_authors", required=True, type=str, help="Model authors in JSON format, e.g., '[{\"name\": \"Qin Yu\", \"affiliation\": \"EMBL\", \"github_user\": \"qin-yu\", \"orcid\": \"0000-0002-4652-0795\"}]'")
-    parser.add_argument("--model_cite", required=True, type=str, help="Model citation in JSON format, e.g., '[{\"text\": \"For more details of the model itself, see the manuscript\", \"doi\": \"10.1101/2024.02.19.580954\", \"url\": null}]'")
+    parser.add_argument("--model_cite", required=True, type=str, help="Model citation in JSON format, e.g., '[{\"text\": \"For more details of the model itself, see the manuscript\", \"doi\": \"10.1242/dev.202800\", \"url\": null}]'")
     parser.add_argument("--model_tags", nargs='+', required=True, type=str, help="Model tags, e.g., cellpose 3d 2d")
     parser.add_argument("--model_license", required=True, type=str, help="Model license, e.g., MIT")
     parser.add_argument("--model_repo", required=True, type=str, help="Model repository URL")

docs/models.rst

@@ -1,51 +1,50 @@
 Models
-------------------------------
+-------------------------------
 
 ``from cellpose import models``
 
-Each model will be downloaded automatically to your ``models.MODELS_DIR`` 
-(see Installation instructions for more details on MODELS_DIR). 
-You can also directly download a model by going to the URL, e.g.:
+Each model will be downloaded automatically to your ``models.MODELS_DIR`` (see
+Installation instructions for more details on MODELS_DIR). You can also directly
+download a model by going to the URL, e.g.:
 
 ``https://www.cellpose.org/models/MODEL_NAME``
 
 All built-in models were trained with the ROIs resized to a diameter of 30.0
-(``diam_mean = 30``), 
-except the `'nuclei'` model which was trained with a diameter of 17.0 
-(``diam_mean = 17``). User-trained models will be trained with the same ``diam_mean`` 
-as the model they are initalized with.
-The models will internally take care of rescaling the images given a 
-user-provided diameter (or with the diameter from 
-auto-diameter estimation in full models).
+(``diam_mean = 30``), except the `'nuclei'` model which was trained with a
+diameter of 17.0 (``diam_mean = 17``). User-trained models will be trained with
+the same ``diam_mean`` as the model they are initalized with. The models will
+internally take care of rescaling the images given a user-provided diameter (or
+with the diameter from auto-diameter estimation in full models).
 
 Full built-in models
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-These models have Cellpose model weights and a size model. This means you can 
-run with ``diameter=0`` or ``--diameter 0`` and the model can estimate the ROI size. 
-However, we recommend that you set the diameter for your ROIs rather than having Cellpose 
-guess the diameter.
+These models have Cellpose model weights and a size model. This means you can
+run with ``diameter=0`` or ``--diameter 0`` and the model can estimate the ROI
+size. However, we recommend that you set the diameter for your ROIs rather than
+having Cellpose guess the diameter.
 
-These models can be loaded and used in the notebook with ``models.Cellpose(model_type='cyto3')`` 
-or in the command line with ``python -m cellpose --pretrained_model cyto3``.
+These models can be loaded and used in the notebook with
+``models.Cellpose(model_type='cyto3')`` or in the command line with ``python -m
+cellpose --pretrained_model cyto3``.
 
-We have a ``nuclei`` model and a super-generalist ``cyto3`` model. There are also two 
-older models, ``cyto``, which is trained on only the Cellpose training set, and ``cyto2``,
-which is also trained on user-submitted images.
+We have a ``nuclei`` model and a super-generalist ``cyto3`` model. There are
+also two older models, ``cyto``, which is trained on only the Cellpose training
+set, and ``cyto2``, which is also trained on user-submitted images.
 
-FYI we are no longer using the 4 different versions and ``--net_avg`` is deprecated.
+FYI we are no longer using the 4 different versions and ``--net_avg`` is
+deprecated.
 
 Cytoplasm model (``'cyto3'``, ``'cyto2'``, ``'cyto'``)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The cytoplasm models in cellpose are trained on two-channel images, where 
-the first channel is the channel to segment, and the second channel is 
-an optional nuclear channel. Here are the options for each:
-1. 0=grayscale, 1=red, 2=green, 3=blue 
-2. 0=None (will set to zero), 1=red, 2=green, 3=blue
+The cytoplasm models in cellpose are trained on two-channel images, where the
+first channel is the channel to segment, and the second channel is an optional
+nuclear channel. Here are the options for each: 1. 0=grayscale, 1=red, 2=green,
+3=blue 2. 0=None (will set to zero), 1=red, 2=green, 3=blue
 
-Set channels to a list with each of these elements, e.g.
-``channels = [0,0]`` if you want to segment cells in grayscale or for single channel images, or
+Set channels to a list with each of these elements, e.g. ``channels = [0,0]`` if
+you want to segment cells in grayscale or for single channel images, or
 ``channels = [2,3]`` if you green cells with blue nuclei.
 
 The `'cyto3'` model is trained on 9 datasets, see the `Cellpose3 paper <https://www.biorxiv.org/content/10.1101/2024.02.10.579780v2>`_ 
@@ -57,27 +56,28 @@ The size models are ``size_cytotorch_0.npy``, ``size_cyto2torch_0.npy``, and ``s
 Nucleus model (`'nuclei'`)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The nuclear model in cellpose is trained on two-channel images, where 
-the first channel is the channel to segment, and the second channel is 
-always set to an array of zeros. Therefore set the first channel as 
-0=grayscale, 1=red, 2=green, 3=blue; and set the second channel to zero, e.g.
-``channels = [0,0]`` if you want to segment nuclei in grayscale or for single channel images, or 
-``channels = [3,0]`` if you want to segment blue nuclei.
+The nuclear model in cellpose is trained on two-channel images, where the first
+channel is the channel to segment, and the second channel is always set to an
+array of zeros. Therefore set the first channel as 0=grayscale, 1=red, 2=green,
+3=blue; and set the second channel to zero, e.g. ``channels = [0,0]`` if you
+want to segment nuclei in grayscale or for single channel images, or ``channels
+= [3,0]`` if you want to segment blue nuclei.
 
 The `'nuclei'` model is downloadable from the website with MODEL_NAME ``nucleitorch_0``, 
 and size model ``size_nucleitorch_0.npy``.
 
 Other built-in models
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The main built-in models are dataset-specific models trained on one of the 9 datasets 
-in the Cellpose3 paper. These models do not have a size model.
-If the diameter is set to 0.0, then the model uses the default ``diam_mean`` for the
+The main built-in models are dataset-specific models trained on one of the 9
+datasets in the Cellpose3 paper. These models do not have a size model. If the
+diameter is set to 0.0, then the model uses the default ``diam_mean`` for the
 diameter (``30.0``).
 
-These models can be loaded and used in the notebook with e.g. 
-``models.CellposeModel(model_type='tissuenet_cp3')`` or ``models.CellposeModel(model_type='livecell_cp3')``, 
-or in the command line with ``python -m cellpose --pretrained_model tissuenet_cp3``.
+These models can be loaded and used in the notebook with e.g.
+``models.CellposeModel(model_type='tissuenet_cp3')`` or
+``models.CellposeModel(model_type='livecell_cp3')``, or in the command line with
+``python -m cellpose --pretrained_model tissuenet_cp3``.
 
 The dataset-specific models were trained on the training images from the following datasets: 
     - ``tissuenet_cp3``: `tissuenet dataset <https://datasets.deepcell.org/>`_. 
@@ -99,28 +99,73 @@ We see no improvement with transformer models in the Cellpose3 paper and the
 but the models are available as ``transformer_cp3`` and ``neurips_cellpose_transformer``. These model take three channels as input. 
 The u-net based Neurips challenge model is available as ``neurips_cellpose_default`` and also takes three channels as input.
 
-User-trained models 
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+User-trained models
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 By default, models are trained with the ROIs resized to a diameter of 30.0
-(``diam_mean = 30``) -- this is necessary if you want to start from a pretrained 
-cellpose model. If you want to use a different diameter and use pretraining,
-we recommend performing training yourself on the cellpose dataset with that diameter so the 
-model learns objects at that size. All user-trained models will save the 
-``diam_mean`` so it will be loaded automatically along with the model weights.
-
-Each model also saves the ``diam_labels`` which is the mean diameter of the 
-ROIs in the training images. This value is auto-loaded into the GUI for use 
-with the model, or will be used if the diameter is 0 
-(``diameter=0`` or ``--diameter 0``).
-
-These models can be loaded and used in the notebook with e.g. 
+(``diam_mean = 30``) -- this is necessary if you want to start from a pretrained
+cellpose model. If you want to use a different diameter and use pretraining, we
+recommend performing training yourself on the cellpose dataset with that
+diameter so the model learns objects at that size. All user-trained models will
+save the ``diam_mean`` so it will be loaded automatically along with the model
+weights.
+
+Each model also saves the ``diam_labels`` which is the mean diameter of the ROIs
+in the training images. This value is auto-loaded into the GUI for use with the
+model, or will be used if the diameter is 0 (``diameter=0`` or ``--diameter
+0``).
+
+These models can be loaded and used in the notebook with e.g.
 ``models.CellposeModel(model_type='name_in_gui')``  or with the full path
-``models.CellposeModel(pretrained_model='/full/path/to/model')`` . If you trained in the 
-GUI, you can automatically use the ``model_type`` argument. If you trained in the 
-command line, you need to first add the model to the cellpose path either in the GUI 
-in the Models menu, or using the command line:
-``python -m cellpose --add_model /full/path/to/model``. 
-
-Or these models can be used in the command line with ``python -m cellpose --pretrained_model name_in_gui`` 
-or ``python -m cellpose --pretrained_model /full/path/to/model`` .
+``models.CellposeModel(pretrained_model='/full/path/to/model')`` . If you
+trained in the GUI, you can automatically use the ``model_type`` argument. If
+you trained in the command line, you need to first add the model to the cellpose
+path either in the GUI in the Models menu, or using the command line: ``python
+-m cellpose --add_model /full/path/to/model``.
+
+Or these models can be used in the command line with ``python -m cellpose
+--pretrained_model name_in_gui`` or ``python -m cellpose --pretrained_model
+/full/path/to/model``.
+
+Finding models on BioImage.IO
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+`BioImage.IO <https://bioimage.io/>`_ is a repository for sharing AI models,
+datasets and tools for bioimage analysis. You may look for Cellpose models on
+BioImage.IO Model Zoo by searching for the tag ``cellpose``. To download a
+model, click on the model card, click the download icon, and choose "Download by
+Weight Format" - "Pytorch State Dict".
+
+Sharing models on BioImage.IO
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+You can also share your trained Cellpose models on the BioImage.IO Model Zoo. To
+do this, you need to export your model in the BioImage.IO format using
+``cellpose/export.py`` and then upload the packaged model to the Model Zoo.
+
+Detailed steps:
+
+1. Train a Cellpose model and check if it works well on your data.
+2. Create an environment ``python -m pip install 'cellpose[bioimageio]'`` or
+   ``'cellpose[all]'`` if you haven't already. Note that most users installed
+   ``'cellpose[gui]'`` without the bioimageio packages.
+3. Export the model using ``export.py`` script. Use ``python export.py --help``
+   to see the usage, or check the example in `its docstring
+   <https://github.com/MouseLand/cellpose/blob/8bc3f628be732a733e923e93c30c11172e564895/cellpose/export.py#L3-L38>`_.
+   In short, you need to name your models, specify if the model runs on
+   cytoplasm/nuclei/both, and provide:
+
+   1. a model filepath,
+   2. a README.md filepath,
+   3. a cover image filepath(s),
+   4. a short description string,
+   5. a license name like ``MIT``,
+   6. a link to your GitHub repo (or Cellpose repo),
+   7. information about the authors and what to cite,
+   8. tags including ``cellpose``, ``2d`` and ``3d`` (Cellpose models handle
+      both).
+
+4. If you are updating a uploaded model, you should also specify the model ID
+   and icon. Don't forget to increment the version number.
+5. Go to `BioImage.IO <https://bioimage.io/>`_, click "Upload", and follow the
+   instructions there.