Leveraging advanced segmentation models in CellProfiler with RunCellpose and RunStarDist

Segmentation is an integral step of any bioimage analysis workflow, which involves delineating areas or objects of interest within an image¹. All the measurements performed on the image, e.g. size, shape, counts, or pixel intensities are then assigned to these segmented areas or objects to interpret the information in a biological context^2,3. Accuracy of segmentation is therefore crucial to avoid spurious results.

Within CellProfiler, segmentation is achieved using the modules of primary, secondary, and tertiary object identification, and for a vast majority of cell types, the outlines and compartments, e.g., nucleus, cytoplasm, and cell membrane can thereby be accurately identified⁴. However, for images where the objects are (i) non-round, (ii) highly heterogeneous in shapes and sizes, or (iii) overlapping⁵, as shown in the examples below, adequate segmentation can be quite difficult to achieve. These challenges are commonly faced by bioimage-analysts and can be addressed by using additional segmentation tools as plugins within CellProfiler.

To aid with challenging segmentation tasks, deep-neural network (DNN) based algorithms, such as Cellpose and StarDist, have risen to prominence in recent years^5-7. Cellpose was originally developed as a ‘generalist model’, trained on a diverse array of images to make it capable of segmenting various cell types without requiring new training data^5,6. In Cellpose 2.0, additional models with different segmentation styles have been added to allow the user to select the one that best fits their image⁶. StarDist, on the other hand, uses star-convex polygons, a shape that resembles cell nuclei, to generate probability maps of nuclear locations within images⁷, and has been reported to be quite effective for segmenting overlapping cells. Currently, both Cellpose and StarDist can be run as plugins (RunCellpose and RunStarDist) within CellProfiler^5-8, using either pre-trained models or the user’s custom models. The RunCellpose module now also supports Ominpose⁹, which is another (newer) generalist DNN-based algorithm that builds on Cellpose.

This blogpost aims to enable CellProfiler users, particularly those with limited programming experience, to leverage the power of these segmentation algorithms, and to this end, provides step-by-step guidance on installing RunCellpose and RunStarDist in CellProfiler. Further information on other supported plugins is present in the CP Plugins Documentation.

Running segmentation plugins within CellProfiler

To use plugins within CellProfiler, the first (and obvious) step would be to ensure that CellProfiler has been installed, which can be done by either building the software from source or by downloading the pre-built software from the website. Building software from source generally means creating the ‘executable file’, i.e. the file that will run the software, from the source files, i.e. the original data that was used to create the software. This allows users to modify the source files to add functionalities to CellProfier, and requires computational expertise. In contrast, downloading a pre-built software is directly downloading the already compiled executable file by clicking the download button in the website, and then running the executable file to install the software. This requires no computational expertise but limits the user to the existent functionalities of CellProfiler.

CellProfiler already comprises several Python libraries, however, using external plugins may require additional / different versions of Python libraries. These are known as ‘dependencies’ and these requirements need to be met for the plugins we intend to install and use. The following flow-chart provides an overview of the options to achieve this - you can directly navigate to the section relevant to you following the links.

Option I                             Option II

Option I: Installation via Python building CellProfiler from source 

1. Install CellProfiler 4.2.6 from source (steps described here: Windows, Mac Intel, Mac Apple Silicon, Linux)
2. Clone the CellProfiler plugins repository by typing the following code in the terminal:

git clone https://github.com/CellProfiler/CellProfiler-plugins.git

3. Set the plugins path in CellProfiler:
   1. Open CellProfiler
   2. Go to File => Preferences and set the path in the CellProfiler plugins directory to the active_plugins folder in the GitHub repository that you just cloned (or, if you didn’t clone the whole repository, whatever location you have saved your plugins into).
   3. Select Save at the bottom of the Preferences window
      
   4. Close CellProfiler
4. To install dependencies for the plugin, type the following code in the terminal, replacing [FLAG] with [cellpose] for RunCellpose and [stardist] for RunStarDist. 

cd CellProfiler-plugins

pip install -e .[FLAG]

5. Open and use CellProfiler

Note:

1. If the error no matches found: .[somepackage] shows up on Mac, please add single quotes, e.g., use pip install -e '.[cellpose]' instead.
2. While it is technically possible to install Cellpose and StarDist within the same environment, it can be challenging because of conflicting dependency requirements, and we recommend creating separate environments for Cellpose and StarDist, along with two separate installations of CellProfiler for each environment.
3. When Cellpose is loaded into CellProfiler for the first time, some model files will be downloaded from the internet, which may take time. This process can be sped up by using a supported GPU with a compatible version of PyTorch, following these instructions.
4. When you try to run your plugin in your pipeline, if you have missed copying over any specific requirements, it will give you an error message in the terminal window that opens with CellProfiler on Windows machines telling you what dependency is missing. This information is not available on Mac machines.

Option II: Using software containers (this is currently only available for RunCellpose)

1. Install CellProfiler either from the source or from the CellProfiler website
2. Follow steps 2 and 3 from the previous section
3. Install and run Docker Desktop.
4. Search for the ‘cellprofiler/runcellpose’ in Docker Desktop and add the following images
   
   

5. Open CellProfiler pipeline and add the RunCellpose module
6. Under Run Cellpose in docker or local Python environment, select ‘Docker’ from the dropdown
   
7. Select the Cellpose docker image: If you are not using a custom model, you can select "cellprofiler/runcellpose_with_pretrained:0.1". If you are using a custom model, you can use either "cellprofiler/runcellpose_no_pretrained:0.1" or "cellprofiler/runcellpose_with_pretrained:0.1", but the latter will be slightly larger (~500 MB) as it includes all of the pre-trained models.

Note: The Docker container will be downloaded during the first run of the RunCellpose plugin. This can take some time, however, this process needs to happen only once.

Some troubleshooting tips:

1. No plugins are visible in the ‘Add Modules’ panel
   1. The plugins path (described in step 3 of Option I) has not been correctly set.
2. The plugin of interest is not visible in the ‘Add Modules’ panel, but other plugins are visible
   1. Likely, the dependencies of the plugin of interest have not been correctly installed. Other plugins that have no additional dependency requirements may still be visible.

References

1. Meijering E. Cell Segmentation: 50 Years Down the Road [Life Sciences]. IEEE Signal Processing Magazine, 2012;29:140-145. doi: 10.1109/MSP.2012.2204190.
2. Boutros M, Heigwer F, & Laufer C. Microscopy-based high-content screening. Cell 2015;163:1314–1325. doi: 10.1016/j.cell.2015.11.007
3. Sommer C, Straehle C, Koethe U, Hamprecht FA. Ilastik: interactive learning and segmentation toolkit. IEEE International Symposium on Biomedical Imaging, 2011;230–233.
4. Stirling DR, Swain-Bowden MJ, Lucas AM, et al. CellProfiler 4: improvements in speed, utility and usability. BMC Bioinformatics 2021;22:433. doi: 10.1186/s12859-02
5. Stringer C, Wang T, Michaelos M, Pachitariu M. Cellpose: a generalist algorithm for cellular segmentation. Nat Methods. 2021;18:100-106. doi: 10.1038/s41592-020-01018-x.
6. Pachitariu M, Stringer C. Cellpose 2.0: how to train your own model. Nat Methods 2022;19:1634-1641. doi: 10.1038/s41592-022-01663-4.
7. Schmidt U, Weigert M, Broaddus C, Myers G. Cell Detection with Star-Convex Polygons. In: Frangi A, Schnabel J, Davatzikos C, Alberola-López C, Fichtinger G. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Lecture Notes in Computer Sciences, vol 11071. Springer, Cham. doi: 10.1007/978-3-030-00934-2_30
8. Weisbart E, Tromans-Coia C, Diaz-Rohrer B, et al. CellProfiler plugins - An easy image analysis platform integration for containers and Python tools. J Microsc 2023 Sep 10. doi: 10.1111/jmi.13223.
9. Cutler KJ, Stringer C, Lo TW et al. Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation. Nat Methods 2022;19:1438–1448. doi:  10.1038/s41592-022-01639-4