Beth Cimini
For those of you who’ve been with us for a long time though, the obvious next question after how to use the new test mode is will my old CellProfiler pipelines work in the new version? We feel the same way – the pipelines you’ve accumulated over the years are precious resources! The good and bad news is that the answer is Yes, mostly. In order to facilitate the speedup and continue the process of streamlining the code, a few things had to go; we also removed some things we felt were causing “option fatigue” for the sake of user friendliness going forward.
We’ve therefore pulled together a list of most of the major and minor changes you’ll encounter when switching between 2.2 and 3.0. You probably will have to spend a bit of time optimizing and re-checking pipelines when you DO make the switch, but we truly believe you’ll find it well worth your while in the long run. We’ve highlighted the biggest changes at the top for your convenience.
If you find something you’ve used before is now broken and you need help fixing it, please do make a new forum thread and let us know and we’ll do everything we can to help you find a new solution. As we continue to make changes and streamline, we’re going to use the data from 3.0’s new telemetry feature to see what modules and settings people generally use in pipelines and which seem to be superfluous; we really therefore appreciate if you turn it on (either on startup or in the preferences menu) so we can see (and not remove!) what’s most important to you in the community.
1) These are the major pipeline-breaking changes to keep an eye out for:
-
Changes to how thresholds are calculated*- these will affect the Threshold module as well as IdentifyPrimaryObjects and IdentifySecondaryObjects.
- CellProfiler 2.X’s way of calculating a 2-class Otsu threshold turned out to be non-standard; we’ve therefore started using the implementation in scikit-image instead to bring it in line with how other image processing tools calculate it. If you were using 2-class Otsu thresholding in any step of your pipeline, please check to make sure that’s still the best thresholding method for you.
- We’ve removed many of the thresholding methods- Binary image, MixtureOfGaussians (MoG), MaximumCorrelationThreshold (MCT), Background, Kapur, and RidlerCalvard. Setting the threshold to “Automatic” will now instead of the removed MCT threshold give you the newly-included Minimum Cross Entropy threshold instead.
- Adaptive thresholding no longer has an option to let CellProfiler “guess” the adaptive window size based on the image size; it must now be set explicitly. If you were previously letting CellProfiler calculate the value, you should check the adaptive window size as the current default value (10) in most cases will be significantly different than CellProfiler’s estimation (1/10th the size of the image).
-
We’ve moved a lot of the options out of Morph, which was getting overly-full of stuff and hard to navigate and maintain. NOTE that as of 3.0.0 there is a bug where pipelines incorporating one of the moved Morph options fail to open properly, we’re aware of it and should have it fixed for 3.0.1 in a few weeks (and can discuss on the forum how to work around it if you need it fixed sooner than that).
- Opening, Closing, Dilation, Erosion, and Fill small holes (now called RemoveHoles) have all now moved to their own modules in the Advanced category.
- Invert and Tophat were both duplicated in other places; they’ve been removed from Morph but still live on in their other homes (ImageMath for Invert, and EnhanceOrSuppressFeatures for Tophat).
-
We’ve made some changes to how to save objects and/or their outlines
- Previously lots of object processing modules (at least 10!) gave users the option to create images of the outlines of objects; this resulted in a lot of duplicated extra code, and often a lot of extra outline images clogging your computer’s memory during a pipeline that were never actually saved or used. We’ve therefore removed that functionality from most modules; if you want images of the outlines, you can still create them explicitly in OverlayOutlines (and as a fun bonus, we’ve added the ability to overlay shaded-in color “blobs” of your objects in the new OverlayObjects module).
- SaveImages no longer DIRECTLY allows you to save objects for loading into other CellProfiler pipelines; you may still do so though by using ConvertObjectsToImage in “Grayscale” or “uint16” to create an object label matrix image which can then be passed to SaveImages.
2) These are smaller but still noteworthy changes to modules that we made to streamline them and to remove a few minor settings that we felt generally didn’t perform well for the majority of users:
-
IdentifyPrimaryObjects
- We are proud to introduce an “advanced settings” pane in IdentifyPrimaryObjects; we aimed to make IdentifyPrimaryObjects less overwhelming to new users. Don’t worry: all the settings you’re used to (except those mentioned above) are still there behind the curtain!
- We’ve removed the Laplacian of Gaussian method of declumping objects.
-
MeasureTexture
- We’ve removed the calculation of the Gabor features.
- We’ve removed the ability to specify which angles to calculate Haralick features; all angles are now calculated. In prior versions of CellProfiler, the feature names were suffixed with the angle in degrees (`_0`, `_45`, `_90`, `_135`). In CellProfiler 3.0, the feature names are suffixed by an angle index (e.g., `_00`, `_01`, `_02`, `_03`). We chose this structure for consistency with 3D measurements, which compute Haralick features over 13 angles!
3) These are changes you’ll need to make in order to keep your workflows up-to-date:
- CellProfiler 3.0 is not able to load old MatLab CellProfiler 1.0 pipelines. If you have one of these, you should use a 2.X build (like the stable 2.2.0) to open these old pipelines and export them into a CellProfiler 3.0-readable format (.cppipe or .cpproj).
- As part of the further “python-ification” of CellProfiler, image files* can no longer be saved as .mat (typically this was only used in atypical cases where the pixel values were negative or decimals greater than 1.0, such as illumination correction images). We now recommend you save those unusual cases as .npy files; CellProfiler 3.0 will still read your old .mat files, but this will almost certainly change when we release 3.1 so make sure to convert your old .mat files to .npy! You can do so simply by opening them in CellProfiler and then using SaveImages to convert them to the new format.
4) Finally, these modules have gotten a name change (but should automatically upgrade to the new names when pipelines are loaded):
- ApplyThreshold is now just called Threshold.
- MeasureCorrelation is now called MeasureColocalization.
- CalculateImageOverlap is now called MeasureImageOverlap.
We carefully considered each of these changes before we made them, but as always we welcome your input. Good luck, and enjoy all the fun new features!
*= MeasureImageQuality still utilizes the previous threshold options; we hope to be able to bring it in line with all the other modules as soon as possible.
**= Note that we’re only talking about .mat image files here; we have no plans at this time to change the saving of whole experiment .mat or .h5 files in the output settings module.