Robust morphology-based classification of cells following label-free cell-by-cell segmentation using convolutional neural networks

Session

Stream 5: Label Free Imaging

Authors

Dr. Gillian Lovell (3), Mr. Christoffer Edlund (2), Mr. Rickard Sjögren (2, 4), Dr. Daniel Porto (1), Dr. Nevine Holtz (1), Dr. Nicola Bevan (3), Ms. Jasmine Trigg (3), Dr. Johan Trygg (2, 4), Mr. Timothy Dale (3), Dr. Timothy Jackson (3)

Affiliations

1. Sartorius
2. Sartorius Corporate Research
3. Sartorius, BioAnalytics
4. Computational Life Science Cluster (CLiC)

Keywords

convolutional neural networks, instance segmentation, label-free, cell morphology, multivariate analysis, phase contrast imaging

Abstract text

Combining high-throughput, live-cell imaging with accessible, non-invasive label-free modalities such as phase contrast imaging provides great spatiotemporal resolution to study biological phenomena. Accurate segmentation of individual cells enables exploration of complex biological questions. However, due to low contrast and high object density, this requires sophisticated imaging processing pipelines such as convolutional neural networks (CNNs).  We previously reported on LIVECell, an open-source, high-quality, manually annotated and expert-validated dataset, comprising over 1.6 million annotated cells of 8 highly diverse cell types from initial seeding to full confluence (Edlund et al., in review). Alongside the dataset, we also trained instance segmentation CNN models with the CenterMask architecture (Lee and Park, 2020). Here, we fine-tune one of our publicly available LIVECell-trained models to enable quantitative analysis of complex morphological change associated with two applications, cell viability and differentiation. While these assays are commonly quantified using fluorescent or luminescent reporter reagents, such as cell death reporters or differentiation markers, the use of these reagents can require time-consuming optimisation steps or can perturb valuable samples. Taking the output label-free segmentation masks from our fine-tuned LIVECell models, we demonstrate the ability to quantify cell death and differentiation using the morphological features of phase contrast images without the use of reporter agents.

To improve performance of LIVECell-trained models on our two applications, we first acquired and annotated cells in phase contrast images using an Incucyte® S3 Live-cell Analysis System, including 100 images of apoptotic SKOV3 cells and 100 images of differentiated THP-1 cells. Fine-tuning the model using varied training set sizes, we find that less than 50 images were sufficient to raise the segmentation accuracy precision score (AP) by over 20 points for each application. This demonstrates a powerful workflow where robust models can be achieved by pre-training on LIVECell and performing minimal additional fine-tuning on application-specific annotated data.

To further exemplify the robustness of model segmentation, we performed multivariate data analysis (MVDA) using common cell morphology metrics, representative of different features including size, intensity, texture and shape. Although cell morphology is commonly examined in terms of a single, user-specified metric such as area, the use of multivariate analysis enables all morphological features to be summarised in a meaningful and quantitative manner.

Cells treated with cytotoxic compounds change morphology as they lose viability. For example, healthy SKOV3 ovarian cancer cells display mixed morphologies with many being large, flat and transparent while others have a typical mesenchymal phenotype, being elongated and dense. In contrast, apoptotic SKOV3 are small, circular and textured and in addition, certain treatments can generate cellular debris. While this wide range of morphologies and debris within a single experimental paradigm can be challenging to segment using traditional computer vision methods, the fine-tuned LIVECell model displayed a high AP score of 62.3 % when compared to ground truth annotations. Using the cell morphological data derived from segmented images of healthy and apoptotic SKOV3 cells, a regression model was trained to score objects from 0 (dead morphology) to 1 (live morphology). The regression model was then used to perform label-free classification of live and dead cells and applied to images of SKOV3 cells treated with staurosporine. The resulting time course of % dead cells indicated the expected time- and concentration-dependent cytotoxic effect, and an EC₅₀ value of 188 nM. This application demonstrates the ability to derive useful quantitative data from cell images without the use of fluorescent reagents.

The LIVECell-trained model was applied to a second set of experimental data which examined the differentiation of THP-1 monocytes to macrophages. This process causes cell morphology to drastically alter from rounded, highly textured monocytes to a flat, adherent, macrophage-like phenotype. THP-1 cells were treated with PMA (100 nM) to induce differentiation and the process was monitored using live-cell imaging over 72h. Fine-tuning the LIVECell-based model with additional images of differentiated cells improved the AP score from 22 % to an impressive 73 %, and data from the segmentation mask were then used to quantify the macrophages. Using the regression method described above to quantify macrophages, the resulting increase in % macrophages was then plotted over time and was consistent with data which used CD11b-positive status as a macrophage identifier.

With LIVECell to enable CNN-model development for 2D cell culture images, we envision such models will serve as the basis for analyses pipelines that target such exciting and physiologically relevant topics in biology and medicine. As demonstrated here, minimal refinement of the open-source models published alongside LIVECell is needed to enable dynamic, real-time cell shape analysis in novel applications.

References

Edlund C*, Jackson TR*, Khalid N*, Bevan N, Dale T, Ahmed S, Trygg J, Sjögren R (in review). Nature methods. LIVECell: A large-scale dataset for label-free cell segmentation. *contributed equally

Lee Y & Park J (2020). Proc. IEEECVF Conf. Comput. Vis. Pattern Recognit. CVPR . CenterMask: Real-Time Anchor-Free Instance Segmentation.  13906–13915.