A comparison of convolutional neural network-based approaches for label-free cell cycle prediction

Session

Poster Session 1

Authors

Elsa Sörman Paulsson (1), Rickard Sjögren (1)

Affiliations

1. Sartorius Corporate Research

Keywords

Convolutional Neural Networks

Phase Contrast Microscopy

Label-free Image Analysis

Cell Cycle Detection

Deep Learning

Abstract text

A fundamental aspect of cell biology research is to interrogate cell-cycle dynamics, requiring accurate identification of a given cell’s position within the cell cycle. In microscopic imaging it is standard practice to use fluorescent probes, such as the Fluorescent Ubiquitination-based Cell Cycle Indicator (FUCCI), to create a strong signal of the cell cycle state. Although fluorescent imaging provides a strong signal to help analysis and has been used in countless biological discoveries, there is mounting evidence that fluorescent sensors can alter biological responses stressing the need for label-free approaches. While there are sophisticated label-free imaging technologies facilitating analysis, simple brightfield and phase contrast imaging remains widespread due to being cheap, accessible, and easy-to-use. Thanks to great advances in deep learning-based image analysis driven by convolutional neural networks (CNNs), image analysis-workflows are now more capable than ever before and there are many promising ways of how to determine cell cycle state directly from label-free images.

We compare two different approaches to use CNN-based machine learning to determine cell cycle state directly from label-free 2D phase contrast microscopy-images. Both approaches use fluorescent images to set up ground truth for machine learning-algorithms to learn to predict the corresponding fluorescent readout for future label-free images. One approach is based on first segmenting single cells and then assigning them into discrete categories based on the fluorescent signal and finally train a CNN-classifier to predict the category for future cells. The second approach uses an in-silico labelling (ISL) approach (Christiansen 2018) and train a CNN to predict the corresponding fluorescent image directly from the phase contrast images and thereafter segmenting them and assigning them into categories based on the CNN-predicted fluorescence (Rappez 2020).

In a case study on cell cycle markers, we used a dataset of SK-OV-3, THP-1 and MDA-MB-231 cells labelled with two-color FUCCI. To perform single cell classification, we first performed label-free cell segmentation using a CNN-based instance segmentation model trained on LIVECell, our recently developed cell segmentation dataset (Edlund 2021). We then fine-tuned a CNN-classifier based on ResNet50 and pretrained on ImageNet to classify individual cells assigned to four classes according to their FUCCI expression. In parallel, we trained an ISL CNN, a modified variant of U-net (Ronneberger 2015) that we call OSA-U-net using one-shot aggregation (OSA) and effective Squeeze-and-Excitation blocks (Lee 2020), to minimize the difference between predicted and measured FUCCI fluorescent images as weighted by a smooth L1-loss. We then segmented the cells and assigned them into classes based on the ISL expression. We found that the ISL approach achieved better classification performance compared the classification approach, F1-score of 83.3 and 64 % for the two channels respectively compared to 79.2 and 53.6 % of the classifier. 

The ISL approach not only performed better at cell cycle classification but it also provides less involved configuration compared to the segment-than-classify approach. The classifier performance is directly dependent on the segmentation accuracy as well as the segmentation post-processing. In comparison, the ISL approach only requires phase contrast and fluorescent image pairs and cell segmentation is a completely separate step making it easier to use. To conclude, our ISL-based workflow provides an easy-to-configure method with promising performance for label-free cell cycle detection.

References

Christiansen, E. M., Yang, S. J., Ando, D. M., Javaherian, A., Skibinski, G., Lipnick, S., ... & Finkbeiner, S. (2018). In silico labeling: predicting fluorescent labels in unlabeled images. Cell, 173(3), 792-803.

Edlund, C., et al. “LIVECell - A large-scale dataset for label-free live cell segmentation” Nature Methods (in review) (2021)

Rappez, L., Rakhlin, A., Rigopoulos, A., Nikolenko, S., & Alexandrov, T. (2020). DeepCycle reconstructs a cyclic cell cycle trajectory from unsegmented cell images using convolutional neural networks. Molecular systems biology, 16(10), e9474.

Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.

Lee, Y., & Park, J. (2020). Centermask: Real-time anchor-free instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13906-13915).