Spatial-temporal graph convolutional neural networks on evolving cell surfaces

Session

Artificial Intelligence

Authors

Mr Edward Offord (1), Dr Judith Lutton (1), Professor Till Bretschneider (1)

Affiliations

1. Warwick University

Keywords

Machine learning, macropinocytic cups, image processing, graph neural networks 

Abstract text

Title:

Summary:

Many cellular processes involve complex deformations of the cell surface that change over time, which are difficult to automatically detect and analyse in generated 3D microscopy surfaces. Attempting to use modern machine learning methods for extracting dynamic features from surface triangulations of a cell is a major challenge, because the topology is constantly changing over time. To incorporate time as a feature to inform feature extraction on cell surfaces this talk presents heterogenous graph neural networks(HGN), using a time based hypergraph of a cell timeseries. Here, we focus on identification of macropinocytic cups on the surface of Dictyostelium cells, structures involved in the uptake of extracellular fluid. The HGN classifies each node into belonging to a cup or not, enabling subsequent studies of the detailed distribution of molecules regulating fluid uptake in cells.  

Introduction:

Recent advances in microscopy and whole-cell segmentation methods make it possible to capture highly complex dynamic cell surfaces in 3D, at an unprecedented level of detail.

However, analysis of the detailed structure of these surfaces presents a major challenge.

We propose the use of graph convolutional neural networks (GCNs) [Zie Zhou et al. 2020] as the ideal tool for feature detection on the surface mesh. GCNs are analogous to convolutional neural networks used in image processing, but are able to account for the irregular structure of a surface mesh.

To incorporate the time component of the developing surface strucutres an adaptation of graph neural networks called heterogenous graph neural networks are presented. Where time relations are generated using topological feature matching developed by FOCUSR [Lombaert H. et al. 2012].

Methods:

Dictyostelium cells expressing fluorescent markers for actin and PIP3 were recorded with a lattice light sheet microscope, yielding movies with planar pixel size 0.104microns, z-spacing 0.162 microns, and frame rate 2--3~seconds. Whole cell segmentation was performed using the curvature-enhanced random walker[E. Josiah Lutton et al.2021].  Triangulated surface meshes were generated from these segmented images using Matlab's isosurface function, giving meshes with approximately $10^5$ vertices.

Graphs are generated from cell surface triangulations, with a 6-dimensional feature vector assigned to each vertex (x/y/z coordinates, two biomarker intensities, and Gaussian curvature).

Two time-series containing 200 frames were processed as above, and split into training ($160$),

validation (20), and testing (20) datasets. Test samples were manually selected such that they are significantly different from existing training data and have a high proportion of macropinocytic cups, in order to ensure a variety of unseen cups are used for testing.

The GCN uses convolutional layers developed for node and graph classification, with the former being relevant here. The convolutional layers reduce the dimensionality of the feature vector until the last layer outputs a probability value, utilizing a sigmoid activation function. The model outputs a probability for each vertex of the mesh to be part of a macropinocytic cup.

To extend this method of graph convolution to have a temporal component time relationships between the verticies in neighbouring cell time frames are related to each other using the FOCUSR matching method. These relations are then recorded as temporal edges as opposed to the previous spatial edges recorded from the mesh surface. Heterogenous graph convolutional neural networks are implemented by applying graph convolutions to spatial and temporal edges separately.

Results and discussion:

Results of the spatial graph neural network are evaluated using the cell segmentation benchmark [Pavel Matula et al. 2015] and the Hausdorff distance, both adapted for application to surface feature detection.  The cell segmentation benchmark is a combination of a segmentation score (SEG) and detection score (DET) used for comparing the accuracy of cell segmentation in images containing one or more cells. We apply this measure by considering segmentation of each cup as analogous to that of a cell in a multi-cell image, with mesh vertices replacing image pixels in this adaptation. The Hausdorff distance is a measure of how far apart two sets are at most.

High DET values with an average of 0.96 recorded suggest that the trained model is able to correctly identify macropinocytic cups with few errors and the SEG score of 0.79 indicates these identified regions have significant overlap with the ground truth.

The Hausdorff distance of 0.18 of the cups diameter is relatively high in these measurements implying the boundary of the macropinocytic cup is harder for the model to correctly identify.

Conclusion:

This talk demonstrates the utility of graph neural networks to answer biological

questions on cell surface dynamics. Despite a relatively simple architecture, the GCN applied to our problem is able to successfully detect macropinocytic cups on the cell surface, which using filters operating on the 3D volume data would be very difficult to achieve. The network is also able to segment these structures with a reasonable accuracy, although this problem is ill-defined due to the irregularity of the cups making it difficult to produce ground truth annotations. Temporal networks improve on the detection score and show promise to improve on the ability from machine learning techniques to detect cups

References

Zie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang,  Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng  Li, and Maosong Sun, “Graph neural networks: A review of methods and applications,” AI Open, vol. 1, pp.  57–81, 2020.

Lombaert, H., Grady, L., Polimeni, J. R., & Cheriet, F. (2012). FOCUSR: feature oriented correspondence using spectral regularization--a method for precise surface matching. _IEEE transactions on pattern analysis and machine intelligence_, _35_(9), 2143-2160.

E. Josiah Lutton, Sharon Collier, and Till Bretschneider, “A curvature-enhanced random walker segmentation method for detailed capture of 3D cell surface membranes,” IEEE Transactions on Medical Imaging, vol.  40, no. 2, pp. 514–526, 2021.

Pavel Matula, Martin Maˇska, Dmitry V. Sorokin,  Petr Matula, Carlos Ortiz-de Sol ́orzano, and Michal  Kozubek, “Cell tracking accuracy measurement based  on comparison of acyclic oriented graphs,” PLoS one,  vol. 10, no. 12, pp. 1–19, 12 2015