Microscopy image analysis for humans (with machines)

Session

Artificial Intelligence

Authors

Dr Peter Bankhead (1)

Affiliations

1. University of Edinburgh

Keywords

image analysis, digital pathology, machine learning, artificial intelligence, software, open-source

Abstract text

Quantitative analysis of microscopy images is hard. Almost every task could be approached in myriad ways, drawing upon a plethora of image processing and machine learning techniques – or inventing new ones. Each design decision made when constructing an analysis pipeline, such as the choice of a specific filter or thresholding method, can have a profound effect upon the results. This makes it difficult to devise and implement robust strategies that truly answer scientific questions with minimal bias, or to compare results across different labs.

The way in which analysis is performed in practice depends upon 1) what the analyst knows, and 2) what tools they have at their disposal. This makes the availability of both training and software crucial. Over the past two decades, open-source image analysis platforms have become indispensable to many researchers working with microscopy data. This is particularly evident in the biosciences, with ImageJ, Fiji, CellProfiler and (increasingly) Python/SciPy being widespread. These highly flexible and generic tools can be applied to a broad range of problems. The challenge, however, is that many users are domain specialists who require image analysis as a means to an end. Consequently, they often lack the time, background knowledge, or coding skills to build and validate the bespoke analysis pipelines that would ideally be used to precisely address their scientific questions – relying instead upon surrogate measurements that are easier to obtain.

QuPath (https://qupath.github.io) is an open-source platform that takes a different approach. Inspired by established open software, QuPath consciously avoids trying to replicate what they already do well. Rather, it aims to provide a complementary, applications-driven approach to streamline image analysis for only a carefully chosen subset of tasks – with horizons that expand over time. By focusing first on emerging imaging modalities, it is designed from the start to address unmet needs in microscopy analysis.

To this end, the first QuPath release in 2016 focussed on evaluating immunohistochemical biomarkers in brightfield whole slide scans of tissue microarrays. This included new, efficient tools to view and annotate ultra-large virtual microscopy images, combined with novel cell segmentation algorithms and a hierarchical data model that scales intuitively to millions of objects. Subsequent releases built upon these foundations to support a broad range of digital pathology applications, and then also fluorescence and multiplexed imaging data. More recently, core features have been added to enable interactive pixel classification, deep learning model inference, and spatial analysis. An active user community has emerged on the Scientific Community Image Forum (image.sc), providing support and advice for how these tools can be used creatively and in combination with QuPath's scripting features to solve many more specific image analysis challenges.

This talk will describe how QuPath came to exist, its relationship to other image analysis and artificial intelligence tools, and what the future holds. It aims to demonstrate that, despite an initial focus on pathology, QuPath can be useful across different domains. It will also discuss the importance of sharing ideas across disciplines, as we aim to develop software that actively helps users follow best practices and make the most of their imaging data.