Phase Object Reconstruction of 4D-STEM datasets using Deep Learning

Session

Stream 3: 3D+ Image Analysis

Authors

Thomas Friedrich (1), Chu-Ping Yu (1), Johan Verbeek (1), Timothy Pennycook (1), Sandra Van Aert (1)

Affiliations

1. Electron Microscopy for Materials Science (EMAT) and NANOlab Center of Excellence, University of Antwerp

Keywords

Phase retrieval, Electron diffraction, 4D-STEM, CBED, Machine Learning

Abstract text

Summary 

In this research, a new approach to reconstruct the phase image of electron microscopy samples is proposed. This method includes solving the phase retrieval problem with deep learning and a reconstruction algorithm based on reversing the electron-sample interaction. The design of the neural network for deep learning and the procedure to process the 4D scanning transmission electron microscopy (STEM) dataset will be covered in this talk, along with some example results showcasing the applicability of the method. 

Introduction 

STEM is a powerful tool for inspecting samples at the nanometer scale. It utilizes a focused electron beam to interact with the specimen and to acquire detailed information from materials. Scanning the specimen over a grid and recording full convergent beam electron diffraction patterns (CBEDs) with appropriate electron cameras [3] results in 4-dimensional datasets with a CBED for each scan position. Having access to full diffraction patterns allows for a variety of data processing methods beyond integrative methods like annular dark field (ADF) imaging. A popular alternative imaging approach is the reconstruction of the phase shift an electron wavefunction experiences when it interacts with electrostatic atomic potentials. This idea is typically used by methods falling into the broader group of ptychographic reconstruction algorithms [4, 5], which have shown to be able to retrieve the phase object accurately. However, ptychographic algorithms usually employ iterative optimization schemes and thus are computationally very expensive and may converge to local minima or even not at all. In this talk, we demonstrate a deep-learning-based approach for rapid phase imaging in which the complex exit wave is retrieved using a convolutional neural network (CNN) and the phase object reconstructed using phase object approximation (POA). This approach reduces the processing time dramatically and potentially enables near-real-time imaging already during data acquisition. The method is tested with both real and simulated datasets with a wide range of variables, showing its adaptability to different experimental conditions. 

Methods/Materials  

We train a U-NET based Neural Network (NN) with simulated data using the multislice algorithm as a forward model to compute electron diffraction patterns. Each of the ≈250000 training samples consists of a 3x3 kernel of adjacent CBEDs as feature and an exit wave (amplitude-phase pair) as label. The simulation parameters and microscope settings are drawn at random within  practically meaningful ranges. The atomic specimens for the simulations are constructed from randomly drawn crystallographic data files from the Materials project [1] and specimen orientations drawn from the set of all low-index zone-axis orientations, with a random rotation around the beam vector. The specimen thicknesses vary between 2-5 Å. The effect of a finite electron dose is modelled by scaling the CBEDs by a constant factor and assuming a Poisson distribution on each pixel. This step is applied as a data augmentation step during the training, resulting in a different dose and dose realization for each training sample in each epoch. This combination of an appropriate forward model, a vast number of structures, continuous microscope parameter ranges and an effective way of data augmentation enables the creation of large datasets without redundancy and thus provides the means to train a neural network to solve the inverse problem of retrieving the exit wave in a general manner at minor risk of overfitting.   

In the POA, the scattering of the electrons can be seen as a phase change to the electron wave, and the magnitude of this change is directly related to the local projected potential. Therefore, the phase difference between the incident and the outgoing waves can be used to reconstruct the atomic potential of a phase object. 

_{O(r)ψin(r)=ψout(r)}

In the proposed method, after retrieving the exit wave ψ_out(r) by the NN, we assume a known incident wave ψ_in(r) based on the size of the probe forming aperture. By dividing the former with the latter, a patch of the object O(r) at the probe position is reconstructed. This process is performed repeatedly at each probe position until the object is completely reconstructed. 

 Results & Discussion  

The reconstruction was tested on a real SrTiO3 dataset. The crystal contains both strong and weak scatterers, which can be problematic for some imaging methods such as ADF imaging, where signals from columns of lower density are easily overwhelmed by the others. To avoid this problem other imaging conditions such as annular bright-field (ABF) imaging can be applied [2], but the contrast from atoms of low Z is often hardly visible. The reconstructed image (Fig. 1) using the proposed method, on the other hand, clearly shows columns of both kinds. For comparison, Fig.1 also shows ADF and ABF images created using a virtual detector slightly larger than the convergence angle and an annular detector collecting from half to full convergence angle, respectively. 

  

Fig. 1 Real data example of SrTiO3, (l.t.r :) reconstructed phase image, ADF image, ABF image. 

 

The approach is further tested with simulations of different electron beam energies and doses (Fig. 2). The simulation is run for a 10 nm thick zeolite, with a convergence angle of 30 mrad, and collection angle for the other two imaging methods followed the previously stated rule. The reconstructed image shows that high-frequency signals are preserved to some degree even at a very low dose, and the oxygen columns can be found between the silicon columns, which is lost in the ADF and ABF images, except at infinite dose. It is worth noting that in both examples the specimen are thicker than the NN was actually trained for, pointing towards a rather robust adaptability of the method.

  

Fig. 2 Comparison of three imaging methods with different beam energy and dose for a simulated, 10nm thick zeolite specimen. 

 

Conclusion  

In this abstract, we proposed a deep-learning method for reconstructing a phase image of thin objects using 4D STEM datasets. We employed a U-NET based architecture that was trained on a large synthetic dataset. The method was tested with some simulated and real datasets and showed that it can perform well under various experimental conditions, including different beam energies and dose levels. 

References

[1] Jain, Anubhav, et al. "Commentary: The Materials Project: A materials genome approach to accelerating materials innovation." APL materials 1.1 (2013): 011002. 

[2] Findlay, S. D., et al. "Robust atomic resolution imaging of light elements using scanning transmission electron microscopy." Applied Physics Letters 95.19 (2009): 191913. 

[3] Mark W Tate et al. “High dynamic range pixel array detector for scanning transmission electron microscopy”. Microscopy and Microanalysis 22.1 (2016), pp. 237–249. 

[4] Andrew M Maiden and John M Rodenburg. “An improved ptychographical phase retrieval algorithm for diffractive imaging”. Ultramicroscopy109.10 (2009), pp. 1256–1262 

[5] Timothy J Pennycook et al. “Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution”. Ultramicroscopy151 (2015), pp. 160–167 

[6] This work was supported by the European Research Council (Grant 770887 PICOMETRICS to SVA and Grant 823717 ESTEEM3). J.V. and S.V.A acknowledge funding from the University of Antwerp through a TOP BOF project. We also acknowledge funding from the European Research Council under the European Union"s Horizon 2020 research and innovation programme via 802123-HDEM.