1. History

One of the earliest instances where the analysis of diffraction patterns as a function of probe position was considered took place in 1995, where Nellist, McCallum and Rodenburg attempted electron Ptychography analysis of crystalline Silicon.^[1]

The fluctuation electron microscopy (FEM) technique, proposed in 1996 by Treacy and Gibson, also included quantitative analysis of the differences in images or diffraction patterns taken at different locations on a given sample.^[2]

The field of 4D STEM remained underdeveloped due to the limited capabilities of detectors available at the time. CCD detectors commonly used in transmission electron microscopy (TEM) had limited data acquisition rates, could not distinguish where on the detector an electron strikes with high accuracy, and had low dynamic range which made them undesirable for use in 4D STEM.^[3]

In the late 2010s, the development of hybrid pixel array detectors (PAD) with single electron sensitivity, high dynamic range, and fast readout speeds allowed for practical 4D STEM experiments.^[4][5]

4D STEM operating principle, as the beam rasters across the surface various detectors simultaneously store position specific locations in real space and diffraction patterns in momentum space. CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=70302515

2. Operating Principle

While the process of data collection in 4D STEM is identical to that of standard STEM, each technique utilizes different detectors and collects different data. In 4D STEM there is a pixelated electron detector located at the back focal plane which collects the CBED pattern at each scan location. An image of the sample can be constructed from the CBED patterns by selecting an area in reciprocal space and assigning the average intensity of that area in each CBED pattern to the real space pixel the pattern corresponds to.

It is also common for there to be a(n) ADF or HAADF image taken concurrently with the CBED pattern collection. The dark-field image taken is complementary to the bright-field image constructed from the captured CBED images.

The use of a hollow detector with a hole in the middle can allow for transmitted electrons to be passed to an EELS detector while scanning. This allows for the simultaneous collection of chemical spectra information and structure information.

3. Detectors

In traditional TEM, imaging detectors use phosphorescent scintillators paired with a charge coupled device (CCD) to detect electrons.^[6] While these devices have good electron sensitivity, they lack the necessary readout speed and dynamic range necessary for 4D STEM. Additionally, the use of a scintillator can worsen the point spread function (PSF) of the detector due to the electron's interaction with the scintillator resulting in a broadening of the signal. In contrast, traditional annular STEM detectors have the necessary readout speed, but instead of collecting a full CBED pattern the detector integrates the collected intensity over a range of angles into a single data point.^[7] The development of pixelated detectors in the 2010s with single electron sensitivity, fast readout speeds, and high dynamic range has enabled 4D STEM as a viable experimental method.^[3]

4D STEM detectors are typically built as either a monolithic active pixel sensor (MAPS) or as a hybrid pixel array detector (PAD).^[3]

3.1. Monolithic Active Pixel Sensor (MAPS)

A MAPS detector consists of a complimentary metal-oxide-semiconductor (CMOS) chip paired with a doped epitaxial surface layer which converts high energy electrons into many lower energy electrons that travel down to the detector. MAPS detectors must be radiation hardened as their direct exposure to high energy electrons makes radiation damage a key concern.^[8]

Due to its monolithic nature and straightforward design, MAPS detectors can attain high pixel densities on the order of 4000 x 4000. This high pixel density when paired with low electron doses can enable single electron counting for high efficiency imaging. Additionally, MAPS detectors tend to have electron high sensitivities and fast readout speeds, but suffer from limited dynamic range.^[9]

3.2. Pixel Array Detector (PAD)

PAD detectors consist of a photodiode bump bonded to an integrated circuit, where each solder bump represents a single pixel on the detector.^[4]

These detectors typically have lower pixel densities on the order of 128 x 128 but can achieve much higher dynamic range on the order of 32 bits. These detectors can achieve relatively high readout speeds on the order of 1 ms/pixel but are still lacking compared to their annular detector counterparts in STEM which can achieve readout speeds on the order of 10 μs/pixel.^[4][5][10]

Detector noise performance is often measured by its detective quantum efficiency (DQE) defined as: [math]\displaystyle{ DQE=\frac{SNR_o^2}{SNR_i^2} }[/math]

Where [math]\displaystyle{ SNR_o^2 }[/math] is output signal to noise ratio squared and [math]\displaystyle{ SNR_i^2 }[/math] is the input signal to noise ratio squared. Ideally the DQE of a sensor is 1 indicating the sensor generates zero noise. The DQE of MAPS, APS and other direct electron detectors tend to be higher than their CCD camera counterparts.^[11][12]

4. Computational Methods

A major issue in 4D STEM is the large quantity of data collected by the technique. With upwards of 100s of TB of data produced over the course of an hour of scanning, finding pertinent information is challenging and requires advanced computation.^[13]

Analysis of such large datasets can be quite complex and computational methods to process this data are being developed. Many code repositories for analysis of 4D STEM are currently in development including: HyperSpy, pyXem, LiberTEM, Pycroscopy, and py4DSTEM.^{[3][14][15][16][17][18]}

These large data sets could enable the implementation of artificial intelligence (AI) and machine learning (ML) algorithms into TEM experiments, where patterns not normally considered by microscopists could be revealed by AI. When grounded in physically meaningful frameworks, AI and ML can lead to high efficiency in feature finding in large STEM datasets. Beyond feature identification, AI and ML can also help inform experimental design due to its effective predictive cost analysis.^[19]

These types of AI driven analysis, however, require databases of information to train on which currently do not exist. Additionally, lack of metrics for data quality, limited scalability due to poor cross-platform support across different manufacturers, and lack of standardization in analysis and experimental methods brings up questions of comparability across different datasets as well as reproducibility.^[13]

5. Selected Applications

4D STEM has been utilized in a wide array of applications, the most common uses include virtual diffraction imaging, orientation and strain mapping, and phase contrast analysis which are covered below. The technique has also been applied in: medium range order measurement, Higher order Laue zone (HOLZ) channeling contrast imaging, Position averaged CBED, fluctuation electron microscopy, biomaterials characterization, and medical fields (microstructure of pharmaceutical materials and orientation mapping of peptide crystals). This list is in no way exhaustive and as the field is still relatively young more applications are actively being developed.

5.1. Virtual Diffraction Imaging

Diffraction pattern evolution as a function of ordering in the sample: In the blue region and ordered blue/magenta ordered region of the sample show different diffraction patterns and at the interfacial region both diffraction patterns are observed. CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=70302742

Virtual diffraction imaging is a method developed to generate real space images from diffraction patterns.^[3] This technique has been used in characterizing material structures since the 90s^[20] but more recently has been applied in 4D STEM applications. This technique is often referred to as scanning electron nano diffraction (SEND) in which 4 pairs of magnetic coils are used to tilt the converged probe and raster the beam across the surface.^[21] A "virtual detector," is not a detector at all but rather a method of data processing which integrates a subset of pixels in diffraction patterns at each raster position to create bright-field and dark-field images. The processing of these images is performed by first averaging the diffraction patterns over all scans. Then a region of interest is selected in the diffraction pattern, a virtual aperture is drawn on the reduced dataset, and the image is constructed using computational methods. This virtual aperture can be any size/shape desired and can be created using the 4D dataset gathered from a single scan.^[22] This ability to simulate different apertures is enabled by 4D STEM methods and eliminates a typical weaknesses in conventional STEM operation as STEM bright-field and dark-field detectors are placed at fixed angles and cannot be changed during imaging.^[23]

With a 4D dataset bright/dark-field images can be obtained by integrating diffraction intensities from diffracted and transmitted beams respectively.^[21] Creating images from these patterns can give nanometer resolution and is typically used to characterize the structure of nanomaterials. These diffraction patterns can be indexed and analyzed for orientation and phase mapping, where real space images can be constructed from scans.^[3] The advantages of performing virtual imaging in 4D STEM include: enabling detector geometries not possible in conventional electron microscopy, allowing for specific use of scattered electrons at any angle which hits the detector, improving signal to noise on images, and resistance to sample bending/dynamic scattering events.^[23]

4D STEM has been used to map interfaces, select intensity from selected areas of the diffraction plane to form enhanced dark field images,^[24] map positions of nanoscale precipitates,^[25] create phase maps of beam sensitive battery cathode materials,^[26] and measure degree of crystallinity in metal-organic frameworks (MOFs).^[27]

5.2. Phase Orientation Mapping

Phase orientation mapping is typically done with electron back scattered diffraction in SEM which can give 2D maps of grain orientation in polycrystalline materials.^[28] The technique can also be done in TEM using Kikuchi lines, which is more applicable for thicker samples since formation of Kikuchi lines relies on diffuse scattering being present. Alternatively, in TEM one can utilize precession electron diffraction (PED) to record a large number of diffraction patterns and through comparison to known patterns, the relative orientation of grains in can be determined. 4D STEM can also be used to map orientations, in a technique called Bragg spot imaging. The use of traditional TEM techniques typically results in better resolution than the 4D STEM approach but can fail in regions with high strain as the DPs become too distorted.

In Bragg spot imaging, first correlation analysis method is performed to group diffraction patterns (DPs) using a correlation method between 0 (no correlation) and 1 (exact match); then the DP's are grouped by their correlation using a correlation threshold. A correlation image can then be obtained from each group. These are summed and averaged to obtain an overall representative diffraction template from each grouping. Different orientations can be assigned colors which helps visualize individual grain orientations.^[21] With proper tilting and utilizing precession electron diffraction (PED) it is even possible to make 3D tomographic renderings of grain orientation and distribution.^[29] Since the technique is computationally intensive, recent efforts have been focused on a machine learning approach to analysis of diffraction patterns.^[30][31]

5.3. Strain Mapping

TEM can measure local strains and is often used to map strain in samples using condensed beam electron diffraction CBED.^[7] The basis of this technique is to compare an unstrained region of the sample's diffraction pattern with a strained region to see the changes in the lattice parameter. With STEM, the disc positions diffracted from an area of a specimen can provide spatial strain information. The use of this technique with 4D STEM datasets includes fairly involved calculations.^[3]

Utilizing SEND, bright and dark field images can be obtained from diffraction patterns by integration of direct and diffracted beams respectively, as discussed previously. During 4D STEM operation the ADF detector can be used to visualize a particular region of interest through a collection of scattered electrons to large angles to correlate probe location with diffraction during measurements.^[21] There is a tradeoff between resolution and strain information; since larger probes can average strain measurements over a large volume, but moving to smaller probe sizes gives higher real space resolution. There are ways to combat this issue such as spacing probes further apart than the resolution limit to increase the field of view.^[3]

This strain mapping technique has been applied in many crystalline materials and has been extended to semi-crystalline and amorphous materials (such as metallic glasses) since they too exhibit deviations from mean atomic spacing in regions of high strain^[3][32]

5.4. Phase Contrast Analysis

Differential Phase Contrast

The differential phase contrast imaging technique (DPC) can be used in STEM to map the local electromagnetic field in samples by measuring the deflection of the electron beam caused by the field at each scan point. Traditionally the movement of the beam was tracked using segmented annular field detectors which surrounded the beam. DPC with segmented detectors has up to atomic resolution.^[33] The use of a pixelated detector in 4D STEM and a computer to track the movement of the "center of mass" of the CBED patterns was found to provide comparable results to those found using segmented detectors. 4D STEM allows for phase change measurements along all directions to be measured without the need to rotate the segmented detector to align with specimen orientation.^[34] The ability to measure local polarization in parallel with the local electric field has also been demonstrated with 4D STEM.^[35]

DPC imaging with 4D STEM is up to 2 orders of magnitude slower than DPC with segmented detectors and requires advanced analysis of large four-dimensional datasets.^[36]

Ptychography

The overlapping CBED measurements present in a 4D STEM dataset allow for the construction of the complex electron probe and complex sample potential using the ptychography technique. Ptychographic reconstructions with 4D STEM data were shown to provide higher contrast than ADF, BF, ABF, and segmented DPC imaging in STEM. The high signal-to-noise ratio of this technique under 4D STEM makes it attractive for imaging radiation sensitive specimens such as biological specimens^[34] The use of a pixelated detector with a hole in the middle to allow the unscattered electron beam to pass to a spectrometer has been shown to allow ptychographic analysis in conjunction with chemical analysis in 4D STEM.^[37]

MIDI STEM operation geometry. CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=70303149

MIDI STEM

This technique MIDI-STEM (matched illumination and detector interferometry-STEM), while being less common, is used with ptychography to create higher contrast phase images. The placement of a phase plate with zones of 0 and π/2 phase shift in the probe forming aperture creates a series of concentric rings in the resulting CBED pattern. The difference in counts between the 0 and π/2 regions allows for direct measurement of local sample phase.^[38] The counts in the different regions could be measured via complex standard detector geometries or the use of a pixelated detector in 4D STEM. Pixelated detectors have been shown to utilize this technique with atomic resolution.^[39]

(MIDI)-STEM produces image contrast information with less high-pass filtering than DPC or ptychography but is less efficient at high spatial frequencies than those techniques.^[3] (MIDI)-STEM used in conjunction with ptychography has been shown to be more efficient in providing contrast information than either technique individually.^[40]