Extracting standardized metallurgical metrics from microscopy images remains challenging due to complex grain morphology and the data demands of supervised segmentation. To bridge foundational computer vision with practical metallurgical evaluation, we propose an automated pipeline for dense instance segmentation and grain size estimation that adapts Cellpose-SAM to microstructures and integrates its topology-aware gradient tracking with an ASTM E112 Jeffries planimetric module. We systematically benchmark this pipeline against a classical convolutional network (U-Net), an adaptive-prompting vision foundation model (MatSAM) and a contemporary vision-language model (Qwen2.5-VL-7B). Our evaluations reveal that while the out-of-the-box vision-language model struggles with the localized spatial reasoning required for dense microscopic counting and MatSAM suffers from over-segmentation despite its domain-specific prompt generation, our adapted pipeline successfully maintains topological separation. Furthermore, experiments across progressively reduced training splits demonstrate exceptional few-shot scalability; utilizing only two training samples, the proposed system predicts the ASTM
We present SAM4EM, a novel approach for 3D segmentation of complex neural structures in electron microscopy (EM) data by leveraging the Segment Anything Model (SAM) alongside advanced fine-tuning strategies. Our contributions include the development of a prompt-free adapter for SAM using two stage mask decoding to automatically generate prompt embeddings, a dual-stage fine-tuning method based on Low-Rank Adaptation (LoRA) for enhancing segmentation with limited annotated data, and a 3D memory attention mechanism to ensure segmentation consistency across 3D stacks. We further release a unique benchmark dataset for the segmentation of astrocytic processes and synapses. We evaluated our method on challenging neuroscience segmentation benchmarks, specifically targeting mitochondria, glia, and synapses, with significant accuracy improvements over state-of-the-art (SOTA) methods, including recent SAM-based adapters developed for the medical domain and other vision transformer-based approaches. Experimental results indicate that our approach outperforms existing solutions in the segmentation of complex processes like glia and post-synaptic densities. Our code and models are available at h
Protein-based therapeutics play a pivotal role in modern medicine targeting various diseases. Despite their therapeutic importance, these products can aggregate and form subvisible particles (SvPs), which can compromise their efficacy and trigger immunological responses, emphasizing the critical need for robust monitoring techniques. Flow Imaging Microscopy (FIM) has been a significant advancement in detecting SvPs, evolving from monochrome to more recently incorporating color imaging. Complementing SvP images obtained via FIM, deep learning techniques have recently been employed successfully for stress source identification of monochrome SvPs. In this study, we explore the potential of color FIM to enhance the characterization of stress sources in SvPs. To achieve this, we curate a new dataset comprising 16,000 SvPs from eight commercial monoclonal antibodies subjected to heat and mechanical stress. Using both supervised and self-supervised convolutional neural networks, as well as vision transformers in large-scale experiments, we demonstrate that deep learning with color FIM images consistently outperforms monochrome images, thus highlighting the potential of color FIM in stress
Understanding the nanoscale carrier dynamics induced by light excitation is the key to unlocking futuristic devices and innovative functionalities in advanced materials. Optical pump-probe scanning tunneling microscopy (OPP-STM) has opened a window to these phenomena. However, mastering the combination of ultrafast pulsed lasers with STM requires high expertise and effort. We have shattered this barrier and developed a compact OPP-STM system accessible to all. This system precisely controls laser pulse timing electrically and enables stable laser irradiation on sample surfaces. Furthermore, by applying this technique to atomic force microscopy (AFM), we have captured time-resolved force signals with an exceptionally high signal-to-noise ratio. Originating from the dipole-dipole interactions, these signals provide insights into the carrier dynamics on sample surfaces, which are activated by photo-illumination. These technologies are promising as powerful tools for exploring a wide range of photoinduced phenomena in conductive and insulating materials.
We present an entanglement-based quantitative phase gradient microscopy technique that employs principles from quantum ghost imaging and ghost diffraction. In this method, a transparent sample is illuminated by both photons of an entangled pair - one detected in the near-field (position) and the other in the far-field (momentum). Due to the strong correlations offered by position-momentum entanglement, both conjugate observables can be inferred nonlocally, effectively enabling simultaneous access to the sample's transmission and phase gradient information. This dual-domain measurement allows for the quantitative recovery of the full amplitude and phase profile of the sample. Unlike conventional classical and quantum phase imaging methods, our approach requires no interferometry, spatial scanning, microlens arrays, or iterative phase-retrieval algorithms, thereby circumventing many of their associated limitations. Furthermore, intrinsic temporal correlations between entangled photons provide robustness against dynamic and structured background light. We demonstrate quantitative phase and amplitude imaging with a spatial resolution of 2.76 $μ$m and a phase sensitivity of $λ/100$ usin
Correlative microscopy is a powerful technique that combines the advantages of multiple imaging modalities to achieve a comprehensive understanding of investigated samples. For example, fluorescence microscopy provides unique functional contrast by imaging only specifically labeled components, especially in biological samples. However, the achievable structural information on the sample in its full complexity is limited. Here, the intrinsic label-free carbon contrast of water window soft X-ray microscopy can complement fluorescence images in a correlative approach ultimately combining nanoscale structural resolution with functional contrast. However, soft X-ray microscopes are complex and elaborate, and typically require a large-scale synchrotron radiation source due to the demanding photon flux requirements. Yet, with modern high-power lasers it has become possible to generate sufficient photon flux from laser-produced plasmas, thus enabling laboratory-based setups. Here, we present a compact table-top soft X-ray microscope with an integrated epifluorescence modality for 'in-situ' correlative imaging. Samples remain in place when switching between modalities, ensuring identical me
Fluorescence microscopy has enabled a dramatic development in modern biology. Due to its inherently weak signal, fluorescence microscopy is not only much noisier than photography, but also presented with Poisson-Gaussian noise where Poisson noise, or shot noise, is the dominating noise source. To get clean fluorescence microscopy images, it is highly desirable to have effective denoising algorithms and datasets that are specifically designed to denoise fluorescence microscopy images. While such algorithms exist, no such datasets are available. In this paper, we fill this gap by constructing a dataset - the Fluorescence Microscopy Denoising (FMD) dataset - that is dedicated to Poisson-Gaussian denoising. The dataset consists of 12,000 real fluorescence microscopy images obtained with commercial confocal, two-photon, and wide-field microscopes and representative biological samples such as cells, zebrafish, and mouse brain tissues. We use image averaging to effectively obtain ground truth images and 60,000 noisy images with different noise levels. We use this dataset to benchmark 10 representative denoising algorithms and find that deep learning methods have the best performance. To o
The advent of modern, high-speed electron detectors has made the collection of multidimensional hyperspectral transmission electron microscopy datasets, such as 4D-STEM, a routine. However, many microscopists find such experiments daunting since such datasets' analysis, collection, long-term storage, and networking remain challenging. Some common issues are the large and unwieldy size of the said datasets, often running into several gigabytes, non-standardized data analysis routines, and a lack of clarity about the computing and network resources needed to utilize the electron microscope fully. However, the existing computing and networking bottlenecks introduce significant penalties in each step of these experiments, and thus, real-time analysis-driven automated experimentation for multidimensional TEM is exceptionally challenging. One solution is integrating microscopy with edge computing, where moderately powerful computational hardware performs the preliminary analysis before handing off the heavier computation to HPC systems. In this perspective, we trace the roots of computation in modern electron microscopy, demonstrate deep learning experiments running on an edge system, an
Ultrasound Localization Microscopy (ULM) relies on the injection of microbubbles (MBs) to obtain highly resolved density maps of blood circulation in vivo, with a resolution that can reach 10 μm ~ λ/10 in the rodent brain. Static mean velocity maps can be extracted but are intrinsically biased by potential significant changes in the number of MBs detected during the cardiac cycle. Dynamic ULM (DULM) is a technique developed for non-invasive pulsatility measurements in the brain of rodents, leading to temporally resolved velocity and density cine-loops. It was previously based on external triggers such as the electrocardiogram (ECG), limiting its use to datasets acquired specifically for DULM applications while also increasing the required acquisition time. This study presents a new motion matching method using tissue Doppler that eliminates the need for ECG-gating in DULM experiments. DULM can now be performed on any ULM datasets, recovering pertinent temporal information, and improving the robustness of the mean velocity estimates.
The desire to image specimens in liquids has led to the development of open-cell and closed-cell techniques in transmission electron microscopy (TEM). The closed-cell approach is currently more common in TEM and has yielded new insights into a number of biological and materials processes in liquid environments. The open-cell approach, which requires an environmental TEM (ETEM), is technically challenging but may be advantageous in certain circumstances due to fewer restrictions on specimen and detector geometry. Here, we demonstrate a novel approach to open-cell liquid TEM, in which we use salt particles to facilitate the in situ formation of droplets of aqueous solution that envelope specimen particles coloaded with the salt. This is achieved by controlling sample temperature between 1 and 10°C and introducing water vapor to the ETEM chamber above the critical pressure for the formation of liquid water on the salt particles. Our use of in situ hydration enables specimens to be loaded into a microscope in a dry state using standard 3 mm TEM grids, allowing specimens to be prepared using trivial sample preparation techniques. Our future aim will be to combine this technique with an
Fluorescent molecules are versatile nanoscale emitters that enable detailed observations of biophysical processes with nanoscale resolution. Because they are well-approximated as electric dipoles, imaging systems can be designed to visualize their 3D positions and 3D orientations, so-called dipole-spread function (DSF) engineering, for 6D super-resolution single-molecule orientation-localization microscopy (SMOLM). We review fundamental image-formation theory for fluorescent di-poles, as well as how phase and polarization modulation can be used to change the image of a dipole emitter produced by a microscope, called its DSF. We describe several methods for designing these modulations for optimum performance, as well as compare recently developed techniques, including the double-helix, tetrapod, crescent, and DeepSTORM3D learned point-spread functions (PSFs), in addition to the tri-spot, vortex, pixOL, raPol, CHIDO, and MVR DSFs. We also cover common imaging system designs and techniques for implementing engineered DSFs. Finally, we discuss recent biological applications of 6D SMOLM and future challenges for pushing the capabilities and utility of the technology.
Surface electric noise, i.e., the non-uniform distribution of charges and potentials on a surface, poses a great experimental challenge in modern precision force measurements. Such a challenge is encountered in a number of different experimental circumstances. The scientists employing atomic force microscopy (AFM) have long focused their efforts to understand the surface-related noise issues via variants of AFM techniques, such as Kelvin probe force microscopy or electric force microscopy. Recently, the physicists investigating quantum vacuum fluctuation phenomena between two closely-spaced objects have also begun to collect experimental evidence indicating a presence of surface effects neglected in their previous analyses. It now appears that the two seemingly disparate science communities are encountering effects rooted in the same surface phenomena. In this report, we suggest specific experimental tasks to be performed in the near future that are crucial not only for fostering needed collaborations between the two communities, but also for providing valuable data on the surface effects in order to draw the most realistic conclusion about the actual contribution of the Casimir fo
Increasing interest in three-dimensional nanostructures adds impetus to electron microscopy techniques capable of imaging at or below the nanoscale in three dimensions. We present a reconstruction algorithm that takes as input a focal series of four-dimensional scanning transmission electron microscopy (4D-STEM) data. We apply the approach to a lead iridate, Pb$_2$Ir$_2$O$_7$, and yttrium-stabilized zirconia,Y$_{0.095}$Zr$_{0.905}$O$_2$ , heterostructure from data acquired with the specimen in a single plan-view orientation, with the epitaxial layers stacked along the beam direction. We demonstrate that Pb-Ir atomic columns are visible in the uppermost layers of the reconstructed volume. We compare this approach to the alternative techniques of depth sectioning using differential phase contrast scanning transmission electron microscopy (DPC-STEM) and multislice ptychographic reconstruction.
Fluorescence microscopy is an important and extensively utilised tool for imaging biological systems. However, the image resolution that can be obtained has a limit as defined through the laws of diffraction. Demand for improved resolution has stimulated research into developing methods to image beyond the diffraction limit based on far-field fluorescence microscopy techniques. Rapid progress is being made in this area of science with methods emerging that enable fluorescence imaging in the far-field to possess a resolution well beyond the diffraction limit. This review outlines developments in far-field fluorescence methods which enable ultrahigh resolution imaging and application of these techniques to biology. Future possible trends and directions in far-field fluorescence imaging with ultrahigh resolution are also outlined.
An approach reported recently by Alexandrov et al. on optical scatter imaging, termed digital Fourier microscopy (DFM), represents an adaptation of digital Fourier holography to selective imaging of biological matter. Holographic mode of recording of the sample optical scatter enables reconstruction of the sample image. Form-factor of the sample constituents provides a basis for discrimination of these constituents implemented via flexible digital Fourier filtering at the post processing stage. Like in the dark-field microscopy, the DFM image contrast appears to improve due to the suppressed optical scatter from extended sample structures. In this paper, we present theoretical and experimental study of DFM using biological phantom that contains polymorphic scatterers.
We review a new implementation of Kelvin probe force microscopy (KPFM) in which the dissipation signal of frequency modulation atomic force microscopy (FM-AFM) is used for dc bias voltage feedback (D-KPFM). The dissipation arises from an oscillating electrostatic force that is coherent with the tip oscillation, which is caused by applying the ac voltage between the tip and sample. The magnitude of the externally induced dissipation is found to be proportional to the effective dc bias voltage, which is the difference between the applied dc voltage and the contact potential difference. Two different implementations of D-KPFM are presented. In the first implementation, the frequency of the applied ac voltage, $f_\mathrm{el}$, is chosen to be the same as the tip oscillation ($f_\mathrm{el} = f_\mathrm{m}$: $1ω$D-KPFM). In the second one, the ac voltage frequency, $f_\mathrm{el}$, is chosen to be twice the tip oscillation frequency ($f_\mathrm{el}= 2 f_\mathrm{m}$: $2ω$D-KPFM). In $1ω$D-KPFM, the dissipation is proportional to the electrostatic force, which enables the use of a small ac voltage amplitude even down to $\approx 10$\,mV. In $2ω$D-KPFM, the dissipation is proportional to th
Synchrotron based photoemission electron microscopy with energy filter combines real space imaging with microprobe diffraction ($μ$-ARPES), giving access to the local electronic structure of laterally inhomogeneous materials. We present here an overview of the capabilities of this technique, illustrating selected applications of angle resolved photoemission electron microscopy and related microprobe methods. In addition, we report the demonstration of a darkfield XPEEM (df-XPEEM) imaging method for real space mapping of the electronic structure away from $Γ$ at a lateral resolution of few tens of nm. The application of df-XPEEM to the (1$\times$12)-O/W(110) model oxide structure shows the high sensitivity of this technique to the local electronic structure, allowing to image domains with inequivalent adsorption site symmetry. Perspectives of angle resolved PEEM are discussed.
Scanning probe microscopy (SPM) images of regularly arranged spatially periodic objects can be processed crystallographically. The resulting information may be used to remove from the SPM image distortions that are due to a less than perfect imaging process. The combined effects of these distortions result in a point spread function that gives a quantitative measure of the performance of the microscope for a certain set of experimental conditions. On the basis of highly symmetric calibration samples, the point spread function of the microscope may be extracted and utilized for the correction of SPM images of unknowns that were recorded under essentially the same experimental conditions. We concentrate in this paper on more theoretical aspects of our method. A blunt scanning tunneling microscopy (STM) tip that consists of multiple mini-tips with electron orbital dimensions may be symmetrized on the basis of prior knowledge on the plane symmetry of a two-dimensional periodic array. This is illustrated with the crystallographic processing of a STM image of a regular array of fluorinated cobalt phthalocyanine molecules on graphite and backed up conceptually by simple simulations.
Aberration-corrected scanning transmission electron microscopes (STEM) provide sub-angstrom lateral resolution; however, the large convergence angle greatly reduces the depth of field. For microscopes with a small depth of field, information outside of the focal plane quickly becomes blurred and less defined. It may not be possible to image some samples entirely in focus. Extended depth-of-field techniques, however, allow a single image, with all areas in-focus, to be extracted from a series of images focused at a range of depths. In recent years, a variety of algorithmic approaches have been employed for bright field optical microscopy. Here, we demonstrate that some established optical microscopy methods can also be applied to extend the ~6 nm depth of focus of a 100 kV 5th-order aberration-corrected STEM (alpha_max = 33 mrad) to image Pt-Co nanoparticles on a thick vulcanized carbon support. These techniques allow us to automatically obtain a single image with all the particles in focus as well as a complimentary topography map.
Advances in fluorescence microscopy enable acquisition of 3D image volumes with better image quality and deeper penetration into tissue. Segmentation is a required step to characterize and analyze biological structures in the images and recent 3D segmentation using deep learning has achieved promising results. One issue is that deep learning techniques require a large set of groundtruth data which is impractical to annotate manually for large 3D microscopy volumes. This paper describes a 3D deep learning nuclei segmentation method using synthetic 3D volumes for training. A set of synthetic volumes and the corresponding groundtruth are generated using spatially constrained cycle-consistent adversarial networks. Segmentation results demonstrate that our proposed method is capable of segmenting nuclei successfully for various data sets.