Image Stitching

Summary

With new microscope technologies, scientists are acquiring terabyte-sized datasets to cover large area of their experiments. An automated optical microscope acquires pictures of a plate with cell cultures. It acquires a grid of partially overlapping images because the microscope field of view is much smaller than the dimensions of the plate being imaged. This process generates hundreds of thousands of images that need to be stitched into a large mosaic to derive meaningful information. Some of these large mosaics are sparse, mainly at the beginning of an experiment before cells grow and cover more areas. This creates overlapping areas with no foreground pixel intensities to compute the translations between tiles, which increases the translation computation uncertainty for the entire mosaic. Moreover, stitching such large image mosaic taxes the computing capacity of a high-end workstation; this becomes overwhelming for live cell experiments. Figure 1 highlights the MIST algorithmic pipeline description and novelties.

MIST has been applied to large and diverse datasets in terms of content, imaging modalities, and microscopes; totaling over a thousand fully stitched images (Figure 2). They are were acquired using five microscopes (Zeiss, Leica, Olympus, Nikon, and Focused Ion Beam Scanning Electron Microscope (FIB-SEM), and Zeiss), four imaging modalities (fluorescent, phase contrast, bright-field, and FIB-SEM), eight content types: (A10 cells, carbon nanotubes, Human Bone Marrow Stromal Cells (HBMSC), Induced Pluripotent Stem Cells (IPSCs), paper nanoparticles, rat brain cells, Human Embryonic Stem Cells, and C. elegans), large range of image overlaps (10-70%), and a large range of grid sizes (5x5 to 70x93).

Description

This work concerns a Microscopy Image Stitching Tool (MIST), a stitching algorithm for small and large two dimensional image grid collections. We developed a new and novel method for optimizing the translations computed by the phase-correlation method using the Fourier transform approach. This new method estimates the microscope stage repeatability from the computed translations of a gridded image tiles. It then optimizes all translation computation using Hill Climbing algorithm bounded to four times the stage repeatability. This minimizes the maximum uncertainty related to the translation computation for any pair of images.

This method is validated on large sparse images with reference dataset, generated for computing the stitching errors on the assembled mosaic.

Major Accomplishments

An open-source tool is made available for free download. This technique was developed in Matlab, for which the source code is available for download along with a compiled executable. This tool was accelerated using GPUs and converted into a Fiji plugin available for download as a Jar file, or from a Fiji update site.

J.Chalfoun, "A power stitching tool", SPIE Newsroom, 2014, DOI: 10.1117/2.1201402.005365 (download pdf) (view article)

T. Blattner et. al., "A Hybrid CPU-GPU System for Stitching of Large Scale Optical Microscopy Images", 2014 International Conference on Parallel Processing, 2014 (download pdf) (view article)

J. Chalfoun, et al. "MIST: Accurate and Scalable Microscopy Image Stitching Tool with Stage Modeling and Error Minimization". Scientific Reports. 2017;7:4988. doi:10.1038/s41598-017-04567-y (download pdf) (view article)