By combining 24 smartphone cameras into a single platform and putting the images together in a form of digital patchwork, researchers succeeded in creating a single camera capable of taking gigapixel images over an area about the size no larger than a piece of paper.
Building on this design, and forming a startup company, the researchers have perfected the process of connecting together dozens of individual cameras with subpixel resolution. The process has advanced to enable a simultaneous rendering of the height of objects as well.
This is possible in the form of a new high-speed, 3D, gigapixel microscope termed a Multi Camera Array Microscope (MCAM). The level of detail can reveal the behaviour of dozens of freely swimming zebrafish or the grooming activity of fruit flies, down to near cellular-level.
Not only is rich detail provided; images also appear across a very wide field of view. This is made possible by virtue of the MCAM now comprising of 54 lenses. The multiple cameras achieve up to 230 frames per second at a spatiotemporal throughput exceeding 5 gigapixels per second.
The realisation of these images is made possible through the use of supporting software that provides the ability to take 3D measurements.
Commenting on the breakthrough, Roarke Horstmeyer, assistant professor of biomedical engineering at Duke University, states: “It’s like human vision…If you merge multiple viewpoints together (as your two eyes do), you see objects from different angles, which gives you height. When our colleagues studying zebrafish used it for the first time, they were blown away. It immediately revealed new behaviors involving pitch and depth that they’d never seen before.”
The software uses algorithms that connect physics with machine learning in order to capture each video stream from each camera and provide a 3D behavioural image across space and time.
An example of the application is where scientists study the behaviour of zebrafish exposed to neuroactive drugs. This includes assessing changes in behaviour due to different classes of drugs, and hence enabling new potential treatments to be screened.
Going forwards, marrying the 3D and fluorescent imaging capabilities of the microscope, promises a new course in terms of how developmental biologists will conduct their experiments.
The research appears in the journal Nature Photonics. The research paper is titled “Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second.”
