AI-enhanced processing promises boost for biomedical imaging
The growing importance of the interface between computational optical imaging and AI for biomedical applications is addressed by a series of more then 80 presentations which took place at Photonics West between January 27-29 under the umbrella title of Computational Optical Imaging and Artificial Intelligence in Biomedical Sciences.
Prof. Liang Gao, Associate Professor at UCLA’s Samueli School of Engineering, who is also one of the conference Chairs, told Show Daily about the rise of this sector and then pointed out some of the program highlights.
“This is a new conference. In recent years there has been a hot trend in bringing AI-assisted technologies into the biomedical imaging,” he said. “We’re looking at whether the AI-based computational advances in vision and graphics can be introduced into biomedical imaging fields to solve the current bottlenecks.”
Advances in high-resolution microscopy, including computational imaging microscopy, have enabled especially high resolution, large field of view imaging. At the same time there have been huge advances in AI-based algorithms for biomedical image analysis. This combination is the background to the new Photonics West conference — “to bring together experts from different fields to brainstorm the next 10 years for this field,” said Prof. Gao.
Conference chair: Prof. Liang Gao, Associate Professor at UCLA’s Samueli School of Engineering. Credit: UCLA.
A related issue is that while the technical performance of lenses and optical hardware may have reached a limit, imaging results can still be improved by using algorithms to enhance the image. Prof Gao added, “We can improve biomedical images even more by using AI to capture and then predict better results.”
In constructing the new conference program, the panel of chairs, which also included Guoan Zheng from the University of Connecticut and Seung Ah Lee from Yonsei University (Korea), has chosen a variety of topics spanning components, AI-driven hardware designs, and computational optical imaging, and, especially the interface between hardware and software.
“There are many research groups looking at these areas, trying to integrate the AI-designed optics into hardware. Their aim is that the AI-integrated hardware can achieve a much better performance than before,” said Prof. Gao.
Certain groups in this field are trying to replace certain parts of AI with hardware in order to perform optical computing more rapidly. This has been pioneered by Aydogan Ozcan’s group at UCLA; Prof. Ozcan is a keynote speaker in the conference.
Key speaker: Prof. Aydogan Ozcan. Credit: UCLA.
Prof. Ozcan’s significant presentations included: Diffractive optical networks enable quantitative phase imaging through random, unknown diffusers; Diffractive visual processors (keynote); and Light-field tomographic fluorescence lifetime imaging microscopy.
Accuracy of enhanced images
A key question in this field is, if the quality of a biomedical image is enhanced by using AI software, how can a researcher be confident that such a software-generated image is an accurate representation of the sample under investigation?
Prof. Gao explained, “There are different approaches. One way to make AI-generated medical images representative is to ensure that the AI models get sufficient training datasets so a model generated can be generally appliable to different samples. Also, regarding how to improve the performance of AI, we can potentially build in some hardware components into the software as well. In this way we can not only make the computation faster but we can also save costs on the computational resources.”
Besides the research and theoretical approach to this field, does the new conference feature speakers who are working in the commercial side of enhancing biomedical images with AI and software techniques?
Prof. Gao gave some examples, saying that certain speakers do have this knowledge and experience: “For example, Dr. Ji Yi from Johns Hopkins University is speaking on imaging mesoscopic microscopy. His research focuses on computational OCT and he has a huge interest in retinal imaging applications; he will likely talk about how such computational OCT can be applied to human imaging.”
Dr. Ji’s invited paper is entitled, “Large-scale dynamic imaging by mesoscopic oblique plane microscopy and computational augmentation.” Another expert on the commercialization of AI-enhanced imaging is Dr. Lei Li from Rice University, who presented an invited paper entitled “Machine-learning enhanced photoacoustic computed tomography.”
A notable speaker from industry was Dr. Shalin Mehta, who is Platform Leader at the Chan Zuckerberg Biohub, a group on nonprofit research institutes across the USA, that brings together physicians, scientists, and engineers with the goal of pursuing grand scientific challenges with 10-15 year horizons.
Prof. Gao said, “Yet another example of a downstream expert speaking to the conference is Assistant Prof. Roarke Hortsmeyer from Duke University, who is also on the conference committee. He has a company commercializing these computational gigapixel cameras so this is a good example of connecting the research to industry and translating these computational optical devices into commercial platforms.”
Prof. Roarke’s presentation is entitled, “High-throughput computational microscopy with diffractive multiplexing across a gigapixel sensor array.”
Liang’s LIFT-FLIM system to be showcased
Prof. Gao’s UCLA group was also the subject of presentations: “It concerns a new computational fluorescence lifetime imaging microscopy technique, based on a collaboration between Profs. Gao and Ozcan on this paper.”
LIFT-FLIM will be described in a paper “Light-field tomographic fluorescence lifetime imaging microscopy,” presented on January 29 by Prof Gao’s student Yayao Ma. The presentation covered some of the enhancements offered by the LIFT-FLIM systems.
Fluorescence lifetime imaging microscopy (FLIM) measures fluorescence lifetimes of fluorescent probes to investigate molecular interactions. However, conventional FLIM systems often require extensive scanning that is time-consuming. To address this challenge, Dr. Gao’s group developed a novel computational imaging technique called light field tomographic FLIM (LIFT-FLIM). Their approach acquires volumetric fluorescence lifetime images in a highly data-efficient manner, significantly reducing the number of scanning steps and, thereby, remarkably improving the 3D FLIM frame rate.
Prof. Gao explained further, “It’s AI-based fluorescence-lifetime imaging microscopy (FLIM). FLIM is a powerful technique in the biomedical sciences, because it can yield functional information instead of just structural information. But its drawback is that it is rather slow — it requires raster scans across three-dimensional volumes — because it has to measure fluorescence decay time instead of only fluorescence intensity.
“So we developed a new approach, which combines light field imaging with AI enabling us to extract three-dimensional FLIM images from only one-dimensional measurements. We’re using the line SPAD camera, which has single photon sensitivity.
“We use AI so we can measure these three-dimensional objects. Actually, we call it four dimensional — three spatial dimensions and one fluorescence lifetime dimension. This is made possible only by AI with these computational approaches. They cannot be done by conventional optical approaches, so this is one example of why AI so important to enable new functionalities for bio-imaging.”
Potential applications
Prof. Gao explained that one potential end use of this new “4D” imaging process, could be label-free cancer imaging: “In our paper we demonstrate that this method can be used to differentiate normal tissues from cancerous tissues without using any contrast agent — only based on the metabolisms of tissue because cancerous tissues normally have higher metabolic rates than normal tissues.
“This kind of difference can be detected through the fluorescence lifetime. With our approach we can perform this wide field computational imaging, so we can do real time three-dimensional lifetime measurement. Potentially it could be translated to intra-operative imaging, and cancer imaging applications. It can also be used for the high throughput drug screening applications, for which metabolic screening is also a hot topic at the conference in that area as well, for drug discovery,” he said.
Commercial routes
With commercialization in mind, Prof. Gao also revealed that in April, 2023, he and UCLA partners had set up a new company, Lift Photonics, in Los Angeles, CA. They are trying to push these technologies from the lab toward the commercial market.
“There is a huge opportunity for AI in this field,” he said. “These kinds of devices can be used for analyzing artifi-cial image inflows, so they can accelerate the counting rate and the classification rate of objects of interest.”
Similarly, for other microscopy or endoscopy systems, some developers are trying to build in AI into the hardware in order to save the time for later data processing, to generate results more rapidly, which is crucial for this kind of a medical intra-operative imaging applications.
There is also commercial potential in digital pathology because for conventional pathology the turnaround time is slow. While for the AI-assisted optical imaging, the whole process can be significantly accelerated.
“Computational imaging has been widely applied into different areas; it’s a mature science in use to investigate these surface chemistries, electron couplings, and those kind of applications. It’s also suitable for certain industrial inspec-tions — and remote sensing applications are another huge area for computational imaging,” he said.
“In remote sensing we want to capture spectral imaging data set using a computational approach to make this data acquisition more efficient. This is known as computational spectral imaging.”
This topic was described in a paper “Compressive hyperspectral imaging,” presented on January 27 by Prof. Gao’s student Qi Cui. The presentation covered some of the enhancements offered by computational spectral imaging systems.
So where are these developers and users obtaining their AI software; are they creating it themselves or are they buying off the shelf packages?
Prof. Gao said he believes that most of them are creating it themselves but not developing it from scratch: “They adapt some existing machine learning models from the computational graphic or computer vision fields and adapt these models specifically for biomedical imaging applications. Considering most of the researchers in our field, we are not naturally computational scientists!”
He thinks that generative adversarial networks (“GANs”) are probably the most widely used machine learning models being used for this kind of image transformation. Some other groups use it for super resolution applications. For these GAN-based networks, it was originally developed for the motion-vision or fake image detection.
Problems yet to solve
Considering the main problems remaining to be solved in his field, Prof. Gao commented, “For people using AI for biomedical imaging, the biggest bottleneck is how to make the results more reliable. In medical diagnosis and interventions, the tolerances are very tight, especially if you want to use an AI assistant for operating images for surgery. In that case, a patient’s life relies on artificial intelligence so reliability is a big issue.”
So this is actually one of the huge challenges of the current field — how to make the modelling reliable enough for diagnosis and medical applications and how to make training data sets reliable, so that modelling can be generalized to a more representative population instead of only for a few groups.
“We should consider the introduction of AI or software approach to the hard-ware designs and constructions. Considering conventional imaging hardware it generates images that are just a replica of the object. But for the new AI assistant approaches, the generated images can be an encoded image, which can be the spatially multiplexed image.”
He concluded, “So with some smart AI processing data processing, gathered data can generate more rich information content — to make this whole measurement pipeline more data-efficient — and accurate.”
This article was originally published in the Photonics West Show Daily in January 2024.
| Enjoy this article? Get similar news in your inbox |
|
