Spectral Kernel Machine (SKM) for intelligent spectral machine vision. The SKM device integrates a spectral encoder with an electrically tunable bipolar photodetector to perform spectral analysis directly at the point of photodetection. This architecture enables diverse applications, including plant hydration sensing, object segmentation, and chemical mixture analysis, using only the sensor's photocurrent, without capturing or processing a traditional hyperspectral data cube. Credit: Science (2025). DOI: 10.1126/science.ady6571
Researchers at the University of California, Los Angeles (UCLA), in collaboration with UC Berkeley, have developed a new type of intelligent image sensor that can perform machine-learning inference during the act of photodetection itself.
Reported in Science, the breakthrough redefines how spectral imaging, machine vision and AI can be integrated within a single semiconductor device.
Traditionally, spectral cameras capture a dense stack of images, each image corresponding to a different wavelength, and then transfer this large dataset to digital processors for computation and scene analysis. This workflow, while powerful, creates a severe bottleneck: the hardware must move and process massive amounts of data, which limits speed, power efficiency, and the achievable spatial–spectral resolution.
How spectral kernel machines work
The new device platform, called spectral kernel machines (SKMs), completely bypasses this bottleneck. Instead of recording large data cubes, SKMs directly encode the spectral and spatial information into the output photocurrent, allowing the sensor itself to perform the task of identifying materials, chemicals, and objects within a complex scene.
"This process mathematically resembles the kernel-machine algorithms used in machine learning," said Aydogan Ozcan, Chancellor's Professor of Electrical and Computer Engineering at UCLA and the co-corresponding author of the study.
"Through this SKM device, we have unlocked optoelectronic sensors that can learn and compute without the need for digital post-processing."
The work was conducted in close collaboration with Professor Ali Javey's research group and Dr. Dehui Zhang at Lawrence Berkeley National Laboratory and UC Berkeley.
Training and performance of SKM devices
Each SKM device can be electrically tuned to enhance or suppress specific spectral signatures. During training, the researchers displayed sensor images, such as colorful birds in forest scenes, allowing the SKM device to randomly sample a subset of the pixels while receiving simple external commands like "identify bird" or "identify background."
From these examples, the device learned the optimal electrical control sequence to highlight bird pixels and suppress background regions. When later presented with new images never seen before, the sensor produced a positive photocurrent only for pixels belonging to the target object, demonstrating that it had learned from prior examples and could "sniff and seek" desired features, much like a retriever dog.
Applications and future potential
The team demonstrated that SKM devices can intelligently sense and analyze complex scenes across the visible to mid-infrared spectrum without relying on conventional hyperspectral image stacks.
In the visible band, silicon-based photoconductors performed semiconductor wafer metrology tasks and feature identification, offering speed and power advantages over traditional digital hyperspectral machine vision pipelines. In the mid-infrared, a room-temperature, electrically tunable photodiode enabled chemical identification and the analysis of mixtures.
The researchers further showcased applications such as plant-leaf hydration sensing and object segmentation, all derived directly from the sensor's photocurrent without the need to capture or process a hyperspectral data cube.
This technology marks a new paradigm in spectral machine vision. By embedding intelligence and machine learning directly into the physics of photodetection, SKM devices eliminate data-movement bottlenecks and significantly reduce energy consumption, enabling ultrafast spectral analysis in a compact, low-power form.
These capabilities make SKMs ideal for mobile devices, autonomous robots, environmental monitoring, industrial inspection, and satellite imaging, among other applications.
"This work redefines photodetection as an automatic physical computation," said Yuhang Li, a graduate student at UCLA, who is a co-author of the work. "We can now rapidly perform complex spectral analysis, directly where the photons are first detected."
The first author of this work is Dr. Dehui Zhang of UC Berkeley, and the corresponding senior authors are Professor Aydogan Ozcan of UCLA and Professor Ali Javey of UC Berkeley.
More information: Dehui Zhang et al, Spectral kernel machines with electrically tunable photodetectors, Science (2025). DOI: 10.1126/science.ady6571
Provided by UCLA Engineering Institute for Technology Advancement