A new artificial intelligence tool can analyze brain scans to guess what a person is watching and recreate that image with remarkable precision. The model can also predict a person brain activity based on what they are viewing. Michal Irani developed the tool alongside colleagues at the Weizmann Institute of Science in Rehovot, Israel.
High-resolution scans drove accuracy
Neuroscientists have spent years trying to reconstruct what people see. Early attempts produced blurry images. Irani and her team used newer datasets collected from high-resolution fMRI scanners where each voxel covers roughly one cubic millimeter of neurons.
The brain decoder relies on two branches. One branch predicts image structure while the second predicts content. A diffusion model then uses these predictions to produce an accurate representation of the original sight.
Training required synthetic data
To acquire enough training data, the team built a universal brain encoder that predicts brain activity from images. Using both the encoder and decoder together allowed the researchers to train their models on unpaused images. Around 70 percent of the training data comes from images not originally paired with fMRI scans.
The system requires only one hour of calibration data for a new person. Previous tools typically needed about 40 hours of data. The findings were presented at the Cognitive Computational Neuroscience conference in New York.
Experts warn of privacy risks
Judy Illes called the work magnificent and noted its potential for neurologic conditions. However, Tommy Sprague warned that similar methods could surreptitiously extract thoughts. Marcello Ienca added that moving to EEG caps could make unauthorized data extraction easier.
Irani wants to expand the technology beyond static images into video and audio. She also aims to decode thoughts, imaginations, and dreams. The team plans to continue developing the technology.
