I’m dumbfounded: UC Berkeley scientists have developed a system to capture visual activity in their brains and reconstruct it as digital video clips. Eventually, this process will allow you to record and reconstruct your own dreams on a computer screen.
Also: Eavesdropping By Reading Brainwaves Is Feasible
According to UC Berkeley neuroscientist and coauthor of the research published today in the journal Current Biology “this is a major leap toward reconstructing internal imagery. We are opening a window into the movies in our minds.” Indeed, this is mind-blowing. I’m simultaneously excited and terrified. This is how it works:
They used three different subjects for the experiments, which were part of the research team because it requires to be inside a functional Magnetic Resonance Imaging system for hours at time. Inside the machine, they were exposed to two different groups of Hollywood movie trailers as the fMRI machine recorded the brain’s blood flow through their brains’ visual cortex.
The readings were fed into the computer program, in which they were divided into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information in the movie to the brain activity. As the session progressed, the computer kept learning about how the visual patterns presented on the screen corresponded to the brain activity.
After recording this information, the activity from the second group of clips was used to reconstruct the videos on the screen. The computer then analysed 18 million seconds of random YouTube video and built a database of potential brain activity for each clip. From all these videos, the software picked 100 clips that looked more similar to the ones that subject watched, combining them into the final movie. Although the movie is low res and blurry, it clearly matches the actual video the subjects were watching.
In this other video, you can see how this process worked in the three people. On the top left you can see the movie the subjects were watching while there were in the fMRI machine. Right below you can see the movie “extracted” from their brain activity, which shows that this technique gives consistent results independently of the human inside the machine. The movies next to that column show the random movies that the computer program used to reconstruct the motion video.
While this is not as simple as just plugging a cable and download your visual memories and dreams, lead research author and one of the lab test bunnies Shinji Nishimoto thinks this is the first step to tap directly into what our brain sees and imagines:
Our natural visual experience is like watching a movie. In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences.
This is the first time in history that we have been able to decode brain activity and reconstruct motion pictures into a computer screen. The path that this research opens boggles the mind. It reminds me of Brainstorm, the cult movie in which a group of scientists lead by Christopher Walken develop a machine capable of recording the five senses of a human being and then play them back. This new development brings us closer to that goal which, I have no doubt, will happen. And, given the exponential increase in computing power, this all will arrive sooner than most mortals expect. [UC Berkeley]