Real-time fMRI decoding: Reading minds using brain imaging
Abstract
Real-time functional magnetic resonance (rtfMRI) imaging is a relatively new technique to compute moment-to-moment changes in brain activations corresponding to a particular experimental manipulation. When coupled with modern machine learning techniques, rtfMRI becomes a powerful tool to read brain states in real time. Two separate experiments were conducted in this study to answer two broad research goals.
The first experiment was conducted to investigate if visual perception and imagination can be decoded in real-time and if presenting neurofeedback of the decoding results can help improved decoding/task performance. 25 elastic net logistic regression classifiers were trained on fMRI responses for 100 random flickering random patterns on a 5 x 5 grid in order to learn the stimulus-to-cortical-activation mapping. The trained classifiers were then used to predict novel stimuli perceived and imagined on the same 5 x 5 grid used during training. Each perception and imagination conditions were performed twice: once with feedback and once without it. The results of the study indicate that visual perception can be decoded in real-time with an average accuracy of 55%. No significant decoding accuracy was obtained for imagined stimuli. Furthermore, lower decoding accuracies were obtained for conditions with feedback compared to the same conditions without feedback. However, the results of this experiment cannot be trusted as some critical mistakes were made in its design and implementation including selecting a wrong voxel size and using a very long visual cortex stimulation (12s) during training. All this led to poor performance in the experiment. Furthermore, the results indicate that subject motivation and experiment duration has a significant impact on task performance during the experiment.
The second experiment was conducted to find if one of the two competing stimulus categories can be attended to and decoded in real time, and if presenting the feedback of decoding can assist in improving task performance. The classifier was trained on pictures of famous faces and places. Subjects were then shown a 50/50 hybrid of a picture of a famous face and a famous place and asked to attend to only one of them. The attended picture was then decoded in real-time and the hybrid mix was updated at every TR (repetition time) such that if the prediction was right, the non-target picture will fade out whereas the target picture will get enhanced, and vice-versa. This is what happened in trials in feedback. In trials without feedback, the hybrid mix remained at 50/50 % mix all times during a trial. The results of the study indicate that the attended category can be decoded with very high accuracy (78.5%). Furthermore, the feedback of the decoded category has no influence on the decoding accuracy. The feedback does however, induce a snow ball effect where one classification leads to a stream of similar classifications in subsequent TRs in a trial. Moreover, the results also indicate that the transition period of the BOLD activity has a reliable structure that can be decoded with above chance level accuracies, thereby decreasing the real-time fMRI delay by as much as 6s.
Apart from these two experiments, the real-time fMRI architecture used for executing real-time fMRI experiments was developed and refined. Moreover, tools for efficient analysis and visualization of fMRI and rtfMRI data were also developed.
Download
Loading...
Loading...
Reload document 
Open in new tab