Deep MindRead: Using a Low-Cost Commercial EEG to Classify Brain Activity
During my sophomore year of undergrad, I had the honor and privilege of working under the guidance of Professor Laurent Itti, PhD at the USC iLabs. It is here that I was given the opportunity to conduct research that I’d been dreaming of doing since high school: applying deep learning to electroencephalography (EEG). I was really happy with the results of the research, and I eventually presented at the Harvard National Collegiate Research Conference. I am pleased to write about my research here as well.
Introduction and Overview
The Viterbi iLabs had recently ordered two Emotiv EPOC portable EEGs, and I was allowed to work on any project of my choosing (with the approval of my PI). The Emotiv EPOC has 14 electrodes, each of which can sample data at 128hz or 256hz (though we were unable to successfully capture 256hz data), collecting raw data at a range typically between 4000–4300µV.
I intended to eventually use this data to train a neural network, so I normalized the data by simply subtracting the mean value and dividing by the standard deviation (transforming the data into a unit normal distribution). I then formatted the EEG data into two second chunks, thus blocks of 14×256 (14 channels, 2 seconds at 128hz).
Intuitively, I felt that a convolutional neural network made the most sense for this task (despite EEG data being time-series data) for several reasons. Imagine a neurologist looking at the data — do they look at the time dependence of the data on the EEG, moving their finger along each channel, reading the data sequentially? Usually not. In fact, readings of EEGs by a medical professional are typically feature-based searches; neurologists look for certain shapes and patterns in the EEG (usually irrelevant to the precise placement in time of the data) spatially to gather information.
As a result, I felt it would make the most sense to use a convolutional network as opposed to an LSTM (which I personally feel are outdated anyway) or other recurrent network structure to process this data.
Data Collection
Data was collected from a subject in one of three states: leisure, sleepy, or focused. In the leisure case, the subject was allowed to perform non-stressful, easy tasks, such as browsing social media. In the sleepy case, the subject had their brain waves monitored as they were tired enough to be nearly nodding off from sleep (usually laying down as well). In the focused category, the subject took a practice chemistry exam. About an hour of EEG data was collected per category, and processed using the method described in the previous section (scaling to a unit normal distribution).
For privacy reasons, the data is kept secure and cannot be shared without explicit permission, due to the sensitivity of the data. At the moment deep learning applied to EEG data is a relatively new concept, so it is unknown what could possibly be done with the data. For arguments sake (I don’t personally believe this is possible), let’s imagine that in the future we develop the technology to read a user’s thought from an EEG graph. In that case, the data released publicly decades ago could be read, resulting in a huge violation of privacy!
Training
After some fine tuning, the following convolutional network architecture was chosen:
The input is a 256×14 “image” of the EEG data, and the output is a vector describing the probability that the two-second snippet of EEG data was from a user who was in leisure, sleepy, or focused. The model was trained for 30 epochs using the Adam optimization algorithm. 20% of the dataset was partitioned into a validation set, used to check whether the model appeared to be overfitting. In the graphs below are shown the results of the training.
Results
I expected that the most challenging task of all would be classification between the “focused” category and the “leisure” category, as the “sleepy” graphs were in many cases noticeably different visually from the others.
Surprisingly, this was not the case, and as is evident from the table to the left, binary classification tasks proved easy for the network in two cases. These results seriously surprised me, as the EEG data looks almost like random noise to me (and really, there is a very low signal to noise ratio).
Discussion and Future Goals
I would like to include some personal notes in this discussion, as writing this piece on Medium allows me to put in more personal details of the story behind this research than I could include in an academic paper or poster.
This research was extremely important to me, as it has always been a sort of science-fiction-like dream of mine to one day be able to type, control devices, move robotic limbs, etc., purely from the mind. Particularly, I have made it one of my priorities in life to develop better and faster ways for “locked-in” patients to communicate, or maybe even move again, via their minds.
When the legendary Stephen Hawking was still alive, it took him minutes to write out a single short sentence due to his near total paralysis. I find this utterly unacceptable, and something we must change. I cannot imagine the level of frustration, pain, and suffering these patients experience on a daily basis, and I am determined to find better ways for these patients to communicate. I’m certain it’s possible, and it’s already being researched in places like Stanford.
This, overall, fits into my lofty, grandiose goal to one day find cures (whether in the form of prosthetics, bionics, medication, or other) for blindness, paralysis, and deafness. Dreams and goals like this are the precise reason I strive to one day become a neurosurgeon. I believe that our incredibly complex and developed senses of sight and hearing, paired with our ability to move and communicate ideas through an enormous variety of ways (art, dance, music, speech, singing, acting, writing, etc.) are critical components of what truly make us human. Sure, other developed animals have sight and hearing, sometimes even better than us, but our highly developed brains allow us to experience this, process it, feel complex emotions to what we’ve just experienced, and react through the movement of our bodies. Patients who are blind, deaf, or paralyzed have been unjustly robbed of this human ability, and it is my goal in life to return to them what has been stolen.
Anyway, back to this research. The results were very fascinating and surprising to me, but really not that impressive in a sense. Detecting whether someone is focused, sleepy, or at leisure is usually easily doable just by looking at the person. Doing something more complicated, like guessing what number someone is thinking of, is impossible to do in this manner. In the future I’d like to further this research with a higher quality EEG, to see if it’s possible to gather more precise information about what a user is thinking, or perhaps reconstruct language from thoughts with very precise and expensive EEGs. The latter has already been done somewhat via intracranial EEG, but it is unclear whether the same can be done externally, non-invasively.
The future looks bright.
