- Joined
- Sep 5, 2023
I've come to show you a new technological horror. The age of mind-reading is upon us, and it's accessible even to regular consumers like you and me.
1. EEG machines
An EEG machine is an electroencephalogram machine. In layman's terms, this is a machine that reads electrical impulses from your body and converts them into a form that can be treated like any other data by your computer. There are a number of EEG machines available on the market, but until recently, these machines have been pretty inaccessible. They're mostly used in medical settings, and so most of them carry with them the inflated price tag that you would expect from medical machinery. However, there have been some efforts to "democratize" (for lack of a better word) these machines in recent years.
The OpenBCI project is probably the most notable, and has been around for a little while now. It's basically a guide on how to produce your own EEG machine, along with software to run the machine. This has definitely made EEGs more accessible to a hobbyist audience, but there are still a number of challenges in working with a tool like this. You have to manufacture it yourself, which is already enough of a pain in the ass, but because this is working with noisy data taken directly out of the human body, that means if you fuck up, the machine might malfunction in unexpected ways. You could end up with a machine that appears to work properly, but is actually feeding in garbage data.
For this project I've spent the last few months on, I chose to use a different device, the Neurosity Crown. This is a simple yet functional EEG machine that works straight out of the box and allows developers direct access to the data that the machine collects (something that, bafflingly, is not given to hobbyist devs of other commercially available hobbyist EEG machines). This device comes with a somewhat hefty price tag, given the fact that it's not really useful for very much. I paid about $1200 for mine. Although I could've had a cheaper device if I built one myself, this likely saved me a lot of tedious debugging work.
2. Data & model architecture
For this project I wrote my own novel neural network architecture to take the input data and transform it into predictions about the state of my brain. First, I'll explain what the data looks like, and then I'll explain how this architecture processes the data into prediction data. (Please note though, I'm not a neuroscientist, and so my explanations of what the data means might not be entirely accurate.)
The Crown produces 4 types of data - raw, raw unfiltered, power spectral density data, and power by band data.
The raw and raw unfiltered data are pretty much exactly what they sound like. It's just raw data, taken from the skin on my head through some electrodes and converted to numbers. The "raw" data isn't really raw though, as it has some noise filtration applied to it to make interpreting the data simpler, hence raw and raw unfiltered.
The PSD data refers to the distribution of signal power across different frequencies. This contains a sort of breakdown of the raw data into chunks, as I understand it.
The PBB data breaks down the data into specific frequency bands. These represent delta, theta, alpha, beta, and gamma waves, which generally indicate things like alertness, sleepiness, focus, coordination, attention, etc.
The training process for this project was broken into two steps. Because the data is very complex, and because for any given feature that you want to predict, most of the input data is meaningless, the model learns in two stages.
In the first stage of learning, the model works as an encoder-decoder model. The full data from the EEG is fed in as an (808,) array of floats. It's then split into its constituent parts and each part is compressed by the neural network into some even more inscrutable internal state. Then, the model must learn to reconstruct the data back into a similar state as it started with. In a sense, it's learning how to compress the data, keeping the important features and discarding the unimportant features. This is generally referred to as a variational autoencoder.
The architecture I designed has a bit of a twist though. In my experimentation I realized that the model performed better if, during this pretraining stage, rather than reconstruct the original data, the model had to 1) denoise a fuzzy version of the input data back into a "clean" version and 2) predict the next state of the model, i.e. predict what my brain activity will look like a fraction of a second later.
Here's a diagram which shows how the model looks during the pretraining stage.
After the first stage of training, the two decoders that denoise the input and predict the next state are discarded. If the pretraining stage has gone well, then the "encoder out" layer contains dense and useful information about the state of my brain activity.
Then, a simple classification model can be attached to the end of the encoder, which can efficiently predict things about my brain activity based on very little training data.
This type of architecture is ideal, because it allows me to use unlabelled data to teach the model about what a "typical" brain state looks like, so later on it can distinguish between minute details with ease. Again, most of the input data at any given time is completely irrelevant to whatever task I'm trying to do with the data, so this process significantly lowers the barrier for training downstream classifiers.
3. Mind reading
In total, I used about 2 million pretraining samples to train the encoder part of the model. Now I only need a couple thousand samples of data and I can make predictions about downstream tasks. Here's some of the projects I did with this.
In this scatter plot, the blue dots are samples where I was thinking about moving my left arm, and the red dots are samples where I was thinking about moving my right arm. Note that this isn't me actually moving my arms, it's me imagining moving my arms. This is a t-SNE encoded visualization of the output from the encoder, reduced to 2 dimensions for easier interpretation. You can see that there is a clear segmentation, and the model performed at 100% accuracy for this task.
Here's another visualization of similar data. In this sample the data is visualized in 3D. Red dots are me thinking about raising my right arm, and blue dots are control data.
This is another project I did with the EEG. In this, I used control data (purple on the ground truth graph) and data where I had ingested modafinil (yellow on the ground truth graph) and visualized the results using PCA. On the left, you can see the model's predictions about whether or not I was on modafinil, and on the right, you can see the ground truth data. This model predicts a value between 0 and 1, 1 indicating modafinil and 0 indicating sobriety. I used 0.5 as the cutoff point, and in this project, I achieved 99.57% accuracy. You can see, towards the left end of the cluster, the couple of data points where the model made some false negative predictions, and towards the center, where it made some false positive predictions.
These models don't seem to generalize well, at least with the amount of training data I have. However, it stands to reason that if trained on unlabelled training data from a large number of people, the model could likely make even better predictions, even across individuals it has never encountered before.
4. Conclusion & future projects
Taking this back around to the subject of most threads on KF, my next project is going to attempt to measure the emotional impact that specific types of content have on me. I'd like to see what consumption of lolcow content does to me in a measurable way. Not just lolcow content of course, all types of content, but particularly lolcow content, since I love involving lolcow data in all of my hobby projects. If you have any suggestions, I'd love to hear them.
Working with this device has taught me a lot and been a lot of fun, but my prediction for the future is somewhat bleak. My ability to do this as a hobbyist was limited, both by the comparatively low-end hardware I'm using versus medical technology, and by my limited knowledge of the functions of the brain. I'm happy to report that with external reading devices like this, the data is unavoidably noisy, and so this likely means that we won't see true sci-fi mind-reading technology that can reconstruct images/speech directly from your thoughts, at least without implanted EEG devices. However, the progress of things like Neuralink is troubling. I'm stunned at how feasible it is to make useful, accurate predictions about mental states with this data, and with cleaner data, the sky is the limit. Getting a brain implant means your thoughts are an open book which anyone could thumb through at their leisure.
I am not looking forward to being arrested for wrongthink, ripped straight from my brain without my consent. How much longer will "hide your power level" even be possible? I'm not sure. Just don't let the government man chip your head, that's all I can say for sure
1. EEG machines
An EEG machine is an electroencephalogram machine. In layman's terms, this is a machine that reads electrical impulses from your body and converts them into a form that can be treated like any other data by your computer. There are a number of EEG machines available on the market, but until recently, these machines have been pretty inaccessible. They're mostly used in medical settings, and so most of them carry with them the inflated price tag that you would expect from medical machinery. However, there have been some efforts to "democratize" (for lack of a better word) these machines in recent years.
The OpenBCI project is probably the most notable, and has been around for a little while now. It's basically a guide on how to produce your own EEG machine, along with software to run the machine. This has definitely made EEGs more accessible to a hobbyist audience, but there are still a number of challenges in working with a tool like this. You have to manufacture it yourself, which is already enough of a pain in the ass, but because this is working with noisy data taken directly out of the human body, that means if you fuck up, the machine might malfunction in unexpected ways. You could end up with a machine that appears to work properly, but is actually feeding in garbage data.
For this project I've spent the last few months on, I chose to use a different device, the Neurosity Crown. This is a simple yet functional EEG machine that works straight out of the box and allows developers direct access to the data that the machine collects (something that, bafflingly, is not given to hobbyist devs of other commercially available hobbyist EEG machines). This device comes with a somewhat hefty price tag, given the fact that it's not really useful for very much. I paid about $1200 for mine. Although I could've had a cheaper device if I built one myself, this likely saved me a lot of tedious debugging work.
2. Data & model architecture
For this project I wrote my own novel neural network architecture to take the input data and transform it into predictions about the state of my brain. First, I'll explain what the data looks like, and then I'll explain how this architecture processes the data into prediction data. (Please note though, I'm not a neuroscientist, and so my explanations of what the data means might not be entirely accurate.)
The Crown produces 4 types of data - raw, raw unfiltered, power spectral density data, and power by band data.
The raw and raw unfiltered data are pretty much exactly what they sound like. It's just raw data, taken from the skin on my head through some electrodes and converted to numbers. The "raw" data isn't really raw though, as it has some noise filtration applied to it to make interpreting the data simpler, hence raw and raw unfiltered.
The PSD data refers to the distribution of signal power across different frequencies. This contains a sort of breakdown of the raw data into chunks, as I understand it.
The PBB data breaks down the data into specific frequency bands. These represent delta, theta, alpha, beta, and gamma waves, which generally indicate things like alertness, sleepiness, focus, coordination, attention, etc.
The training process for this project was broken into two steps. Because the data is very complex, and because for any given feature that you want to predict, most of the input data is meaningless, the model learns in two stages.
In the first stage of learning, the model works as an encoder-decoder model. The full data from the EEG is fed in as an (808,) array of floats. It's then split into its constituent parts and each part is compressed by the neural network into some even more inscrutable internal state. Then, the model must learn to reconstruct the data back into a similar state as it started with. In a sense, it's learning how to compress the data, keeping the important features and discarding the unimportant features. This is generally referred to as a variational autoencoder.
The architecture I designed has a bit of a twist though. In my experimentation I realized that the model performed better if, during this pretraining stage, rather than reconstruct the original data, the model had to 1) denoise a fuzzy version of the input data back into a "clean" version and 2) predict the next state of the model, i.e. predict what my brain activity will look like a fraction of a second later.
Here's a diagram which shows how the model looks during the pretraining stage.
After the first stage of training, the two decoders that denoise the input and predict the next state are discarded. If the pretraining stage has gone well, then the "encoder out" layer contains dense and useful information about the state of my brain activity.
Then, a simple classification model can be attached to the end of the encoder, which can efficiently predict things about my brain activity based on very little training data.
This type of architecture is ideal, because it allows me to use unlabelled data to teach the model about what a "typical" brain state looks like, so later on it can distinguish between minute details with ease. Again, most of the input data at any given time is completely irrelevant to whatever task I'm trying to do with the data, so this process significantly lowers the barrier for training downstream classifiers.
3. Mind reading
In total, I used about 2 million pretraining samples to train the encoder part of the model. Now I only need a couple thousand samples of data and I can make predictions about downstream tasks. Here's some of the projects I did with this.
In this scatter plot, the blue dots are samples where I was thinking about moving my left arm, and the red dots are samples where I was thinking about moving my right arm. Note that this isn't me actually moving my arms, it's me imagining moving my arms. This is a t-SNE encoded visualization of the output from the encoder, reduced to 2 dimensions for easier interpretation. You can see that there is a clear segmentation, and the model performed at 100% accuracy for this task.
Here's another visualization of similar data. In this sample the data is visualized in 3D. Red dots are me thinking about raising my right arm, and blue dots are control data.
This is another project I did with the EEG. In this, I used control data (purple on the ground truth graph) and data where I had ingested modafinil (yellow on the ground truth graph) and visualized the results using PCA. On the left, you can see the model's predictions about whether or not I was on modafinil, and on the right, you can see the ground truth data. This model predicts a value between 0 and 1, 1 indicating modafinil and 0 indicating sobriety. I used 0.5 as the cutoff point, and in this project, I achieved 99.57% accuracy. You can see, towards the left end of the cluster, the couple of data points where the model made some false negative predictions, and towards the center, where it made some false positive predictions.
These models don't seem to generalize well, at least with the amount of training data I have. However, it stands to reason that if trained on unlabelled training data from a large number of people, the model could likely make even better predictions, even across individuals it has never encountered before.
4. Conclusion & future projects
Taking this back around to the subject of most threads on KF, my next project is going to attempt to measure the emotional impact that specific types of content have on me. I'd like to see what consumption of lolcow content does to me in a measurable way. Not just lolcow content of course, all types of content, but particularly lolcow content, since I love involving lolcow data in all of my hobby projects. If you have any suggestions, I'd love to hear them.
Working with this device has taught me a lot and been a lot of fun, but my prediction for the future is somewhat bleak. My ability to do this as a hobbyist was limited, both by the comparatively low-end hardware I'm using versus medical technology, and by my limited knowledge of the functions of the brain. I'm happy to report that with external reading devices like this, the data is unavoidably noisy, and so this likely means that we won't see true sci-fi mind-reading technology that can reconstruct images/speech directly from your thoughts, at least without implanted EEG devices. However, the progress of things like Neuralink is troubling. I'm stunned at how feasible it is to make useful, accurate predictions about mental states with this data, and with cleaner data, the sky is the limit. Getting a brain implant means your thoughts are an open book which anyone could thumb through at their leisure.
I am not looking forward to being arrested for wrongthink, ripped straight from my brain without my consent. How much longer will "hide your power level" even be possible? I'm not sure. Just don't let the government man chip your head, that's all I can say for sure
