Todd Sharp "Uploads His Brain to the Cloud" via an ESP-12, to Analyze Gaming Performance

from hackster.io

Developer Todd Sharp has been working on a project dubbed Brain To The Cloud (BTTC), "uploading [his] brain to the cloud" using an Espressif ESP-12 linked to an off-the-shelf "toy" electroencephalogram headset — creating a sort of FitBit-for-the-brain.

"The fact that the Mind Flex contains a real, legit EEG chip that read your mind seemed almost too good to be true," Sharp explains of his interest. "Instead of just reading the data, or using the data to 'control' something else, what if I were to read the data while performing some task and see what the data reveals about my performance during that task? I would need to complete an activity with quantifiable data to properly compare the brain activity to the task results to see if my attention levels correlated to the task's success or failure."

Sharp picked a pleasurable activity to monitor — playing Call of Duty — and set about modifying the headset to stream its data to the cloud for analysis, to see if there's a relationship between the cognitive functions measured by the Mind Flex and in-game performance.

"I decided to use a much smaller ESP-12 board […] because the ESP-12 board contains onboard Wi-Fi, and I figured I'd be able to keep it self-contained inside the Mind Flex housing," Sharp explains. "So my approach was to use the Mind Flex to capture my brain activity by connecting the 'transmit' pin on the EEG chip to the 'receive' pin on the ESP board and program the board to upload the readings to an MQTT topic."

"Separately, I'd need to capture my videogame stats via scheduled calls to the 'unofficial' Call of Duty API. I created a Micronaut application and deployed it to the cloud to do that, with a task that runs every hour to persist the data into my Autonomous DB instance. Now, I know what you're thinking. Indeed, this project required a very grueling, demanding, and time-consuming data collection effort. And - you're right. At least, that's what I told my wife as I the next month playing Call of Duty with the Mind Flex strapped to my head."

Interestingly, the captured data "seemed to indicate something," Sharp notes: That paying less attention actually improved game performance, above a minimum attention level. "In hindsight, this actually makes sense," he explains. "First-person shooters require attention and focus, but paying too much attention can be detrimental.

"You have to almost intentionally not focus on every little detail happening on the screen and let your brain react to the minor changes in the environment that might indicate the presence of an enemy and redirect your crosshairs in reaction to the visual or auditory stimuli. If you explicitly try to aim and shoot, you often miss the target. You tend to play best when you're reactive instead of proactive."

Sharp's full write-up is available on his blog; the source code for the project has been published to GitHub under the permissive MIT license.