A Primer on Neural Technology

Plus, some personal news

A Personal Update

I’m pleased to inform you all that I will be joining George Mason University’s Mercatus Center next week as a Research Fellow focused on artificial intelligence. In that capacity, I’ll be working with Matt Mittelstaedt and Eileen Norcross on both state-level and federal AI policy issues, and helping to build out Mercatus’ new AI and Progress program. As a longtime fan of many Mercatus scholars, including its Chairman, Tyler Cowen, it is a particular honor to be joining this team.

Since I set on this path six months ago, many of you have helped me by highlighting my work, giving me opportunities to write, and offering advice. I thank you all profusely.

I plan to maintain my regular cadence of posts here on Substack.

Neural Technology: Where We Stand

As AI systems become more capable, perhaps even approaching “superintelligence,” the role of humans in the economy might change profoundly. Perhaps, as Noah Smith has recently suggested, humans will continue to have a comparative advantage over machines. The comparative advantage argument, however, seems to me to assume one of two things: Either machines really do remain fundamentally weaker than humans at some economically valuable tasks (perhaps tasks we cannot even imagine today), or a machine’s time (compute) will be at least somewhat scarce, and hence expensive. If a superintelligent AI system costs, say, $500 per hour to run, then humans will have a comparative advantage at many lower-value tasks.

One can easily imagine both of these assumptions turning out to be false. Perhaps humans will retain a durable comparative advantage at some economically useful work; that would be a fortunate outcome indeed. It is difficult to reason coherently about this, since AGI-level systems do not yet exist; we will have to wait and see. It is essentially impossible to imagine, however, that the cost of artificial superintelligence would remain high in any durable way absent a policy regime that restricts either compute or the energy devoted to compute. A policy regime like that is inherently fragile, because 1)a fundamental innovation in computing, such as quantum, neuromorphic computing, or thermodynamic computing, could increase the energy efficiency of compute by several orders of magnitude, and 2)any country that chose not to observe the policy would have a natural economic advantage in the long term.

If neither of these assumptions ends up proving true, a more durable long-term solution is required. I have argued before that a merge between man and machine is a potentially promising route. In addition to potentially alleviating the economic and labor problem, it also helps to solve the alignment problem by inextricably conjoining the interests of human beings and artificial intelligence.

OpenAI and other firms have been hinting that superintelligent systems are close. The issue of a merge is an under-discussed curiosity today; it may become quite dire someday soon. It seems an appropriate time, then, to take a close look at where neural technologies stand. Can we model the brain? Let’s dive in.

Focus

This is a huge topic, and I’m going to narrow my focus by only discussing technologies that can realistically be incorporated into consumer devices in the nearish term. This means I will not go into depth about fMRI (functional magnetic resonance imaging), PET (positron emission tomography), and MEG (magnetoencephalography), all of which require equipment which is difficult to commercialize with current technology. Each of these, however, can be a valuable source of data for training AI models, a topic to which I will return later.

Furthermore, I am focused on technologies that will augment the cognitive capabilities of able-bodied people. Many of the technologies described below will likely have profound benefits for quadriplegics, people with blindness or other sensory impairments, and a range of other conditions. When they happen, they will resemble miracles; people having their vision restored, or perhaps even learning to see for the first time; people who cannot speak being given a voice; people who cannot walk being given the ability to do so again. Indeed, it is possible that people who currently have these disabilities may one day be among the first to experience superhuman vision, strength, and speed, simply because they will be first in line to try the most cutting-edge neural technologies. All these things seem not just possible, but, with sufficient time, likely. Yet they will not be my focus here.

Steve Jobs is rumored to have insisted that the 1998 iMac be designed with a handle so that it could be thrown out of the window if it started to frustrate its user. For this same fundamental reason (and many others), I believe that non-invasive neural technologies will be preferable to invasive ones. Non-invasive techniques will therefore be my focus here, though I’ll discuss invasive approaches occasionally.

With those constraints in mind, I’ll look at technologies that allow both “read” (understanding brain states) and “write” (changing brain states).  

Understanding Brain States (“Read”)

Fundamentally, most neural technologies understand brain activity by examining its biomarkers. All thought is mediated by electricity, which means that thinking generates electro-magnetic waves that can be measured. The most direct way to gauge neural activity is to measure electrical waves themselves using electrodes. These can be implanted in the brain (intracranial electroencephalography, or iEEG) or placed on the scalp (electroencephalography, or EEG). Neuralink, Elon Musk’s neuroscience company that recently had its first successful human implantation, uses iEEG to allow patients to operate a computer with their mind. Most of the material below focuses on EEG rather than iEEG, but much of it applies to both.

Different brain waves correspond with different levels of neural activity in a (mostly) intuitive way. Higher frequency electrical waves correspond with heightened states of awareness, from deep sleep under 4 Hz (Delta) all the way up to Gamma (30-100 Hz), which corresponds with focus, use of working memory, and learning. The waves’ amplitudes, which can be a proxy for the level of arousal of a particular brain region, must also be taken into account.

Of course, it’s far more complicated than that. Most importantly, your brain is not producing one wave at a time; it produces many, in different regions and of different frequencies and amplitudes both between regions and within one region. Electrodes must therefore be placed on different parts of the head to measure different areas of the brain. In clinical settings, devices can have as many as 256 electrodes, collecting as many as 256,000 data points per second. Consumer devices tend to have between 2 and 32 electrodes, but that still results in hundreds of thousands, if not millions, of data points if the device is worn for any appreciable length of time. Even measured crudely, the brain produces a torrent of data.

AI techniques help make sense of this in real time. Using non-invasive EEG and machine learning, scientists have learned to measure pain perception, attention and fatigue, mood (fairly crudely: happiness, pleasantness, disgust, anger, sadness, etc.), sleep states, detect early onset Parkinson’s with 94% accuracy, and predict seizures up to one hour before a seizure occurs with 99.6% accuracy. Intriguingly, it is also possible to measure motor control impulses; the thoughts associated with moving your arm, for example, can be translated into the motor commands to move in the real world, even if the nerve is severed. This is how some prosthetic arms work. This also can be translated into computer interfaces: It is possible to imagine moving, say, an application window, and for AI and other software to translate those thoughts into the desired movement without any other action by the other. Similarly, it is possible to use your mind to control devices that are not connected to your body, such as robotic arms. Telekinesis, with no brain implants or other invasive technologies, already exists; in fact, it is relatively old hat.

Most academic papers involving EEG and AI use older AI model architectures such as convolutional neural networks and recurrent neural networks. Very few papers, at least that I have seen, make use of the transformer architecture that has undergirded many of the most dramatic recent advancements in AI (such as language models). I have to imagine that transformers could yield significant advancements in making sense of the brain (I’ll return to this topic a bit later), so it is a puzzle to me why we have not seen more papers trying this. Could it really just be academic status quo bias? That seems like an incomplete explanation at best. If anyone reading has a better idea, please do ping me.  

EEG suffers from a few major drawbacks. The first is that the body, and the world around you, generate competing electrical signals that have to be dynamically removed during processing. The second is what is known in the literature as “spatial resolution,” or the level of granularity the technology enables. Non-invasive EEG has a spatial resolution of centimeters, which means millions of neurons. Both of these problems are reduced with intracranial EEG (iEEG), which has a spatial resolution of square millimeters, covering tens or low hundreds of thousands of neurons. The obvious tradeoff of iEEG is that it requires invasive surgery.

EEG suffers from very low penetration; it can only measure activity on the outer surface (the cortex) of the brain. iEEG, on the other hand, can have a penetration of several centimeters depending on the specific type of electrode used.

The other major benefit of iEEG over EEG is that EEG signals must travel through a variety of media before they reach the electrodes on the scalp. Most importantly, this includes the skull itself, which suffers from very low conductivity (“suffers” for our purposes here; you definitely should be grateful that your skull can diffuse electromagnetic waves). The thickness, density, and composition of the skull is not consistent between people or even over the surface of a single skull. These differences must be modeled carefully, a challenge which iEEG does not have to overcome.

Another “read” technology that merits attention is functional near infrared spectroscopy (fNIRS). Like fMRI, fNIRS measures hemodynamics—blood. As brain activity increases, so too does the brain’s consumption of oxygen. This activity can be inferred by measuring the relative presence of oxygenated versus deoxygenated hemoglobin, a protein that carries oxygen via the blood. Higher brain activity in a particular region of the brain leads to more oxygenated hemoglobin relative to deoxygenated hemoglobin in that region. fNIRS are non-invasive devices that use LEDs placed on the scalp to shine near-infrared light into the brain; oxygenated and deoxygenated hemoglobin absorb this light differently, and hence they also scatter the unabsorbed light differently. This scattered light is then measured by photodiodes also placed on the scalp to infer brain activity in different regions.

fNIRS is obviously less intuitive and elegant than EEG, which just measures the brain’s electrical activity directly. However, it has some key benefits. Most importantly, it can peer considerably deeper into the brain with more reliability than EEG: it can measure subcortical activity up to 3 centimeters under the surface of the brain, whereas EEG has much more trouble picking up these signals without substantial attenuation. The spatial resolution is comparable to EEG, while temporal resolution is a bit lower because blood moves more slowly than electrical waves. Like EEG, fNIRS sensors are relatively cheap and easy to work with, and there are several consumer devices on the market employing the technology. The uses cases are often similar to EEG: sleep tracking, detecting biomarkers for cognitive impairment and mental health disorders, and measuring focus/fatigue.

I expect both EEG and fNIRS to improve in the coming years as hardware costs decline and, more importantly, as more advanced AI models begin to be used. In particular, I expect transformer-based models to be able to extract richer information—more detailed thoughts, moods, intents, etc. There may be fundamental limits on how much detail can be determined non-invasively; that is anyone’s guess.

Invasive techniques will be even more capable. While the recent demonstration of Neuralink in a human did not shock anyone who has been paying attention to this field, it is important to see these technologies make it out of the lab and into the real world.

Let’s now turn to the even more complicated issue of how to build devices that can “write” to the brain by modulating brain activity.

Modulating Brain Activity (“Write”)

What does it mean to modify a brain state? In the broadest sense, almost anything or anyone can be thought of as a “neuromodulator” in the sense that they can alter your conscious experience of the world. Neuromodulation devices merely do so mechanically.

Much like the “read” technologies, neuromodulation generally relies on the manipulation of electromagnetic waves to induce particular states of mind. You can think of non-invasive neuromodulation as the opposite of non-invasive brain activity monitoring: devices are placed on the scalp that transmit, rather than measure, signals of various kinds. Transcranial direct current stimulation (tDCS) and transcranial alternating current stimulation (tACS) both do so with electrical signals, while transcranial magnetic stimulation (TMS) does so with magnetism. tACS and TMS are both expensive and currently only viable in clinical settings. tDCS has been sold in some consumer devices.

The problem with tDCS is that it does not work well or consistently because the skull exists to insulate the brain from external signals. It is very hard to deliver targeted stimulation using any of these approaches; the signal tends to diffuse broadly, eliminating the intended effect. None of them seem like promising avenues.

More recently, a different approach has shown considerable promise: transcranial focused ultrasound, or tFUS. tFUS sends high frequency sound waves, far past the range of human hearing, into the brain (note: the sounds are also inaudible to dogs and cats, though other animals, such as rats, can hear them). These sound waves can be targeted at a level of depth and precision that is impossible with the electromagnetic approaches. tFUS has been demonstrated to improve mood, visual acuity, and memory (the latter being in rats). It has even been used to modify visual perception by inducing phosphenes (you know that visual sensation when you shut your eyes and press lightly on them with your fingers? Those are phosphenes. tFUS can trigger them in a person with their eyes open). Better yet, tFUS is inexpensive, and the number of ultrasound elements that can be fit into one device has been increasing at a steady pace.

How tFUS works is not fully understood. Why would high frequency sound waves cause one’s conscious state to change? Some have speculated that it is the result of tiny vibrations caused in brain tissue; others believe it is caused by cavitation, tiny bubbles in brain tissue that can affect brain activity. It may directly trigger neurotransmitters such as dopamine and serotonin to be released. It could, of course, be some combination of these and other mechanisms. The Food and Drug Administration believes it is safe within certain broad limits.

A New York-based startup called Prophetic AI is working to bring a tFUS device to the consumer market. The device is a headband with both electrodes (for EEG) and ultrasound transducers. It can be paired with an app on your phone, which runs a machine learning model Prophetic has created. The model, a transformer (finally!), is trained on combined EEG/fMRI data collected in clinical settings. fMRI, briefly, is the gold standard for brain imaging: It creates 3D images of the entire brain, with the ability to see neuronal activity in detail throughout. The model takes EEG inputs from the headband as inputs, and outputs predicted ultrasound patterns to induce a desired conscious state.

The company is currently focusing on lucid dreaming, which is when a dreamer becomes aware of the fact that they are dreaming and able to control aspects of the dream. They also hope to include modes that can enhance focus and induce a happy mood in users when they are awake. The latter two capabilities have been demonstrated in several studies, while the lucid dreaming functionality is more speculative. They plan to begin beta testing by the summer of 2024. The device is not considered a medical device by the FDA, and is below the agency’s threshold for high-risk levels of ultrasound, so it may be available on digital shelves within a year or so.

Some of the same problems we saw earlier apply to tFUS as well: getting signal through the varied terrain of the skull reliably will be a challenge for any non-invasive neural technology. A technique called audio ray tracing, which Apple’s Vision Pro headset uses to create “theater-like” surround sound out of tiny speakers, is useful here, among other approaches. In the Vision Pro, this involves mapping the geometry of the user’s room and simulating the path a sound wave would follow if it were coming from a specific spot in that room. The same idea has been used to great effect to simulate the behavior of light in video games. (Note: I am unsure if Prophetic is using this approach, but it has been used in scientific labs to improve the performance of tFUS).

Let’s now put all this together and imagine what a futuristic, non-invasive neural interface might look like in practice. Such a device would require substantial surface area over the head to accurately measure and modulate a variety of cortical brain activity. Something smaller, like a pair of ear buds, may be able to perform coarse analysis of brain activity, but I suspect that anything involving manipulation of a computer would require something much larger, like a visual reality headset with electrodes, LEDs, or ultrasound transducers (or some combination thereof) around the band.

This could be paired with a wristband or ring to give the user the sensation of touch when interacting with virtual objects (this is an area of research known as neurohaptics, very promising but omitted from further discussion for brevity). In addition to controlling user interface elements using the brain (or a combination of eye movement and thought), the device could modify its behavior based on your mood, level of attention, etc. It could help to improve both focus and restfulness using neurofeedback, which involves showing users their brain data to help them build up better cognitive habits, often in a gamified manner.

All these capabilities exist as separate devices or have been demonstrated in clinical settings; it is reasonable to imagine them merged into one device. This should be thought of as a minimum level of functionality. What else might be down the road?

It could also monitor your brain continuously for signs of cognitive decline and mental health problems. It could be paired with a “large action model,” an AI model that predicts digital actions to take in response to user intent. While it might not be able to understand detailed thoughts, it could conceivably interpret standard “voice assistant” requests (“Hey Siri, show me my calendar” or “Alexa, turn on my bedroom lights”).

If it were to use tFUS, it could induce a variety of conscious states. It could make you happy, or reflective, or focused. These would likely feel distinct from how these states usually feel; they would not be connected to memories or feel caused by any aspect of your external environment. Rather, you would suddenly feel happy. As AI models improve, you may be able to generate conscious states as easily as you can generate images today with DALL-E or Midjourney. Using the same technology, you may be able to enter states of intense focus during which you could consume large quantities of information or achieve abnormally high levels of productivity.  

If the device in question did not claim to diagnose or treat a medical condition, did not make any medical claim, and remained below established safety thresholds, it would not, so far as I can tell, be subject to FDA approval.

This is what seems attainable in the relatively near term. Some of it is well-proven; some of it is on the cusp of possible. Of course, it is possible to imagine a chip planted inside the brain that allows us to absorb books worth of information within seconds, communicate with superintelligent AI models with exceptionally high bandwidth (and perhaps in the realm of pure, high-dimensional thought rather than in language), and communicate telepathically with other humans.

Would such capabilities require invasive devices? Probably, though the concept of “invasive” may well change by the time such things are on the table. Perhaps futuristic devices will be injectable into the bloodstream, assembling themselves only after they’ve entered the body. Perhaps there will be substantial progress in optogenetics, which involves genetically engineering neurons with opsins, proteins that respond to specific wavelengths of light (by shining the light, you can then activate specific neural circuits).

As with many other frontiers of technology, there is a challenging, promising, and at least somewhat fraught path ahead. Despite mankind’s inventiveness, the brain is the most complex object human beings have yet discovered. There are interesting similarities between how frontier AI systems and brains work, but we should not get carried away with crass analogies: there is much that eludes us about the brain.

I expect, though, that the coming years will see a host of miracles become quotidian realities. I expect that the line between man and technology will continue to blur. We are at the beginning of a revolution that will take decades to unfold. Much more is possible than most people realize today. We lose our sense of wonder at our peril.