Sensory Substitution: The Brain's Ability to Adapt to Loss

The adult brain weighs about three pounds and looks fairly unassuming. The organ’s seemingly uniform appearance does little to betray its underlying complexity. Over a period of several years, countless neurons are born into the daunting task of finding their designated locations and making connections with other neurons. Somehow, the specificity of this developmental process manages to consolidate these brain cells into an organ that is responsible for all of human consciousness and skill.

However, the rate at which this development occurs is variable, and the majority of neurodevelopment occurs early in life. A child’s brain is in a state of rapid change and development, with new connections forming as cognitive abilities mature. This era is often called the “critical period,” and has for years been a research area for scientists who want to understand the plasticity of the brain.

Some of the first evidence of a critical period of development comes from experimentation by neuroscientists Hubel and Wiesel. Together, they completed a study in kittens in which they blocked one eye from receiving light from birth. This process, called monocular deprivation, resulted in animals that could not process visual information from the blocked eye. These cats were not blind in the traditional way, since their retinas were intact and functional; instead, it was the visual cortex that was abnormal. Hubel and Wiesel concluded that in order for the visual cortex to fully develop, visual sensory information is required during the critical period of development [1].

In 1897, G.M. Stratton examined the visual cortex’s ability to adapt to abnormal visual sensory information outside of this critical period. The adult neuroscientist wore a set of mirror glasses that inverted all visual information, much like the lens in the eye already does. After a period of a few days, his visual sense adapted to the new sensory information and he began to function normally. His movements were generally coordinated with his modified visual environment, and it was apparent that his brain had changed in order to interpret the upside-down world [2].

But just how plastic is the brain both during and after the critical period? Amazing examples of adaptation occur in the central nervous system after bodily trauma [3].

Cortical Reorganization

While it may not be obvious simply by inspection, the structures of the brain and the connections between them each serve a specific function and are fairly consistent between individuals. For example, the area of the brain that controls movement lies in the motor cortex, a structure just in front of the central sulcus, the dividing line between the frontal and parietal lobes. The portion of the brain that receives sensory information relating to touch, the somatosensory cortex, exists just behind the central sulcus. These regions are well defined in healthy individuals, and each function owns the cortical “real estate” that it requires. However, this “real estate” is in high demand.

In unusual cases, such as when a sensory ability is lost, regions of the brain corresponding to remaining abilities can overtake structures that can no longer perform their designated function. This phenomenon, called cortical reorganization, is reminiscent of the astounding plasticity observed in the nervous system during early development.

One of the most accessible and intriguing examples comes from the medical practice of amputation. After amputation of a limb, the somatosensory cortex, the area responsible for receiving and localizing haptic (touch) information, can reorganize. The somatosensory cortex is organized somatotopically, meaning that areas of the body are mapped to stereotyped locations in this cortical region. After amputation, it is common for the region that was once dedicated to the removed limb to be reorganized such that it becomes responsible for neighboring, intact regions instead. The nervous system can actually alter its connectivity so that the regions of the cortex that would otherwise become useless obtain other functions [3].

Cross-Modal Plasticity

Amputation provides an example of reorganization that can occur between two regions of the same cortical structure, the somatosensory cortex. However, the next step is to ask whether this type of plasticity can occur between separate regions of the brain that originally had different functions. If a particular sense is lost, can other senses be enhanced in order to compensate? If so, is this due to cortical reorganization between functional regions, or simply increased connectivity within a single region?

To answer this question, we look to J.P. Rauschecker’s research. Similar to Hubel and Wiesel, he subjected young mice and cats to binocular deprivation, and observed how their other remaining sensory abilities compared to those of control animals. To test this, he measured the animals’ ability to localize sound, and saw a marked improvement in auditory proficiency of those that had lacked visual stimuli. This behavioral result was mechanistically reinforced by observations of an anatomical expansion in the auditory and somatosensory cortices in these animals. Without visual information early in their development, these animals never developed the neural circuitry necessary to see; this allowed for an enhancement of their other senses due to cortical plasticity [4].

A study produced by L.G. Cohen also found that early blindness in humans can lead to the recruitment of the visual cortex in the processing of somatosensory information. To determine this, Cohen subjected individuals to transient transcranial magnetic stimulation of the visual cortex and other cortical areas to disrupt their functions. Concurrently, he asked subjects to complete tasks that assessed tactile perception. Magnetic stimulation of the visual cortex produced significantly more errors in the tactile processing of blind individuals when compared to control trials without stimulation, whereas stimulating the visual cortex did not significantly affect the sighted individuals’ performance on the tactile task. This suggests that these blind subjects were using the visual cortex to process tactile information [6].

It seems that individuals that lose their vision may have their other senses heightened due to cortical reorganization, especially if that loss occurs early in life [5]. The utilization of visual cortex by other sensory modalities indicates that reorganization is indeed possible in the human brain even past the critical period of development, and opens the gate to new research endeavors in fields such as sensory substitution.

Sensory Substitution

If the nervous system has the ability to change its connections in response to a sensory loss, could it be possible to take advantage of this reorganization? Instead of a mild increase in other sensory abilities, could this natural reorganization be further exploited to completely compensate for a lost sense? To answer this question, we turn to the groundbreaking work of Paul Bach-y-Rita.

In his publications on the topic of sensory substitution, Bach-y-Rita has stated that this phenomenon may be more commonplace than it sounds. When blind individuals use a walking stick for assistance, they are essentially substituting tactile sensation for vision, and the walking stick is serving as a primitive human-machine interface (HMI). In fact, Braille is also recognized as a sensory substitution system in the same way. While these systems are informative to the user, the individual’s vision is not entirely substituted. Other methods, however, have come much closer to larger-scale substitutions [7].

Researchers have attempted to use the auditory system as an alternative to the visual system in a complex example of auditory-visual sensory substitution. Capelle et al. utilized a predetermined pixel-frequency relationship to transduce information from the retina onto the cochlea [7][8]. Each pixel that would normally be seen on the retina is converted to a pitch heard in the ear. This system is used in conjunction with a head mounted camera, so visual patterns can be converted into auditory information. As far-fetched as this may sound, both blind and sighted individuals trained on this device have been able to identify visual patterns [9].

Another promising form of sensory substitution utilizes an unlikely location—the tongue. Researchers have found that the tongue is an ideal transducer (or HMI) for tactile-vision sensory substitution due to its high sensitivity. The sensory receptors on the tongue happen to be very close to the surface and easy to electrically stimulate. This has led to the development of the Tongue Display Unit (TDU), a device that can convert optical images from a camera into a map of electrical stimulation on the tongue. While it may seem that this stimulation would probably just feel like something is touching the tongue, users report “seeing” the stimulated shapes in a spatial domain. It seems that these individuals have adapted to recognize the stimulation from the TDU as not just simple tactile feedback, but something else entirely. Similar to G. M. Stratton’s experiment, the TDU seems to have tapped into the astounding ability of the human brain to adapt to and interpret a new type of sensory information [7].

In a more recent push to further research in this field, neuroscientist David Eagleman is currently attempting to use crowd funding to develop a wearable vest for sensory substitution or augmentation. This vest is intended to be worn under normal clothing, and to have the capability to transduce auditory information (or other data) into haptic sensation delivered to the torso. This idea is unprecedented, as Dr. Eagleman is intending to use this vest to add sensory modalities that are not normally available to humans. He hypothesizes that with normal training, individuals using this vest will learn to “feel” hidden sensory information delivered through the vest. This augmentation could provide the user the ability to “feel” stimuli such as ultraviolet light or even the measures represented in gauges in the cockpit of an airplane. With successful sensory addition, a device such as this could allow the wearer to analyze and interpret sensory information that would not be available to the otherwise.

Conclusion

The HMIs used for sensory substitution have not only given their users an increased ability to sense the world around them, but also provided insight into the ability of the nervous system to adapt. This plasticity continues to be the topic of intense study, as researchers aim to unlock the ability to manipulate cortical reorganization in useful and interesting ways.

References

1. Hubel, D. H., & Wiesel, T. N. (1964). Effects of monocular deprivation in kittens. Naunyn-Schmiedeberg's Archives of Pharmacology, 248(6), 492-497.
2. Stratton, George M. "Vision without inversion of the retinal image." Psychological review 4.4 (1897): 341.
3. Chen, R., Cohen, L. G., & Hallett, M. (2002). Nervous system reorganization following injury. Neuroscience, 111(4), 761-773.
4. Rauschecker, Josef P., and Ulla Kniepert. "Auditory localization behaviour in visually deprived cats." European Journal of Neuroscience 6.1 (1994): 149-160.
5. Büchel, C., Price, C., Frackowiak, R. S., & Friston, K. (1998). Different activation patterns in the visual cortex of late and congenitally blind subjects. Brain, 121(3), 409-419.
6. Cohen, L. G., Celnik, P., Pascual-Leone, A., Corwell, B., Faiz, L., Dambrosia, J., ... & Hallett, M. (1997). Functional relevance of cross-modal plasticity in blind humans. Nature, 389(6647), 180-183.
7. Bach-y-Rita, P., & Kercel, S. W. (2003). Sensory substitution and the human–machine interface. Trends in cognitive sciences, 7(12), 541-546.
8. Capelle, Christian, et al. "A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution." Biomedical Engineering, IEEE Transactions on 45.10 (1998): 1279-1293.
9. Arno, Patricia, et al. "Auditory substitution of vision: pattern recognition by the blind." Applied Cognitive Psychology 15.5 (2001): 509-519.