People recognize words in conversation primarily from the sounds they hear. However, scientists have long known that what humans perceive goes beyond sound. The brain actually constructs its own unique interpretation, factoring in both the sights and sounds of speech.
For example, when combining the acoustic patterns of speech with the visual images of the speaker's mouth moving, people sometimes reconstruct a syllable that is not physically present in either sight or sound. Although this illusion suggests spoken syllables are represented in the brain in a way that is more abstract than the physical patterns of speech, scientists haven't understood how the brain generates abstractions of this sort.
A new study at the University of Chicago, in Chicago, IL, identifies the brain areas responsible for this perception (Neuron, Dec. 20, 2007).
"When the speech sounds do not correspond exactly to the words that are mouthed, the brain often conjures a third sound as an experience," explained lead author Uri Hasson, PhD. "This experience may often vary from what was actually spoken."
For example, if a person says "pa" but mouths the word "ka," you would think you'd hear "pa" because that is what was said. However, due to the conflicting verbal and visual signals, "the brain is far more likely to hear 'ta,' an entirely new sound," he stated.
This is called the McGurk effect, named after Harry McGurk, a developmental psychologist who first noticed the phenomenon.
The scientists at the University of Chicago used functional magnetic resonance imaging (fMRI) to demonstrate that Broca's region is responsible for the type of abstract speech processing that underlies this effect.
Although people experience speech as a series of words like print on a page, the speech signal is not as clear as print and must be interpreted rather than simply recognized, Dr. Hasson said.
The study provides a glimpse into how such interpretations are carried out in the brain, he noted. These types of interpretations might be particularly important when the speech sounds are unclear, such as when conversing in a crowded bar, listening to an unfamiliar accent, or coping with hearing loss.
"In all of these cases, understanding what is said requires interpreting the physical speech signal to determine what is said," Dr. Hanson reported, "and scientists now know Broca's region plays a major role in this process."
The National Institute of Mental Health supported this research.