The McGurk Effect

(n.) a perceptual illusion caused by the confusion or misinterpretation of combined visual and auditory information

We occasionally make a foray onto YouTube here at HH, and recently we posted a video about one of the most bizarre linguistic phenomena discovered in recent decades. Goggles and lab coats at the ready, people, we’re getting experimental.

The McGurk Effect was discovered in 1976 by the British psychologist and linguist Harry McGurk. An expert in child language acquisition, McGurk reportedly discovered his “effect” entirely by accident when, during preparation of a separate language experiment, he happened to replay the audio of one phoneme (language sound) over video of another. And the result was—well, something very unusual indeed.

(FYI, if you haven’t watched the video yet, now would be a good time to do so. Otherwise, SPOILER ALERT!)

As explained in the video, as much as we might think of speech perception as being a purely auditory, sound-based process, the McGurk Effect neatly proves that there is in fact just as much (if not more) visual information being analysed—it’s just that it all happens so quickly and automatically, that we’re unaware of it happening.

But if the information being provided by our eyes and our ears don’t match, then our brain doesn’t quite know what to do. So watching a video of someone saying far while hearing audio of them saying bar leads to some considerable confusion.

When this happens, in some cases the brain ignores the auditory information entirely and instead trusts the information being provided by the eyes without question—if that’s you, then watching the tape in the YouTube video, you’ll be convinced that the fourth word was far.

In other cases, however, the brain mixes the two conflicting streams of information together, thereby convincing itself that what it’s seeing and hearing is, in fact, neither of the things that it’s actually seeing or hearing—and if that’s you, then like our guinea pig Anthony in the video, you probably thought the fourth worth on the tape was var rather than far. Likewise, in McGurk’s original experiment, he found that playing the sound ba-ba over a tape of someone saying ga-ga led to him interpreting it as da-da.

So what does all this prove? Well, the McGurk Effect demonstrates just how much visual information is utilised by our brains while we’re processing speech, and how quickly our brains are to ignore auditory information—which we might think of as the cornerstone of speech perception—in favour of visual information. This has all kinds of implications on how we acquire language in the first place, what kind of sensory hierarchy must be going on in our brains, and what our brains have to do when this process is disrupted by, for instance, blindness, deafness, or trauma to the language-processing part of the brain. And the more we understand about that, the better we’ll become at fixing it when it goes wrong.

For more information on the McGurk Effect, you can take a look at McGurk’s original 1976 paper, Hearing Lips and Seeing Voices—or take a look at another version of this experiment on YouTube, courtesy of the BBC’s Horizon.

#YouTube #science