The McGurk Effect

What you see really is what you get.

Or at least that is what The McGurk Effect says.  The effect was first published by (you guessed it) McGurk (and MacDonald, who presumably feels a little bitter about the whole naming thing) in 1976 in the prestigious journal, Nature.  It was one of those quirkily named articles that you know the authors spent way too much time thinking of: Hearing Lips and Seeing Voices.  

The basic idea is that seeing someone talk helps you understand what they are saying.  When you can't see someones lips, it makes comprehending their speech more difficult.  I personally know this is true because if I am having a conversation in a loud place (like the local public house), I rely heavily on the speaker's visual cues to understand what they are saying.  The McGurk Effect takes this notion to the extreme -- hearing "Sound A" when a person aurally says "Sound B", but their mouth makes the motions of "Sound A."  So, like I said: What you see is what you get.  You can fool your own brain with the video of this on YouTube.

I learned about this in the context of the show Scientific American Frontiers with Alan Alda on PBS (a quite wonderful show, I might add, that I always find entertaining and educational whenever I manage to see it).  It was an episode entitled Cars That Think, and The McGurk Effect was mentioned in the context of a voice recognition feature for a car that was being deveoped by IBM.   It turns out that voice recognition still sort of sucks -- you already know this if you have ever been on the phone with one of those bloody automated phone systems that thinks it can understand you.  The show quoted something like 80% accuracy (a bit generous, methinks) for standard voice recognition.  But if you add in the capability of analyzing lip movement, they claimed that the accuracy became essentially 100%.

I enjoy these sort of tricks your mind plays on you as a result of the way your brain typically processes information in hopes of helping you.  The most common example that comes to mind are optical illusions.  But I feel like The McGurk Effect is a more extraordinary example because your senses actually conflict with one another and force your brain to choose an interpretation -- a sort of battle of the senses.  It is interesting that in interpretting the spoken word, something most closely associated with hearing, that your brain chooses to believe the visual information.