I never analyzed it until you mentioned the distraction factor, Miss D, but I know I have to take a step or two back when there's some anomaly like you mentioned -- seriously missing teeth, extensive braces, gaudy jewelry pierced through the lips or tongue.
If I don't step back and speech read more than lip read (you silent observers know what I mean) I get too focused on the object and lose track of clues.
That's a definite barrier.
