I tend to notice when they're off! I hate having to pay attention to the audio. lol
My wife used to complain when I put them on (which I thoguht amusing cos she put the telly's volume up to 35!!) during doctor who but she's shut up about it. perhaps she's finally accepting that they help me (and it's better than asking what they just said).
My family always complained when I put the CC's on at home. I didn't need them as much then as I do now so I yielded, turned the telly up and coped.