its very common for hearing people to not look at each other whent hey are talking, as its not required for communication.
on the flip side, one thing that really bothers me as a hearing person, is when im at work, or around my deaf friends, watching tv or something, if anyone moves thier...