Great thread you created there FlyFree 
My ex husband does the same thing, everytime he talks to me, he would be looking down or somewhere else, I have asked him serveral times why he wouldn't look at me when he talks to me, he said it just a habit of doing so, and I noticed he does the same thing to his family and his friends, good thing it wasn't because my deafness, it was more of how he is when he talks to others....so I believe that why others do it as well too, they just too busy to look up to the customers or they are just plain bored who knows....LOL