Yeah. sometimes, it does happens, and i tend to sign to them like they were a deer in headlights. 
Once in a while they tried to talk to my small kids, and i wouldnt allow that and ask them to write down to me if they want to talk to me. I get use to it because i was born Deaf. I dont complain but do my best to make them THINK what they were doing to me or anyone.
I am sorry if you feel that way.. AGree , the best thing is to let that go but if you are in a mood to humor the server, to use your creative to make the server to notice you. ha..