I've been in this scenario before. People see me signing to my deaf friends and I overhear them talking about us. I've never understood why people can be so offended by such a beautiful language. I have noticed, though, that people are much more open to Deaf culture here in St. Augustine than in other places.
I get really angry at my Hearing classmates because they always ask me why I sign to my Deaf classmates instead of using the interpreters. Yeah, I'm glad the interpreters are there because there are a lot of things I don't yet know how to sign and they're a helpful tool in learning. I use the interpreters in a pinch, if they're standing right there but usually I'll fingerspell something rather than go find an interpreter. I'm just more comfortable communicating with someone directly rather than through another person.
Some of my classmates have gone so far as to say "Well, if they're going to go to school with Hearing people then they should learn to lipread". It makes me angry when people say things like that because from what my Deaf friends have told me, lipreading is hard to learn how to do.
Sorry for the rant but I just see things like this happening so often and it makes me mad.