Sorry hear your experience, and I MIGHT be guilty one. I think it has to do with the sense itself. To be honest, majority of Deaf people felt left out by Hearing people in their childhood, and I think it builds their resentment against those who refused learn ASL, and is afraid that once meet and bye, whats point of trying to be friendly with them?
I am not pointing at who is at fault. Because, I can understand why Hearing people don't understand about Deafness and confused to begin with. After what I have heard, seen, thoughts and trying to understand this, and this ACTUALLY got me getting curious on what I am missing out from Hearing world, whats so special about sounds, what is it really? I am tone DEAF and never understand single wave of sound, and I believe there are plenty waves of sounds and they all are not the same to hearing people but to me, it is all same.
I believe it will take several decades before Hearing people finally figured out, and it may be just as simple as "Difference in types of the senses".
Anyway, its sad fact that majority of Deaf people have been left out by hearing people too often to the point its easy for Deaf to stereotyping another who is either losing their hearing or try to learn sign language from them.
My little word of advice to you, read under my signature of one of known child with autism, his quote (That is J-MAC) and I believe he is right on this one. If you still don't know who is J-MAC, google "J-MAC" and it will explain you everything. So, don't give up.