Nope, that is not it. It is really simple. My entire world is hearing (my wife & kids plus extended family plus social circle plus church...you get the idea). I don't have the time nor energy to spend time learning sign. I have never needed it and why start now. Also, who do I use it with? Learning sign is like learning a new language. I believe languages are use it or lose it affairs. The older you get the harder it gets. As for being uncomfortable, that is not a problem as I have been around the deaf many times over the years. I'm cool with it...no big deal. Most of the deaf I had dealt with were 'tweeners too and we all got along fine (not needing sign to communicate).
Not all deaf need sign and it should not be something held against them. It is one thing when one deals with the Deaf community that requires sign. Under those circumstances, I be more than happy to try sign (whether I be good at it or not...a whole different story).
My beef with that fateful day was the total rejection of me without even trying to understand why I didn't know sign or why I fitted into the hearing world. There wasn't any attempt to communicate...just judgement from what they thought. I can say that it is much better today like those in my church who are deaf and sign. I have been to one of their socials and it was very interesting to me watching all the hands flying around (I could grasp some of what they were saying but I'm lost otherwise...
) Some would lipread very well and I had good conversations with them. It was great but it is not my world...I have been in the hearing world for too long. I just haven't felt the need to really get into the deaf community. Who knows, I may decide to learn it one day but not right now.