You know your story is incredibly similar to the reason why I decided to learn Sign.
It didn't seem to matter what job I was working at I was regularly running into Deaf people. Although I didn't mind using pen and paper to communicate, I got to thinking it'd just be a whole lot easier to communicate if I learned ASL.
Don't be afraid to find a class and give it a shot. I find that 99% of the Deaf people who you'll sign with are very pleased you are making the effort. I've only run into 1 person in 15 years of signing who was nasty and they were probably (technically) HOH.
She was speaking with the "deaf accent" and was clearly not understanding what I was saying. When I tried signing with her she said "I DON'T sign!" (as if that was beneath her) and stomped away.
All my other attempts have been very well received.