I myself am learning ASL and I can't hear a thing. I do speak. I went to a Deaf Coffee this past Friday night, and for me, it was very hard, but I did pick up some things.
I go to a Deaf church, and I am usually very lost. I pick up a few things here and there, but the signing is so fast that I am lost.
I know I am not much help, but I also know that the more events you can get to, the more it will help.