When in a hearing group.. yes.. I often find myself left out.. so I usually walk away and mind my own business... I would only stay in group if and when I have a friend willing to inform me of whats going on... and able to actually answer my questions without going "Oh it's not important" (I find that to be rude!)
I think being in a hearing social group takes a lot more effort on us. So I think we just got to pat ourselves on our backs and be proud of our own effort to try and follow, also a high five for the friend.
