deafanddumb,
Not all buildings have accessible Braille signage. I'm totally deafblind and have been to a number of buildings without Braille signage or directories. In some cases where there was Braille signage, the Braille contained mistakes or was written in Grade I (uncontracted) Braille as opposed to Grade II (contracted) -- a slight annoyance of mine since Grade I takes twice as long to read. (In comparison, it's similar to something being interpreted in SEE as opposed to ASL.) Then again, if I had to pick between Grade I Braille and no Braille, I'd obviously pick Grade I. 
I can understand your frustration regarding TV captioning. While I've never been able to access TV captioning myself, I have Deaf friends who can and they report similar frustrations. Some of them also report that TV programs aired late at night do not have captions like those during the day or in prime time. 
As for the people who started yelling at you, that's just plain ignorant. I've had people do similar things like deliberately blocking the path of my guide dog and I to see how my dog reacts. I've also had people wave their hands in front of my face or ask how many fingers they're holding up. When I'm asked the latter, I say something like, "If you're not smart enough to know that, then you have more important things to worry about." Leaves them speechless every time!
