The best tip I ever got was not to try and read the fingerspelling by letter, but to think of it phonetically. So if someone fingerspells W-A-S-H-I-N-G-T-O-N, don't try to remember all the letters because you'll have forgotten "W-A" by the time you get to "O-N." Think:
"wwwaa...sshhh...inngg...tonnn"
if that makes any sense. In other words, make every letter a sound. When I read fingerspelling I know my lips move because I'm subvocalizing, but it makes it MUCH easier to understand.
Also I agree with the others who have said that this is a common problem. It's one of the hardest things about learning ASL. There are videotapes and DVDs out there specifically for practicing receptive fingerspelling, if you don't get enough practice socializing with ASL users.