I don't disagree that math is a different kind of language. But as far as I know, no one has ever done a study about a sensitivity period w/math. If they have, I'd be really interested. That's not what I'm unfamiliar with...I'm unfamiliar with the specific neurological relationships between math and the brain (aside from pointing at a certain part and saying, "this is where logic and math skills are supposed to be").
Math is language, yes. But it's not a made-up one. It's a language that already exists - we're just not fluent yet.
As far as language goes...being immersed in language AFTER you know your first is one thing...but never acquiring any language is another. it would be a lot easier for me to learn German right now than Cantonese. :P some things are wired early.