I am in a similar situation, well different as im not in the U.S. , but in a small sort of settings as a small city university , all hearing and very similar mixture of students as you decribed. And yeah going to Gally would be great, but
ASL is NOT not my first sign, let alone I have very rusty NZSL on top of that
but yeah I can see how you wish to be fully participating in group dicussions and so on, as being a student. Absolutely, because like I feel lonely in this university that I go to, tinnitus on top of the physical deafness I have, has made me 10 times more shy because I can't handle the stress and the blare of the noise (in the head) when being right there where (eg, coffee shop or concourse) I could possibly get to know some hearings, and on other side, they seem to freak out dunno why. In NZ there somewhat a real confused mix of how hearing people percieve deaf people now thanks to sign language politics and SOMEtimes i suspect the way interpreters broadcast (show off) their elitism as 'professionals' like almost as if terps are medical professionals !! , i know this might come across wierd but that's my little rant here , < slight off the topic sorry.
now back on track,
But also I can see the downside, such as immaturity, intolerance, tall poppy sydrome, even elitism (from deafism) etc you might face there. It seem to boil down to your heart's wish, you want a great education and have a boost in ASL.
In your situation , as it seem youd have far more chance than I to get there in Gally, and hell, why not? it is a place to be YOU, and its a the silliness of hardliners is outwieghted by your own wishes and im sure the majority are cool. Go for it man