You can have an interpreter (ASL, etc.) with advance notice.
USA doesn't have a official language. It's not English, not Spanish, nor French (U.N. official language). I would say it is a right to not say, especially in a anything a specific language ( ironically, there's a "Code of Silence" policy at work), but it doesn't really get you anywhere if you compare "qualifications" about using English. Most businesses or managers don't know how to respond appropriately to someone who's HoH/deaf.
I thought about this awhile back. "Can I not wear my hearing aid, so as to force more written documentation". I didn't doubt so much as what I hear and process as I am able to recall arguments and pieces of information sometimes a lot better than many other people in the organization I work for. Sometimes I ask for clarification and try to stay caught up on policies and procedures. If something doesn't make sense, I ask. Basically the whole idea was to hold management more accountable for what they were saying because a week later, the direction of the policy would change. It's a very unstable state agency. I thought it (hearing impairment) could have contributed to this sort of state, but I realized more than it wasn't my hearing at all, it's just people covering their A**.