I can communicate with hearing people but I find myself feeling much more isolated in a group of hearing people chattering away using spoken language than I am with a group of people using ASL. As long as I am not isolating myself from another human being, I am not cut-off nor I feel that is a form of isolation.
The way you put it gave me the impression that ASL users are not simply good enough and that deaf people must interact with non-signers. Some simply dont want to and I dont see anything wrong with that if it makes them happy.