I would venture a guess that the reason you feel you are doing something wrong is because that is the attitude that your parents conveyed to you in many, subtle ways. *sighs*
The oralists use your least favorite sentence all the time, but with a bit of a chnage in wording: "The majority of the population uses spoken language, therefore, the deaf must use spoken language." Different words, same meaning.
I have always been impressed with your realistic perspective on things, deafskeptic.