There is a major life change. People often grieve over any change in their lives, even if it turns out to be a blessing.
The choice for spoken language is a very personal one. Many parents want to be able to communicate fully with their child and they believe that communication will be compromised if they use a language that they are just starting to learn. How are they supposed to be a fluent language model if they know no sign? How do they explain everything there is to know in the world, using a language they have a very beginner grasp on. They don't know how to read book, share stories, and explain religon in a language that had never seen before. They know English, they can teach it, use it and model it. They want their child to have access to a full language model, and full communication, not tiny bits of baby sign with no grammar or fluency. They believe that they can give the best language model, fluency, and communication through spoken language.
Perhaps they are mistaken in believe that, but I believe that is the reason the vast majority choose spoken language.