I agree that only trained service dogs should be allowed in places of public where pets are not normally permitted. It takes a dog of a very specific temperament and high level of skilled training in order to be worked as a successful service dog.
The person with the dog must meet the legal definition of "disabled" and the dog must meet the legal definition of "service dog" in order to be permitted in places of public accommodation in the US.
Yes, there are some instances of mental illness being disabling, but this is usually not the case. It is for the doctor and the patient to determine if the person is actually disabled by the mental illness and the dog must be trained to do something for the owner that the owner cannot do for themselves because of the disability. Just tagging along because the dog makes the person feel better is not considered a trained task or work as per the US Department of Justice.
While I am generally not opposed to well-socialized, well-groomed, and well-behaved dogs in public places, this is more often than not, not the case.