Pardon me, is it legal for Neo-Nazi supporters who are mostly skinheads to fly the Nazi flag in public? I haven't seen one flying in the USA. Have you? If no, why the hell can't they fly it? The public doesn't allow it.
In other words, people speak out. If there are more votes against this flag, there's nothing you can do except obeying a new law.
My deaf black ex co-worker told me that many black people won't forget about slavery in the South. When they see this flag, they look at it as the South still doesn't respect their freedom.