I was lactose intolerant without even really knowing it. I just found that I was eating better when I switched to a vegan diet (which is naturally lactose free).
I find vegan icecream much tastier then ice cream made with cow pus. I used to keep having sickly after taste whenever I ate icecream.
There are plenty of dairy substitutes so there is no problem about going dairy free. It's much better for you too.