If, he spends his quality time with " her " than be with you, then it is not worth it to worry about him or have him back. If, I were in your shoes, I wouldn't want him back. I will find someone else who is better than him and of course, I will learn that lesson by finding someone else who is NOT going to be like him.
Find a " hot " fish who is more interested in being with you rather than a " cold " fish who is NOT interested in being with you. 
There's plenty of fishes out there for ya. 
