That has happened to me before. They handed my card to the other person. I just say, "hey that's my card" and take it from them. Depending on their attitude, if they gave me less than stellar service, I will give them no tip or a low tip, and then write down on the back of the receipt that I wasn't happy with how I was treated.
If it was great service with a great waiter with no problems, I will write down a cheerful note on the back saying thanks for a great service. 
