Shel, at any time are they planning on getting their own place, or are they with you for life? Or..... maybe, are you planning to ask them to think about getting their own place sometime soon? Sounds like you would prefer to have just your own family now.
Was this arrangement some kind of helping out while they get on their feet? I think that could go on forever if allowed. I'm sure it was nice with just your family, and you are not really complaining about your room mates. I'm putting myself in your position, and would never be able to live with another family in my house, that would drive me nuts, don't mind me.