I'm 99% certain it would completely suck with children. It could be cool on your own, or maybe even as a couple, but with kids I can't see it. Not to mention the social stigma the kids would face at school, especially in the best district.
I'm not so certain, if only because how many obstacles and problems people imagined for me when I told them the idea, compared with how surprisingly easy it was when I actually went for it.
I think mindset is still the biggest factor.
For example, there are all sorts of things you can get picked on for at school, and I think the most important bit is teaching them to navigate the process itself.
Going to what is almost certainly a rich kid area while your child lives out of a minivan that gets towed around every three days is 100% going to make them a social pariah and miserable at school. Maybe when they're 16 and can drive the car away on their own feelings could change.