There's not likely to be much to go back to for most of the people who've left.
If it was the former case, then approximately 7b people would be valid refugees.