I think it is more complicated than just a matter of being correct or not. Common advice in some creative professions is "don't bother trying to do X for a living, you'll never make it." The point of the advice is not whether or not it is literally correct and the person is better off not bothering - in the general case, it's probably true. The point is that it acts as a filter for those not motivated enough. It's a backhanded sort of test implicitly presented to the aspirant.

Someone who really, really wants to make a billion dollar startup against all odds is going to ignore your advice anyway. In fact, they would ignore any AI's advice on the topic as well. But that kind of arrogance is precisely what's required to be able to pull it off. Someone who quits the moment an AI tells them "don't do it" was not cut out to accomplish such a goal to begin with.

And maybe in the end the startup will only be worth a couple million dollars, but the hubris to believe they could go even further would be what got them that far at all. So "can build a billion dollar startup" ended up being false, but something else was gained in the end.

We get into a bit of a weird space though when they know your opinions about them. I'm sure there are quite a few people who can only build a billion dollar startup if someone emotionally supports them in that endeavor. I'm sure more people could build such a startup if those around them provide knowledge or financial support. In the limit, pretty much anyone can build a billion dollar startup if handed a billion dollars. Are these people capable or not capable of building a building a billion dollar startup.

EDIT: To be clear, I somehow doubt an LLM would be able to provide the level of support needed in most scenarios. However, you and others around the potential founder might make the difference. Since your assessment of the person likely influences the level of support you provide to them, your assessment can affect the chances of whether or not they successfully build a billion dollar startup.