Then they're effectively "leaving money on the table" by avoiding female founders, because they're being too risk-averse. The risk of false accusations that will blow up are too small to worry about.
If they're not outright bad investors, they are at least investing a suboptimal way.
I wonder if male investors are actually acting rationally leaving women out of their portfolios. I mean, we all know how a pretty face can scramble the thought processes of a male, and there actually are scientifically verified cognitive biases [1] that work against rational thinking in a multi-gender environment.
Not saying, of course, that this justifies any kind of harassment, just that maybe some of these investors have learned in the hard way that their judgment gets clouded when dealing with women.
But the Halo Effect page that you linked says that males were influenced by the halo effect when rating both male and female subjects so this isn't a strong argument for not hiring/investing in women.
And in a MythBusters I watched the other night, they "Busted" the myth that men get dumber around an attractive female. I'm pretty sure it was from the newest season if you wanted to look it up.