You would have a point if the men who find women too ugly to have sex with weren't, in reality, still trying g to have sex with those women.
But they are. It never ends. Let's face it, even a true coyote ugly becomes a bragging right of sexual exploit amongst men.
So it's not really a double standard, is it?