They fear it because they cannot imagine a world without hierarchy. Someone must be in charge, in their thinking, or nothing will get done. Therefore, if it's not men, by default it must be women.

They know the wouldn't enjoy being subordinate, second class, disposable, etc any more than we do.

They cannot fathom a fluid rather than a rigid hierarchy, or what Elle Beau has described in one of her articles as a hierarchy of actualization.

And that is why they fear letting Patriarchy go. No matter how badly it hurts them, in all the ways, at least there's certainty and at least they're not women.