Those "holes" only exist because of a lot of work to level out that pyamid and a lot of bloodshed.
Are you going to pretend that Obama would have ever had a shot at the presidency without the Civil War or that Peter Thiel would have been allowed to openly inherit or build wealth as a gay man without the sexual revolution of the 60s?
Or can we recognize that without those events, one would still be being bought and sold and the other would have faced ostracism and financial disvestment, possibly even imprisonment, castration, or being beaten to death in a back alley somewhere. If Hitler had his way, it would have been concentration camps for them both. And let's not pretend like there aren't still plenty of religious types currently squawking about the joys of conversion therapy and how schoolchildren should know about how slavery taught black people trades.
White supremacy is alive and well and it is a functional derivitive of patriarchy.
Certain folks would not be getting their knickers in such a bunch if they did not deeply believe in their secret heart of hearts that positions of power and prestige should go to:
Male over female
White over colored
Straight over LGBTQ
Abled over less abled
Christian over non-christian
Strong and beautiful over weak and ugly
I'll remind you that you don't end up with class divisions and wealth disparities in the first place without some mechanism in place to grow them by stealing wealth from others in one way or another. That mechanism is supremacist thinking of valuing one type of human and their labor over another type of human. That mechanism is patriarchy. Your argument is a chicken and egg argument because the idea of patriarchy makes you feel bad.
Don't get me wrong, it should make you feel bad. No matter who you are in life, once you 'get' patriarchy you WILL feel bad. And you should. It sucks. It sucks for almost everyone living under it. Naturally, you should feel bad. If you're a liberal minded type person who thinks everyone should have equal opportunities in life then you're going to feel bad about the flip side of it too. As well you should. Perfectly natural.
It's okay to feel bad. Feeling bad inspires change. Even if all you have the power to do is change the way you think and interact with the world as an individual, that's not nothing.
It's not like anyone is blaming you for what you were born into and raised to participate in. You didn't have any say so in that any more than the rest of us did; make no mistake either, we ALL participate. None of us can change the circumstances of our births. All we can do is have courage and be kind to one another by not accepting bullshit supremacist narratives or providing cover for them with fallacy arguments.
