Both. The answer is learning to be more observant, that's all. Both of everyone around you and of what your friends do and say.
It's not an absolute; if it was a lot fewer women would fall prey to rape.
We've all been conditined to ignore things, to put politeness or being a peacemaker above taking care ofnourselves an others.
You've got to break that conditioning and retrain those thoughts patterns into appropriate threat assessments. Simple, but not easy.
Pick one of your friends at random. Hold them in your mind. Take a long hard look. Recall your interactions. Recall what you noticed about how he/she interacted with others. Take some time and really see.
Now ask yourself some questions.
Does your friend understand consent?
Does your friend respect boundaries?
Does your friend prefer to chase after sex or does your feel value relationships (not necessarily romantic ones)?
Does your friend drink a lot? Does your friend encourage women he is with (if male) to overdrink when he is with them?
Has your friend ever said anything that could be framed as complaining about all the thigs he has to do to get sex? Or how about something like how much he enjoys 'the dance" or talking women into giving in?
How does he talk about sex in general?
How do you feel about him being alone around your daughter or wife? Does that give you a good feeling? Or an icky, wary one?
Does he make a lot of rape jokes? Does he say anything indicating a belief that rape is earned by certain women?
And so forth and so on.
These aren't proof of rape, of course.
What they are is indicators that a man is ok with rape culture and so therefore would be more prone toward sexual abuse, misogyny, sexual harassment, or rape.
Think of it like a slide rule. The more boxes that get checked, the more likely he's crossed thenline in one way or another.
Sexual abuse doesn't exist in a vacuum. It's a pattern of behavior. People tell you who they are and what they believe over time. You just have to listen.