It doesn't. First of all, tall is relative, a preference is not a requirement or a standard. For the most part, women do prefer men who are taller than them, at least a little. My feelings are that this is less because women care about and more because men do. Lot of short guys have little big man syndrome. Their feelings about being of smaller statute kind of overtake everything, they're not fun to be around. They don't date well many times.
You can take that for what it's worth. Lot of girls had crushes on shorter boys as adolescents knowing that more than likely you were gonna shoot padt us soon enough; remember we got growth spurts faster on average than boys. We learned our lesson and when it came to date for permanence, we remembered those lessons.
Lastly, the number of men as short as you suggest is statistically insignificant. It doesn't even register below 5'3". And at that height, there's a footnote that the data does not meet the standard for reliability or precision. Comparatively, there's significantly more women 5'3" and below. Even if all women everywhere absolutely refused to date any man shorter than her, he's still going to have plenty of women in his dating pool. Far far more than if it were the other way around.
Height is not the barrier men imagine it to be, other than what they themselves create in their own minds.
https://www2.census.gov/library/publications/2010/compendia/statab/130ed/tables/11s0205.pdf