Those men will always equate you making the best choice for you as a rejection of them, regardless of whether or not it has anything to with them or not.