You still need space for them to settle. And you need an economy so they can thrive. Brexit kind of nixed that, didn't it?
Prividibg affordable housing for them is doable too, but then all the white people who have been denied that will scream too. You've got a significant wealth equality problem too.