More than half of Britons believed their culture was threatened by ethnic minorities living in the UK, a report said.
A quarter felt migrants took jobs away and a third thought they removed more from society than they contributed, the Aurora Humanitarian Index public opinion survey for this year said.
After Brexit the UK may be less capable of addressing the refugee crisis, part of an overwhelming lack of confidence in world leaders to address the issue, the survey found.
Only a minority felt Prime Minister Theresa May was best positioned to resolve it. Support for humanitarian action was also in steep decline.
The report said: "This year's findings demonstrate an overall decline in the support for humanitarian action based on scepticism in the ability to make an impact and ambivalence in defending social values over self-interest."
The Conservative General Election manifesto has said the current figure of 273,000 people heading to Britain was "still too high" and promised to slash the number to "sustainable levels". No timetable has been set.
The 2017 Aurora Humanitarian Index surveyed nearly 6,500 people in 12 countries.
Its UK findings included:
:: 56% of people felt local culture was threatened by ethnic minorities in the UK;
:: 24% felt migrants took jobs away but half believed their impact was neutral;
:: 34% thought minorities took more from society than they contributed;
:: After Brexit the UK is expected to be less capable of addressing the refugee crisis.
:: Prime Minister Theresa May was named by only 15% as best positioned to solve the challenge surrounding people displaced by war and conflict - the same proportion as those favouring Syrian president Bashar Assad.
:: The research was carried out before the Manchester bombing but terrorism was regarded as the most pressing global humanitarian challenge by two thirds of those surveyed in Britain.
:: Forced migration and hunger were also seen as important issues.
World experts will gather in Armenia this weekend as part of the Aurora Humanitarian Initiative to mark the contribution of those who help in the direst of circumstances.
Co-founder Ruben Vardanyan said: "The sense of apathy towards humanitarianism today highlights an urgent need for engagement in every sector.
"However, this negativity is counter-balanced by the incredibly positive attitudes of youth towards humanitarianism and the individual impact on the refugee crisis.
"All of us need to educate and motivate the young people around the world so they not only understand their capacity for meaningful impact, but are inspired to act upon it."