Veteran pro-Europe Tory Ken Clarke has issued a warning to David Cameron not to pander to "ignorance and bigotry" in an attempt to counter the electoral threat of Ukip.
The former cabinet minister also said some of his Tory MP colleagues belonged in Ukip, rather than in the Conservative Party.
Clarke accused "noisy right-wing" Conservatives of trying to drive the Prime Minister into taking a position on immigration and the free movement of labour which would make it impossible for Britain to remain in the European Union.
With the Tories fighting to prevent the loss of a second seat to Ukip in the Rochester and Strood by-election - where Conservative defector Mark Reckless is standing for Nigel Farage's party - Clarke said there appeared to be "a wave of almost hysteria" about immigration.
"All the excitement is caused by right-wing people who want the Prime Minister to make a speech that makes it impossible to negotiate our continued membership," Clarke told the BBC Radio 4 Today programme.
"Well, ignore the daft ambitions of people whose main interest is just getting out of Europe. Let's look after Britain in the modern world."
Downing Street has confirmed that Cameron intends to go ahead with a keynote speech setting out proposals at curbing immigration to Britain from other EU member states.
Clarke said that, while controls were necessary, it was important not to impose measures that would simply damage Britain's ability to compete in the global economy.
"We have to demonstrate that we are a strong, sensible party that does sensible things on immigration because there isn't a politician that isn't in favour of controlling immigration," he said.
"But what we mustn't do is start competing with ignorance and bigotry and what we mustn't do is start doing damage to our economy by imposing restrictions on people we need to come here.
"There is a section of the population that used to support the BNP and so on that is just bigoted and anti-foreigners and there are popular politicians who just arouse prejudice.
"But most people - particularly young people - understand that we are in a global economy. You get used to people who don't speak your language and you meet people at work who are foreign."
Clarke said there were some right-wing Tories like Reckless and fellow defector Douglas Carswell whose views really belonged in Ukip rather than the Conservative Party.
"I have always thought that some of them should obviously be in Ukip rather than our party," he said. "Two of them have gone, neither of those surprised me very much. Ever since I first met them I thought their opinions rather more resembled Ukip's than any I was familiar with either from the Thatcher government or the present one."
Earlier, the outgoing head of the European Commission claimed that Margaret Thatcher would have rejected Ukip's Eurosceptic arguments as he accused Conservative MPs of "surrendering" to Farage's party.
Jose Manuel Barroso, who will stand down as commission president next month after 10 years, told the Daily Telegraph he believed the late Baroness Thatcher was a supporter of open labour markets and enlargement of the European Union rather than "protectionism".
"I am not surprised at the protectionism in some other countries, but in Britain I am surprised. Your tradition is exactly the opposite," he said.
"As a friend of Britain looking from the outside, I remember well Margaret Thatcher. I worked with her when I was young, as deputy foreign minister of Portugal. The Conservative sentiment was a sentiment of openness. So I am surprised when I see so many Conservative politicians surrendering to the arguments of Ukip."