Gordon Brown was wrong to brand pensioner Gillian Duffy a bigot because of her concerns over immigration, Ed Miliband said.
The Labour leader said the ex-prime minister should not have criticised the Rochdale woman's comments during the 2010 general election campaign.
The fiasco was a defining moment of the run-up to the poll, with Brown recorded making unguarded remarks about Mrs Duffy after she confronted him about eastern European workers.
"I don't believe our immigration system has been working for working people over the last few years. Worrying about immigration, talking about immigration, thinking about immigration, does not make people bigots," the Labour leader wrote in an article for the Daily Mirror.
Miliband also said he would force job agencies to give British workers "a fair crack of the whip" under a Labour government, and called for tougher fines for businesses which knowingly employ illegal immigrants.
And Miliband, whose parents were immigrants to Britain, demanded tighter rules to stop foreign workers being exploited.
"We also need to make sure that everyone gets a fair crack of the whip when it comes to job opportunities," he wrote.
"In some industries the majority of workers are hired on a temporary basis via recruitment agencies.
"In the meat processing sector, up to a third of workers are hired this way. Because they are hired on a temporary basis, these workers often have fewer rights.
"And because sometimes these agencies only advertise for new recruits abroad, or find other ways of just recruiting foreign workers, local people often feel locked out of the recruitment process altogether. That cannot be the right direction for our country."
He added: "We already have laws to stop workers being discriminated against.
"Most people will think that if vacancies are advertised abroad, they should be advertised locally.
"If they are only advertised abroad, that's just not fair.
"The Government should look at how the law can be better enforced to prevent this."