Counting is under way in the Oldham West and Royton by-election, with Jeremy Corbyn facing the first test of his Labour leadership at the ballot box.
Labour appeared on course to retain the seat, although Ukip hopes to make significant inroads into the previous 15,000.
The by-election follows the death of long-serving Labour ex-minister Michael Meacher.
Nigel Farage claims Mr Corbyn's election as Labour leader will help Ukip tempt voters to switch allegiance.
Labour frontbencher Jonathan Ashworth said his party needed to find a response to Ukip's threat in its heartlands.
He told BBC2's Newsnight: "Ukip did well at the general election. They didn't take seats but they certainly took votes off Labour.
"If they have done that tonight then we need to think about how we take that on.
"But Jim McMahon is a superb candidate and he is going to be a superb MP if he gets elected."
He added: "In a town like Oldham, it's a Labour town so I don't want to see Ukip doing well. If the reports are coming in that they think they are doing better than expected then obviously as a Labour Party we are going to have to think about that and decide how to respond to Ukip."
Mr Meacher had a majority of 14,738 in May, but Ukip points to the party's strong performance in last year's Heywood and Middleton by-election as evidence that it can shock Labour in its Northern heartlands.
Ukip's candidate in Oldham West, John Bickley, nearly overturned an 11,000 Labour majority at the Heywood and Middleton by-election last October, losing by just 600 votes.
He fought that seat again in May, with Labour's majority recovering to 5,300.
Mr Farage has made a series of visits to Oldham, while Mr Corbyn has only campaigned there once - and was forced to cancel an appearance last week as he dealt with internal divisions over air strikes on Islamic State in Syria.
Labour's candidate in the seat is council leader Jim McMahon, who defeated more left-wing candidates including former MP Chris Williamson in the selection process.
Mr Corbyn said: "The British people need the Labour Party's united action in support of social justice more than ever. We must redouble our efforts to achieve a fairer Britain. This starts today - in Oldham."
He added that Mr McMahon's council experience meant he knew the effects of austerity measures: "We need a Labour MP who knows the impact of cuts and who will join us in Parliament in taking the fight to this Tory government."
The other candidates are: Liberal Democrat Jane Brophy; Conservative Party James Daly; Green Party Simeon Hart; and Sir Oink A-Lot of the Official Monster Raving Loony Party.