George Osborne has described being "shocked" to discover that some of the wealthiest people in the country pay "virtually no" income tax.
The Chancellor said he had seen "anonymised" tax returns submitted by multi-millionaires using aggressive avoidance schemes to dramatically reduce their tax bills.
HM Revenue and Customs (HMRC) found that the income tax rate among some of the highest earners was, on average, only 10%.
Osborne said the HMRC study convinced him of the need to "take action" to ensure high earners pay more income tax.
In last month's Budget he limited how much people could offset their tax bills by investing in businesses or donating to charity.
Anyone seeking to claim more than £50,000 of tax relief in any one year will have a cap set at 25% of their income from 2013.
HMRC found hundreds of millions of pounds of income tax is avoided by using legal loopholes to reduce their bills.
The main methods included writing off business losses, offsetting the cost of business mortgages, and borrowing on buy-to-let properties - all against their income tax bill.
Others took advantage of tax relief on donations to charity.
"I'm talking about people right at the top. I'm talking about people with incomes of many millions of pounds a year.
"The general principle is that people should pay income tax and that includes people with the highest incomes."
Charities have warned that limiting tax relief on donations will reduce philanthropic giving.
But Mr Osborne said: "I was very clear in the Budget that we are specifically looking at making sure we are still encouraging philanthropy and charitable giving. But that is a specific issue we can deal with."
Mr Osborne has been criticised by Labour for cutting the top rate of income tax from 50p to 45p, in a move that will cut the tax bills of people earning more than £150,000 a year.
But he said the 50p rate introduced by the last Labour government was "spin over substance".
"You produce a press release which says Britain has a 50p tax rate while you preside over a tax system where some people are paying zero per cent tax," he said.
"I've come up with a Budget that has reduced the 50p rate to 45p, so we don't have the highest income tax rate in the world. But I've also asked people who are currently paying zero to pay income tax."
Mark Serwotka, the general secretary of the Public and Commercial Services union which represents civil servants, said he was surprised at Osborne's surprise.
"The only shocking thing is that someone who was shadow chancellor for five years before becoming chancellor apparently did not realise that our public finances are deprived of tens of billions of pounds every year through tax avoidance and evasion by the super-rich," he said.
"Osborne should spend less time courting the very wealthy and agreeing to cut their taxes even further and more time talking to his own staff in HM Revenue and Customs who have been warning about this for years.
"But now he has seen the light, he must accept his government's cuts are not only damaging they are entirely unnecessary."