The Prime Minister said it "had not been a great week" as he admitted that he botched the handling of the row over his financial affairs.
David Cameron told the Conservative Party spring forum on Saturday that they should "blame" him for the way this week's revelations were handled.
Cameron told activists that he was "angry" about what was being said about his father, who he "loves".
The Tory leader is facing an inquiry over why he failed to register his £30,000 stake in his father’s offshore investment trust.
Cameron confirmed on Saturday that he would be publishing his tax returns as he accepted the blame for the controversy over his connection to his late father's offshore business interests.
Speaking at the spring forum in London, he said: "It has not been a great week. I know that I should have handled this better, I could have handled this better.
"I know there are lessons to learn and I will learn them.
"Don't blame Number 10 Downing Street or nameless advisers, blame me."
He added: "I was very angry about what people were saying about my Dad.
"I love my Dad, I miss him every day. He was a wonderful father and I am very proud of everything he did.
"But I mustn't let that cloud the picture."
The leak of more than 11 million documents this week from the Panamanian law firm, Mossack Fonseca, cast an unprecedented light on the way the rich and powerful are able to use tax havens to shield their wealth.
Ian Cameron, the Prime Minister’s father who died in 2010, was reportedly named as a client of the firm.
The Prime Minister has insisted he does not have “anything to hide” about his financial affairs.
Cameron has faced intense pressure to detail his interests since the Panama Papers leaks included details of Blairmore Holdings - which used “bearer shares” to protect investors’ privacy.
On Friday, Labour leader Jeremy Corbyn condemned the "weasel-worded statements" issued by the Prime Minister.
Corbyn said: "It took five weasel-worded statements in five days for the Prime Minister to admit that he has personally profited from an undeclared Caribbean tax haven investment deal.
“His determination to conceal that arrangement over many years raises serious questions over public trust in his office and his willingness to be straight with the public.
“Tolerance of tax avoidance and tax havens, and inaction on tax evasion, is denying funds to the public purse and leads directly to cuts in services and benefits that are hurting millions of people in Britain.
“The Prime Minister has lost the trust of the British people. He must now give a full account of all his private financial dealings and make a statement to Parliament next week.
"Only complete openness from the Prime Minister, and decisive action against tax avoidance and evasion, can now deal with the issues at the heart of this scandal.”
