Creating a British Bill of Rights is unnecessary as it will simply affirm the measures in the European Convention on Human Rights (ECHR) but damage the country's moral authority internationally, peers warned.
The cross-party House of Lords EU Justice sub-committee said Michael Gove's plan to scrap and replace the Human Rights Act 1998 - introduced by Labour to bring the ECHR into British law - could deal a blow to the UK's standing in organisations like the EU and the Council of Europe.
Moreover, scrapping the Act would lead to an increasing reliance on the more strongly enforced EU Charter of Fundamental Rights in British courts, in what the committee deemed a "perverse consequence" of the Justice Secretary's plans.
Meanwhile, with the proposal lacking the support of the devolved nations, it could end up as an English Bill of Rights with uneven laws governing the nations of the UK.
A Tory manifesto pledge, the Bill of Rights was omitted from the Government's first legislative programme amid opposition from backbenchers and it is unclear whether it will feature in the forthcoming Queen's Speech on May 18.
Home Secretary Theresa May recently called for Britain to pull out of the ECHR but Mr Gove made it clear he wanted to remain in the convention while reforming the law at home.
The committee told the Government to "think again" before proceeding.
Its chairwoman, Baroness Kennedy of the Shaws, said: "Our evidence from the Secretary of State for Justice was the first time the Government has explained why it wants to introduce a British Bill of Rights.
"The arguments seemed to amount to restoring national faith in human rights and to give human rights a greater UK identity.
"The proposals he outlined were not extensive, and we were not convinced that a Bill of Rights was necessary.
"Many witnesses thought that restricting the scope of the Human Rights Act would lead to an increase in reliance on the EU Charter of Fundamental Rights in UK courts, which has stronger enforcement mechanisms.
"This seemed to be a perverse consequence of a Bill of Rights intended to give human rights greater UK identity.
"We heard evidence that the devolved administrations have serious concerns about the plans to repeal the Human Rights Act.
"If the devolved Parliaments withheld their consent to a British Bill of Rights it might very well end up as an English Bill of Rights, not something we think the Government would want to see."
A Ministry of Justice spokeswoman said: "This Government has a mandate to reform and modernise the UK human rights framework.
"Our Bill will protect fundamental human rights, but prevent their abuse and restore some common sense to the system. We will fully consult on our proposals."