One in four police "stop-and-searches" have been deemed unlawful by the police watchdog, which also warned that black and minority ethnic groups were stopped and searched more than white people.
And it warned that stop-and-search concerns had slipped down the agenda of chief constables after the furore over Stephen Lawrence's murder and allegations of racism against police.
Around 27% of the 8,783 stop and search records examined did not include sufficient grounds to justify the lawful use of the power, the watchdog discovered.
One in four police "stop-and-searches" have been deemed unlawful
The report was undertaken after renewed concern about the way police use stop and search on the back of the 2011 riots.
Home Secretary Theresa May ordered Her Majesty's Inspectorate of Constabulary to conduct its first ever inspection of the use of the powers in all 43 forces in England and Wales.
"Officers are not adhering to the guidance on too many occasions," HM Inspector of Constabulary Stephen Otter said. "It has slipped down the chief constables' agenda since the Stephen Lawrence Inquiry report."
Otter warned that use of the powers was becoming a "habitual" practice that was "part and parcel" of officers activity on the streets.
The watchdog found that a quarter of respondents believed that certain groups of people in society are likely to be stopped and searched more often than others, with a third attributing this to unlawful discrimination.
And the number of people concerned about racism increased to around 55% among black and minority ethnic respondents.
"The police have a very long way to go before they regain the trust of black and ethnic minority communities," Cecilia Goodwin of the Stop and Search Legal Project and a member of BSB Solicitors.
"People are still being stopped every day, several times a day, just for how they look. And the damage has been done in those communities, people believe the police are out to get them.
"It was a key component of what sparked the riots, yes some people wanted to steal things, but others were furious about they way their communities were being targeted."
Last month, a report found that racial profiling of stop-and-search suspects makes absolutely no difference to crime figures, according to figures from the Equality and Human Rights Commission found.
The study was a follow up to the Commission's report Stop and Think, published in 2010, which threatened police forces with legal action after it found black and Asian Britons were still being unfairly targeted for stop and searches.
The HMIC report found it was "surprising how little effort was given to monitoring how well stop and search powers were used to prevent crime and catch criminals."
More than one million stop-and-searches are recorded every year - equal to more than 300,000 hours of police officers time - but on average only about 9% of the incidents result in an arrest.
The inspection found that the majority of forces - 30 out of 43 - had not developed an understanding of how to use the powers of stop and search so that they are effective in preventing and detecting crime.
Among its recommendations, HMIC said police chiefs and the professional-standards body, the College of Policing, should write a clear specification of what makes up the effective and fair use of stop and search powers in the authorised professional practice document.
"This was really just like writing the manual of how it should just work, this is not rocket science," Mr Otter said. "These are not new ideas, they should just simply be done."
A Home Office spokeswoman said: "The Home Secretary is grateful for HMIC's report, which she commissioned in December 2011. She has made it very clear that the Government supports the ability of police officers to stop and search suspects within the law.
"But if stop and search is being used too much or with the wrong people, it is not just a waste of police time, it also serves to undermine public confidence in the police.
" That is why last week the Home Secretary announced a public consultation into the use of stop and search.
"The Government will respond to the HMIC report and the replies to the public consultation with specific proposals by the end of the year."
Neena Samota, of the StopWatch campaign, which campaigns against the disproportionate use of the powers, said: "Most police forces continue to view the power to stop and search a vital part of policing communities, keeping them safe and reducing crime.
"StopWatch strongly contests this view; existing data on ethnic disproportionality, lack of public trust in police and
complaints procedures raise more questions around legitimacy and lawfulness.
"We welcome the HMIC report which confirms the over-reliance of stop and search powers by the police."