Prisoners in custody for murder and other violent offences are among hundreds who have been released by mistake.
Forty-eight suspected or convicted criminals were freed in England and Wales because of blunders in 2014/15, new figures obtained by the Press Association reveal.
In the last decade 505 prisoners have been let out in error - a rate of just under one every week.
Critics labelled the findings "disturbing", while the Prison Service insisted incidents are "very rare" and have been falling.
In one episode, Martynas Kupstys was let out of HMP Lincoln while on remand for murder in August last year. He waited for three hours at a nearby bus stop before being found and returned to custody.
Kupstys was later jailed after he was convicted of the murder of Ivans Zdanovics, 24, who died in a house fire in January 2014.
In another incident a prisoner was freed from HMP Hewell in Worcestershire in July last year after an apparent mix-up involving another inmate with the same surname. He was brought back to prison a day later.
Tory MP Philip Davies, a member of the Commons justice committee, said: "The first duty of the Prison Service should be protection of the public.
"These disturbing figures show that once a week the Prison Service release the wrong prisoner, and have done so for many years.
"This is nothing more than a shambles which puts the public unnecessarily at risk."
Shadow justice minister Andy Slaughter said the public will be "stunned", adding: "This is a further sign of the crisis in our prisons where overcrowding and violence are rife."
Lucy Hastings, director of charity Victim Support, said victims will be "alarmed and frustrated".
She said: "We know it can be distressing and worrying when offenders are released from custody – releases made in error can make this many times worse."
Statistics released by the Ministry of Justice following a Freedom of Information request show 41 individuals were wrongly released from prison and seven from court custody in 2014/15.
The number, which includes both prisoners who are serving sentences after being convicted and those on remand, was one fewer than the 49 mistakenly freed in the previous year.
Just under a quarter of all those wrongly freed in the two years were serving sentences or charged with robbery or violent offences including assault and battery.
One inmate was in custody on a firearms charge and another was being held for possessing an explosive substance.
Two of those freed in error in 2014/15 had not been returned to custody as of the end of last month, including an alleged sex offender released from court.
Six of those incorrectly released in the previous year had not been brought back to custody as of September.
In FoI responses the MoJ said the fact a prisoner is released in error does not necessarily mean that they will remain unlawfully at large if they are not brought back to custody as there are circumstances where they will not have to return.
The figures show there were three releases in error from HMP Manchester, one from HMP Belmarsh and one from HMP Woodhill over the two years. All three are categorised as high-security prisons.
HMP Bullingdon in Oxfordshire had the highest number of erroneous releases with a total of 10 between 2013 and 2015.
Historical figures dating back to 2005/6 show the number of erroneous releases peaked at 68 in 2009/10, having more than doubled in two years, before falling to the current level.
A Prison Service spokesman said: "Public protection is our top priority. These incidents are very rare but we are not complacent.
"The number of releases in error have fallen by almost a third since 2009 and the vast majority are returned to custody very quickly.
"The Prison Service investigates each incident and they are reported to the police for further action."