The head of the prison and probation service has described the threat of Islamic radicalisation behind bars as significant.
Speaking to BBC Panorama, Michael Spurr the chief executive of the National Offender Management Service of England and Wales (NOMS) said: "There is a significant risk, given the fact that we manage some very dangerous people.
"Our job is to minimise that risk becoming a reality - that somebody in prison becomes radicalised and commits a terrorist offence."
He warned there could be a "whole range of different potential scenarios (where) people could be hurt" if NOMS failed in its job to protect the public from extremists.
Over the last ten years the number of Muslims in prisons in England and Wales has doubled, with the figure reaching 11,729 in 2013.
There are about 100 al-Qaida-inspired Islamist terrorists behind bars.
Also on the programme and speaking for the first time since his release from prison for trying to bring Sharia Law to the streets of London, is Jordan Horner, who has taken the Islamic name Jamaal Uddin.
He claimed he had converted other prisoners during his time in prison.
Mr Horner said: "The prison officers witnessed people become Muslim and in front of them I was giving them what we call Shahada, an invitation and acceptance of Islam.
"They was becoming Muslim in front of the prison officers and they felt sort of powerless.
"They said I was trying to divide Muslims from non-Muslims, trying to get them to follow an extreme version of Islam."
He added that in less than a year he was transferred between three different jails in an effort to disrupt his activities.
In December 2012, Horner was filmed at a protest alongside Michael Adebowale who, five months later, murdered soldier Lee Rigby in Woolwich.
Horner became the first person to be placed on a five-year, landmark anti-social behaviour order, intended to stop him promoting extreme versions of Islam.
Panorama's reporter Raphael Rowe also witnessed the moment prisoner and Muslim convert Michael Coe was met on his release from jail by two convicted Islamic extremists.
Mr Coe, whose Islamic name is Mikaeel Ibrahim, said he converted to Islam while in jail after deciding it was "the way forward".
He was invited to take the faith by Dhiren Barot, himself a Muslim convert currently serving a minimum of 30 years in prison after admitting a plot to bomb New York landmarks.
Ibrahim told the programme-makers he was no extremist, despite appearing in a video supporting al-Qaida chief Osama Bin Laden who was killed in Pakistan by US special forces in May 2011.
Damian Evans, the governor of Whitemoor high security prison in Cambridgeshire - which holds about ten convicted terrorists and three times as many inmates with extremist links - said such prisoners pose a significant challenge but also told Panorama "we are generally good at identifying risk and acting on it".
Her Majesty's Chief Inspector of Prisons Nick Hardwick said Muslim prisoners have consistently reported feeling more unsafe than non-Muslim prisoners and "if they don't feel they're being protected by staff, they'll join a group that they think can provide that protection".
Justice minister Jeremy Wright said: "The police and security services do a difficult but important job making sure some of the most dangerous terrorists in this country end up where they belong - behind bars.
"Once there, we must make sure they cannot inflict their extreme views on others.
"The challenge that our prison staff face should not be underestimated but the public can be reassured - we are committed to tackling extremism."
Panorama From Jail to Jihad? airs on BBC1 at 8.30pm.