The new head of Britain's electronic eavesdropping agency has accused internet firms of being "in denial" of the role their networks play in terrorism and demanded they open themselves up more to intelligence services. GCHQ director Robert Hannigan said they had become the "command-and-control networks of choice" for a new generation of web-savvy criminals and extremists, such as Islamic State (IS) jihadists.
Better arrangements had to be developed to allow security and intelligence agencies to police online traffic, he said in an outspoken article for the Financial Times, warning firms that their users did not want their social networks used "to facilitate murder or child abuse".
"GCHQ and its sister agencies, MI5 and the Secret Intelligence Service, cannot tackle these challenges at scale without greater support from the private sector, including the largest US technology companies which dominate the web," he wrote. "I understand why they have an uneasy relationship with governments. They aspire to be neutral conduits of data and to sit outside or above politics. But increasingly their services not only host the material of violent extremism or child exploitation, but are the routes for the facilitation of crime and terrorism.
"However much they may dislike it, they have become the command-and-control networks of choice for terrorists and criminals, who find their services as transformational as the rest of us. If they are to meet this challenge, it means coming up with better arrangements for facilitating lawful investigation by security and law enforcement agencies than we have now."
The question of state surveillance of communications was thrust under the spotlight by the exposure - by US whistleblower Edward Snowden - of secret mass data collection programmes run by the US and UK authorities. Mr Hannigan conceded that GCHQ had to "show how we are accountable for the data we use to protect people" and was "happy to be part of a mature debate on privacy in the digital age.
"But privacy has never been an absolute right and the debate about this should not become a reason for postponing urgent and difficult decisions. To those of us who have to tackle the depressing end of human behaviour on the internet, it can seem that some technology companies are in denial about its misuse," he went on.
"I suspect most ordinary users of the internet are ahead of them: they have strong views on the ethics of companies, whether on taxation, child protection or privacy; they do not want the media platforms they use with their friends and families to facilitate murder or child abuse.
"I think those customers would be comfortable with a better, more sustainable relationship between the agencies and the technology companies."
Mr Hannigan cited the fact that IS stopped short of showing actual beheadings in gruesome videos announcing the killings of Western hostages - incuding British aid workers David Haines and Alan Henning - as proof of extremists' increasing expertise in online propaganda.
"They have realised that too much graphic violence can be counter-productive in their target audience and that by self-censoring they can stay just the right side of the rules of social media sites, capitalising on western freedom of expression," he said.
Smartphone and other mobile technology had "increased the options available exponentially" to conceal terrorist activity, he said, including applications "proudly advertising that they are 'Snowden approved'." Emma Carr, director of Big Brother Watch, said: "It is wholly wrong to state that internet companies are failing to assist in investigations.
Continued:
"The Government and agencies have consistently failed to provide evidence that internet companies are being actively obstructive. These companies have consistently proved through their own transparency reports that they help the intelligence agencies when it is appropriate for them to do so, which is in the vast majority of cases.
"Public debate on this issue would make the country stronger and more unified, yet we have so far failed to achieve this in the UK. Perpetuating falsehoods about the nature of relations between internet companies and the intelligence agencies is certainly not going to help."
The Prime Minister's official spokesman said David Cameron "very much shares the view that's being expressed there around the use of web-enabled, internet access technologies by violent and extremist groups amongst others, and the need to do more".
Former ambassador to the United States Sir Nigel Sheinwald was appointed as special envoy on intelligence and law enforcement data sharing in September and had begun work on addressing the issue. That role has a specific responsibility to discuss data sharing with the US authorities, with US companies," the spokesman said.
Jim Killock, the executive director of the Open Rights Group, said it should not be for technology companies to decide when to hand over data to the intelligence services. He said: "If tech companies are becoming more resistant to GCHQ's demands for data, it is because they realise that their customers' trust has been undermined by the Snowden revelations.
"It should be down to judges, not GCHQ nor tech companies, to decide when our personal data is handed over to the intelligence services."
Former foreign secretary Sir Malcolm Rifkind, the chairman of the Intelligence and Security Committee, said some of the technology firms had an "amateurish" approach to monitoring alleged criminal behaviour. He told BBC Radio 4's World At One: "I think they should recognise that they perhaps have a greater duty than many of them acknowledge to monitor what is happening on their systems.
"They do that to a certain degree - if they hear of evidence that may imply that a serious crime is about to be committed, they often do inform the police. But their monitoring systems seem to be fairly amateurish in comparison to what they could be."
He added: "If you build into your systems methods that prevent the intelligence agencies, even when they have lawful authority, from being able to find out what's actually being said from one terrorist source to another, that is making the problems of protecting the public infinitely more difficult than they would otherwise be."
Journalist Glenn Greenwald, who worked with Mr Snowden on his revelations about official eavesdropping, said Mr Hannigan's comments represented "probably the most extreme fear-mongering screed that has come from a senior national security or intelligence official since the Snowden reporting began".
Mr Greenwald told Channel 4 News: "It invoked the scary spectre of Isis in the very first paragraph and from there went downhill in trying to scare people into believing that this reporting has made them vulnerable to terrorism. It's exactly the kind of irrational fear-mongering that renders reasoned discourse very difficult."
Alejandro Mayorkas, deputy secretary of the US Department of Homeland Security, told Channel 4 News: "The tech companies recently developed a new programme that prohibits the Government from investigating individuals that might very well be of concern to us from a national security perspective, and that is something that we are quite at odds with.
"On the other hand, the freedom of expression - the First Amendment right of individuals - is something that we do hold sacred. The line between freedom of expression and support and furtherance of terrorist activity is a line that we draw as clearly as we can and we in the Department of Homeland Security tackle quite vigorously."
Mr Mayorkas added: "Our strongest ally, our strongest partner, our strongest colleague in this effort is the UK."