Last week (25 June 2014) saw an important announcement from the New York Attorney General. He announced that Barclays Bank is to be prosecuted concerning its operation of a 'dark pool'. A dark pool is basically a private trading area which a bank can operate on behalf of its clients, or anyone else to whom the bank grants access. It is dark because it doesn't operate to the same level of transparency as conventional exchanges. The accusation is that Barclays allowed high frequency traders into its dark pool and allowed these traders to prey on the trading activity of the other investors within the pool, including Barclays' own clients.
This is an astonishingly important announcement for two reasons: First, it is important for Wall Street but it also important for Facebook, Google, Big Data, data protection, the Internet of Things and thus, quite possibly therefore the future of humanity itself.
First Wall Street: What is happening within Barclays' dark pool is almost certainly similar to what is happening in the dark pools operated by almost all the major banks. It is also pretty similar to what is happening in the 'light pools' that constitute the official Wall Street stock exchanges (just read Michael Lewis's 'Flash Boys', published a few weeks ago if you want validation of this). This will therefore be a test case and rather than go after one of the Big Beasts, the Attorney General has sensibly chosen to pick off an already wounded juvenile. Barclays (in the US) is a foreign bank, it is a peripheral player (albeit one with a very large dark pool) and it is already discredited by its actions in rigging inter-bank lending rates. It is therefore easy prey, but bringing it down will provide the ammunition necessary to tackle, or at least discipline, the major players. You can bet that there are a lot of people on Wall Street right now really focused on how this case plays out, even if the mainstream media has yet to really wake-up to its significance.
But this isn't about just about Wall Street. What is playing out here are the first attempts to understand and regulate the world of the algorithm. High frequency trading is driven by algorithms and exploits one of an algorithm's principle characteristics, which is its speed in processing large amounts of data. High frequency trading illustrates the power of algorithms and also their potential for abuse. High frequency trading is not illegal (yet), but it is abusive. It is only not illegal because the law makers don't really understand how algorithms work and no-one has worked out a way to stop people who do understand them from using them in an abusive way. Interestingly the Attorney General has not tried to establish that high frequency trading is illegal, rather that Barclays misrepresented its dark pool as offering protection from the abusive behaviour of high frequency traders.
Algorithms colonised Wall Street for two reasons: first Big Data was already there in the form of the vast amount of information flowing through the financial markets and; second, Wall Street could afford to pay top-dollar for the relatively small group of geeks who actually understand algorithms. But this is about to change. The pool of geeks is expanding and pools of data, large enough for complex algorithms to operate within, are now developing in many other places, driven by the growth of Big Data and the Internet of Things.
Which brings us to Facebook. In many ways Facebook is a dark pool, except the data within it isn't data about financial trading, it is data about human behaviour. Now I don't want to suggest that Facebook is trading this information or necessarily inviting access to this data for organisations who are going to behave in an abusive or predatory way. In a somewhat ironic sense of role reversal, the PRISM affair has revealed that government agencies (i.e. the NSA and the UK's GCHQ) are the equivalent of the high frequency traders. They are the people who want to get into Facebook's dark pool of data so they can feed it through their algorithms and Facebook has been doing what (little) it can to resist their entry. But of course there is nothing at the moment to really stop Facebook (or for that matter Google or Twitter) from allowing algorithms into their data pools. In fact, we know they are already in there. While there may not be abusive activity taking place at the moment there is nothing to stop abusive behaviour from taking place, other than the rules of integrity and behaviour that Facebook and Google set for themselves or those that might be set by the people Facebook or Google allow into their pools. Remember also that Facebook needs to generate sufficient revenue to justify a valuation north of $80 billion - and it is not going to do that simply through selling advertising, it is going to do that by selling access to its pool of data. And, of course, the growth of Big Data and the Internet of Things is creating vast data pools that exist in far more shadowy and less obvious places that Google and Facebook. This is a recipe for abusive and predatory behaviour, unless the law-makers and regulators are able to get there first and set-out the rules.
Which brings us back to New York versus Barclays. It is not just Wall Street and financial regulators who need to focus on this: this could prove to be the opening skirmish in a battle that will come to define how society will operate in the world we are now entering - the world of the algorithm. I can't lay claim to understanding how this may play out, or how we are going to regulate the world of algorithms. The only thing I do know is that the abusive use of algorithms flourishes in the dark and the daylight of transparency is their enemy. Trying to drive a regulatory stake through the heart of every abusive algorithm is a near self-defeating exercise - far better is to create an environment where they don't have a competitive advantage.