The opportunities made available by Big Data are truly mind-boggling. For years retailers and other consumer facing businesses had dreamt of being able to know about their customers' likes and how best to communicate with them. Suddenly the big data revolution in the past few years has offered a paradise of information to help answer these questions, in ways that companies in the 20th century couldn't even comprehend.
But for many in the products and services industries, it feels like progress in using Big Data to answer these critical business questions has ground to a halt. In a survey of senior business leaders at 1800 large companies in North America and Europe, only 4% were classified as being "data elites" and successfully using data to improve business performance, whilst over a quarter reported seeing "no or little benefit" from their initiatives. This trend has been a head-scratcher for many. Although the fruits of Big Data success are clear to all, there is still a mystifying inability to use data effectively. Execution has been far from a formality.
A major contributor to this trend is that companies aren't changing their culture when introducing these initiatives to ensure that they become true data-driven organisations. It requires a seismic change in the way that companies run when opinions are swapped for evidence - and requires more than just investment.
The supermarket chains have always been at the forefront of using data, with Tesco and Walmart being early trailblazers in using membership cards to glean vital consumer data and harness it to become two of the biggest retailers in the world. But all were upstaged by Amazon.
The Seattle-based company rose from start-up to tech titan in a matter of years, against all the predictions of commentators who believed the monolith of American supermarkets, Walmart, would simply crush them. Their competitive advantage was not that they had more data than the industry incumbents - far from it, they had considerably less. But it was their use of data, and the culture of evidence-based decision making, that was the key to achieving success.
A former employee was quoted on Entrepreneur.com as saying "Bezos's core innovation was to place data at the center of his corporate culture." Bezos demanded that all decisions were backed up by strong evidence, not by opinions. This mantra permeated throughout the company; 'if it can't be measured, it isn't worth doing'. He constantly raged against the HIPPO (Highest Paid Person's Opinion) winning arguments, the invisible force he believes drives decision making at most large corporations.
How can companies become more like Amazon, and develop a data-driven culture - something that will be necessary in the digital age?
Business leaders need to focus on the people, their communications and meetings rather than the data itself. It is the way in which decision making is made that should be most influenced by data, but it is the hardest change to make. Constantly questioning "where is the evidence?" from the C-suite downwards can immediately begin to help employees rethink their use of data.
Setting the expectation of using data is a tough change for any organisation. It is in fact much easier to start a new culture from scratch, as has happened at many tech start-ups. But in the digital age, embracing big data is not a choice - it is a requirement for survival. Building a culture of evidence-based decision making is the foundation that needs to be built.