Interest in Big Data analytics has increased sharply over the last twelve months, and usage by commercial, non-profit and public sector organisations surged. !
What's driving this? One reason is that storage costs have fallen so far that more and more organisations are able to "store everything." Another important factor is the growing number of sophisticated analytics tools that are available.
Last year venture capitalists invested $1.3 billion in Big Data analytics companies, an increase of 217% over investment in the previous 12 months, and 19 deals were struck in the last quarter of 2013 alone, according to GP Bullhound, an international technology investment banking group.
These and other investments are now bearing fruit, and thanks to the raft of new products on the market companies are able to gain insights into their data far more easily and effectively than ever before. By next year Gartner predicts that the majority of data analytics projects will be led by everyday users rather than IT departments.
As a result, organisations are able to use Big Data analytics to help them make a much broader range of business decisions than was possible in the past. Arguably the earliest usage was to make decisions about loans - leading to the formation of the credit rating industry - but today Big Data analytics is used in areas as diverse as logistics, budget planning, customer retention programs, and marketing campaigns including keyword selection and A/B testing.
Big Data is being intensively used in retailing: US retailer Walmart collects 2.5 petabytes of data every hour from its customers, and this is used by its data analysts to make decisions about everything from new products to the right offers to generate for individual customers to increase their weekly spend.
It has also given rise to entirely new fields of analytics. For example, many human resources departments are increasingly turning to "people" analytics to find the right people both internally and externally to fill vacancies.
One company that has done this is Xerox - it adopted people analytics to help make recruiting decisions for its call centres. Its people analysis revealed that candidates with certain personality types are more likely to stay in the job long enough for the company to recoup the $5,000 it invests in training new recruits. Thanks to this insight Xerox cut the attrition rate amongst call centre workers by 20% in six months.
Big Data analytics is also becoming increasingly important in the banking and financial services sectors to help with activities such as the calculation of risk in large portfolios and the detection and prevention of financial crime. And in the field of healthcare, Big Data analysis using systems such as IBM's PureData is helping physicians tailor treatment and medication more accurately.
Increasingly, Big Data analytics is being adopted in the manufacturing sector too. For example, General Electric now puts sensors in its jet engines and gas turbines to generate data which can be used to determine when equipment needs servicing. Since more than 50% of the company's revenue comes from servicing the equipment it sells, analysing this data is transforming the company's service business.
Not surprisingly, the surge in demand for data analytics has had a profound effect on the analytics job market. As part of InterQuest Group, IQ Analytics is a leading recruitment specialist focused on the analytics market space; they have experienced a dramatic increase in the demand for skilled and experienced professionals over the last two to three years. IQ Analytics provides opportunities for talent on both sides of the Atlantic, in the US and the UK analytics markets; their own analysis suggests the UK catching is up to its US colleagues in terms of both demand for and sophistication of, analytics professionals, and as a result the target income in these roles continues to rise.
The UK's education sector continues to show an increasing awareness of and response to the much publicised skills gap in this field and as a result this is one area of the jobs market that graduates with the relevant skills are finding it relatively easy to enjoy rewarding post-graduate employment.