The signs have been visible for a while that Big Data is crossing the chasm and moving into everyday operations including the world of entertainment. Initially Big Data was used to predict blockbusters but now they are being used to write them, indicating that data scientists will be the next most critical job for a wider range of industries..
Recent advances in digital production and distribution by Netflix, Hulu, and Redbox have extended the overall productivity and reach of the film industry resulting in fierce competition for financing films.The increasing availability of data coupled with the abundance of sophisticated technologies, tools, and applications gives filmmakers an opportunity to forecast revenue before a movie is released.
Now the decision making power of Big Data has gone even further. Netflix used Big Data to select House of Cards and its actors, and actually knew it would be a big hit before they aired ; and Share Dimension in the Netherlands is recommending where and when films should be shown to maximise revenues. The latest development is news of Vault, a startup with an analytics engine that can read film scripts and recommend changes as well as highlighting the best actors, studios and locations for the film.
The big question around analytics technology like this is whether it will be accepted by the industry. There's certainly plenty of evidence to suggest that a change is needed:
- Studios are increasingly reliant on their biggest titles, which have migrated from being "summer blockbusters" to a year-round carnival
- Industry experts such as Spielberg have predicted that studios will not be able to survive a few of these films failing
- Audiences are becoming more choosy about which films they see, as ticket prices rise and alternatives become more compelling
- The industry has been casting around for "gimmicks" such as 3D and now VR which audiences have not accepted
- Films increasingly need to be globally successful and need to somehow appeal to ever-broader audiences
To me, this looks like an industry ripe for disruption, where executives are used to gambling and where the opportunity cost of refusing the new technology (while their rivals adopt it) is likely to outweigh reluctance to embrace algorithms. It looks a lot like the financial services industry in the 1980's.
This is also the moment where Big Data entrepreneurs will be able to turn a vision into reality and change the world, by bringing the benefits of Big Data to the masses. Big Data, analytics, and tools are changing the world, but just because they can read scripts and correlate data sets doesn't mean that human intelligence no longer has a role. In fact, I would argue that these algorithms will free human data scientists from the activities they (objectively) aren't so good at, giving them better information with which to make the creative decisions the computers can't.
The examples above show that Big Data isn't only becoming available to smaller businesses, but that it's affecting industries that were considered very difficult to disrupt. If highly creative film and TV industries; strongly regulated and private medical research industries; and the subjective and human-taste world of chefs are being disrupted by Big Data, it's unlikely that any industry is immune.
With Big Data crossing the chasm, within a few years most industries will have been thoroughly disrupted and creating significant new innovations will be difficult. Today, businesses have the opportunity to get ahead of their competitors by hiring data science teams and embracing Big Data.
There is a lot more innovation like this still to come, especially in transportation, logistics and healthcare. If data scientists want to be part of this trend and apply their skills to solve some of the biggest challenges, they will need to think outside the box and tackle big problems that have big rewards.
In order to make the most of Big Data and the disruption it will bring, this is the ideal time for companies to start preparing, identify how Big Data can make a difference to their business and build a strategy around it; hire a Big Data team with the right mix of people for their size and challenges; and invest in the right tools, bearing in mind that the right tool may quickly change during a disruptive time.