How AI for video will transform the watching experience for everyday people
It's Christmas morning December 25, 2016 at 6:17 am. You can't sleep because this year, for some reason, you seem to be more excited about what's under the tree than in years past. But you know the rest of the home is still sleeping so you reach for your phone.
On the screen is a text message you've never seen before. It's from Disney?
Star Wars, Rogue One came out in theaters today, but according to your text you can watch the entire thing this morning from bed. Inside a text message.
Sure it seems a bit strange to watch a movie inside a text message, but at one point it seemed crazy to carry around a supercomputer in your pocket that can access all the world's knowledge floating around in some "cloud".
But today, for the first time ever, a machine is able to understand what's happening inside movies and TV shows. When you log into Netflix and find yourself scrolling endlessly trying to find something to watch using only a picture and a few sentence synopsis, you wonder why there isn't a better way.
With the recent advances of AI for video, we can use mathematical algorithms to section movies and TV shows into clips, help recommend more interesting content to watch, and the holy grail, search for something using plain language.
The latest Apple TV + Siri marketing alludes to some of the benefits of this new AI:
Credit: Sean Everett
Credit: Apple Siri
How does AI for video do this?
The basics are pretty easy to understand. First, translate speech to text. Compare text to Wikipedia to help the machine understand concepts. And then rank by relevance based on the popularity of the content. The latest Avenger movie might rank much higher as a suggested movie rather than a 10-year old independent documentary.
But can we take this further, beyond what Siri, Amazon and Google are doing for their online video businesses? Of course we can, but it means we need to go a bit deeper into what it means to understand what's happening inside a video.
Here's an example of a next-generation concept you might experience in an upcoming self-driving car.
Credit: Sean Everett
In the connected car space, Uber is already trying to enable things like this through it's relatively recent Trip Experiences API. They show examples of different alternatives, like the one below for listening to music, but what if you applied it to watching video during your commute or trip across town?
Credit: Uber
Imagine you had a very simple interface to help you find what you wanted to watch: "I have 10 minutes and want to laugh". The 10 minutes part Uber already knows based on how far away you are from your destination. The second part, searching for videos based on your mood, comes from this AI for video concept. If there's a million pieces of content on YouTube, but not all of them are tagged for funny or have keywords for your innate and preferred sense of humor, then this experience would never be possible.
But today, a machine is able to detect that you're into less George Carlin, and more Ellen Degeneres, when it comes to tickling your funny bone.
So back to our original example of watching a movie like Star Wars in a text message. It could happen via the messaging apps like WhatsApp or Facebook Messenger.
Credit: Sean Everett
Because you've watched other Disney action adventure movies like Star Wars The Force Awakens and Captain America Winter Soldier, the software is smart enough to rank that as a priority and offer it to you. Maybe you can start watching the beginning few minutes of the hook for free, and then choose to continue watching the entire movie with ads or buy the movie outright to watch without ads.
It may seem strange that any major film studio would want to disrupt its distribution relationship with theater chains, but as we all are well aware, box office attendance is down.
And when you do the simple math. Reach 1 billion people through Facebook on opening day multiplied by $15 per digital "ticket", you're looking at a whopping $15B in opening weekend box office receipts. That digital theater just gave Disney 7x the revenue than the entire Force Awakens box office take of $2B.
When you consider the global reach of Facebook or video in SMS (billions of people) who may not have access to a theater or even a TV, you start to realize that a theater experience is a first world video platform without the reach of an entire-world online video platform.
Whether Bob Iger or Mark Zuckerberg agree with this premise is anyone's guess. But we know that if consumers are willing to pay their hard-earned dollars, then it probably makes sense.