We are at the crossroads of a century where human interaction with technology is unprecedented. There is an overall fascination among businesses with predicting how the consumers will behave. There has been a gradual shift from “what you do” to “what you might do”. Advances in information technology, data gathering, and analytics are making it possible. Companies are using ‘predictive analytics’ to understand consumer’s shopping and personal habits to target them better.
Big data, AI, Statistical Models, Heuristic Models, Algorithmic programming to predictive analytics are becoming ubiquitous to understand huge data sets, unearth patterns to draw insight and act upon it. We have heard a lot about achievement through algorithmic thinking – though the best-known claim is still Brad Pitt in “MoneyBall.”
Retailers such as Walmart and Target are able to match sales history, demographic data, cookie data to their product SKU to predict the need and the timing of price markdown. There are a bunch of players who play in this space such as Semantics3, Indix, Wiser to help out the bricks n mortar players.
"We are leaving the world of search, and entering the world of recommendation " ~ Chris Anderson
Next, let us look at the recommendation engine of Netflix, which decides what you’ll watch next. I chose Breaking Bad as something that I like. Netflix gives me top picks recommending similar genre and or category. In this case, Walking Dead, Lost, House of Cards in Top Picks while Pulp Fiction in "More Like Breaking Bad" Category. Once I start watching different movies and TV series, Netflix will do better recommendations. This is a push model compared to the pull model where one has to browse through (VHS tapes at BlockBuster ) DVDs at RedBox.
The company estimates that recommendation drives 75% of viewer activity. It keeps track of what you played, searched for or rated, as well as the time and device. This data is fed into their algorithms, which then match your behavior with similar users to infer your preferences. The viewing behavior of the same person is different depending upon the time of the day, day of the week or even the device he is using. Netflix goes to the depth of understanding where they show a particular movie or TV show optimized to your multi-device screens.
Enter Predictive Analytics, Netflix takes past viewing data, sifts through it to understand patterns of movie watching and then kicks of the Algorithm to recommend. It takes a deep understanding of data science, machine learning to help the machine predict what the viewers like to watch. The algorithms will see another paradigm shift as Netflix rolls out its services to 130 countries. Seems simple, it is not.
Take the example of Facebook, where the recommendation is not that simple. Facebook Newsfeed recommends other pages based on the Pages you Like. There is a lot of False Positive built into the recommendations. The reason being that “a person who Likes a page called “Save The Dolphins” doesn’t mean that the person would like to see “Save The Turtle” as an advertisement in their news feed. This is a classic Prediction dilemma.
For a moment think about the quote by Mr.Anderson, Search is a Pull strategy (similar to BlockBuster or RedBox) while Recommendation is a Push strategy (Netflix).
Xavier Amatriain, who led Algorithms Engineering team, at Netflix to improve personalization, search, recommendations cites that it is not an easy problem to crack in his blog. Recommendations depends on:
Relations to others
We see a growing prominence placed on data analytics. More and more businesses will continue to rely on audience learning techniques and develop a better sense of what individuals with a particular taste or behavior patterns prefer. This knowledge will continue to allow them to make predictions about how these individuals might react to a particular design, content message or ad campaign type.
As I was about to sum up this article and submit, I see LinkedIn's own recommendations for me to write posts. The recommendations seem based on News Recency - Yahoo, Google and Twitter are in news for one or the other reason. I gather Deepak Agarwal and his team at LinkedIn are working on a recommendation engine. I hope to see LinkedIn making interesting content recommendations.
Industries that are seeing seismic shifts and adapting to consumer preferences and tastes are Airline Ticketing, eCommerce, Gaming, Hotel, Retail Industry while industries such as Market Research, TV, Music, Recruitment, will have a direct impact. Predictive Analytics will pave the way to serve Personalized messaging and Proactive messaging.
I ended up watching Better Call Saul which is on the top Left Row & Column (meets the eye first kind of recommendation).
Please share your thoughts on Predictive Analytics and Recommendation Engines below this post. You can read other interesting posts here.