Hi there. In the previous weeks, you have learned how to predict future and how to estimate the quality of your predictions. Feel free to throw your glass ball away, because it's done with the help of big data. It is time to build your own big data product. Hey, we have a question from the audience. No, buddy, you don't need the magic wand anymore, turn it back to Hogwarts. I personally have been working in the field of recommender system developments for several years at and. To be honest, I'm really fond of recommender systems or RS for short and let me share my passion with you. In the up coming videos, I will give you a brief overview of recommender systems and then you will build your own scalable recommender system. It's time to do fancy things, let's go. The first question, what is a recommender system? If you read news on the Internet then you have definitely seen personalized news recommendations. When you buy items online, you usually see the recommended items. The most frequent recommendation in e-commerce websites is with item A, people usually buy X, Y and Z. Movie recommender system is another popular example. You can rate movies with likes and dislikes, or can provide a more granular feedback in the form of n star rating system. The system will automatically find out what you like most. It can be a movie genre, a favorite actor or actress, film location and so on. You might even not understand why you like these or those movie, but recommender system will generate hypothesis with high probability of trustworthiness. Music recommender system makes your running exercise more enjoyable. It makes your day more active or passive and even can change your mood. It depends on the type of music you listen, active, sad or dancing. I think each of you remember how a song intensifies your day-to-day activities. Works like magic, right? Imagine what can a well crafted playlist do, practically everything but for doing the dishes. And it is achievable with the help of recommender systems. There are too many different domains. Most common of them you see on the screen. And to make your life even more complicated, there are cross domain recommender systems. Nowadays one e-commerce website can sell books, movies, clothes, and even furniture. I speak about big e-commerce aggregators. What is interesting, recommender system can build a user model based on users browsing history. The history of his or her clicks and purchases in the movie domain, and then apply these findings to the users behavior in another domain, for example, books. People usually define several categories of recommender systems which you see on the screen. Non-personalized recommender systems are the default choice when you don't have any information about the user. It is usually called top items, most popular items or trending items over some period of time such as a day, week or overall. There are recommender systems which use content based algorithms. Items which you buy, books which you read, music which you listen to, all have attributes which identify them. For example, a book has authors, a publisher, a year of publication, a title, contents and so on. Based on this information, you can build a classification or regression model to predict users like or dislike of the arbitrary book item. Content based algorithms recommend items that are similar to those the user liked in the past. If you don't have user history, you still can benefit from content based algorithms. A user can provide some information about his or her preferences such as, favorite movie genre. Therefore, you will be able to match his or her interest with the relevant items. Neighborhood models cover the second class of recommender systems. It is a well know paradigm, people who buy A item also buy B item. There is a common problem with e-commerce websites following this paradigm. If you have the top selling item such as bananas in the grocery store, then recommender system is tucked into bananas problem. Whatever you buy, the recommender systems suggested to buy bananas in addition. In a movie domain it gets another name, Harry Potter problem. As you can understand the reasons are the same. It is a minor problem and we will discuss how to solve it in future videos. The main point of neighborhood models can be easily described in the following way. The recommender system will find a person who's behavior is similar to yours and then suggest you the items highly rated by another person. Of course, it take into consideration the items which you are already aware of and will not suggest you to buy an item which you have already bought. And breathtaking collaborative filtering algorithms, why are they so breathtaking? Simply because if you had known them in 2009, that you could have had a real chance to win $1 million. It is one more reason to invest in your education and to be precisely correct, neighborhood models are representatives of a wider family of collaborative filtering. The main idea of collaborative filtering algorithms is that people with similar tastes behave the same way. CF algorithms construct a latent user and item representations. We will talk more about CF algorithms in the upcoming lessons. So stay tuned. Knowledge based algorithms play an important role in high risk domains and domains with limited information. The good example is realty business. On average, a person doesn't usually buy a house frequently. Most of your service customers can have no buying history at all, so how do you identify which house to suggest? The only way you can do is bring in users domain knowledge to build a model. The recommender system can provide an interface to specify constraints such as number of bedrooms in the house. Then each page is constructed in a way to help users navigate their relevant content. For example, if you're looking for a house or an apartment in the city center, then the system shouldn't recommend you houses in the suburbs. But if you're looking for a house in the suburbs, in the west, then it is possible that you can see the houses in the suburbs in the South. It is exactly human domain knowledge built into the system. Each type of recommender system has its own strength and weaknesses. In real life, people use hybrid recommender systems which gives you power to combine the best functionality from all of them. In the slide, you see a common choice for different domains. Where you have several types of algorithms listed for one domain, hybrid recommender system will be a natural choice.
