Design Recommendation System

Heuristic Solution

In fact, there are lots of hacks we can do to build a simple recommendation system. For instance, based on videos a user has watched, we can simply suggest videos from same authors. We can also suggest videos with similar titles or labels. If we use the popularity (number of comments, shares) as another signal, the recommendation system can work pretty well as a baseline.

Collaborative filtering

One can hardly avoid mentioning collaborative filtering (CF), which is the most popular technique used in recommendation systems.

user-based collaborative filtering

item-based collaborative filtering

Feature engineering

Q: what features can be used to build the recommendation system?

Example: Youtube Recommendation System

• Like/share/subscribe – As mentioned above, they are strong signs about a user’s preferences.

• Watch time

• Video title/labels/categories

• Freshness

Infrastructure

Offline

In fact, for most machine learning systems, it’s common to use offline pipeline to process big data as you won’t expect it to finish with few seconds.

Online

Fetches more than needed and do filtering, ranking on the fly

Reference

Gainlo Blog: Design a Recommendation System

Wikipedia: Collaborative Filtering