Design Photo Sharing App

http://blog.gainlo.co/index.php/2016/03/01/system-design-interview-question-create-a-photo-sharing-app/

Photo-sharing service like Instagram, Flickr, where users can upload photos to share them with other users

Data Sharding

a. Partitioning based on UserID

find the shard number by UserID % 10

Issues with this partition scheme:

• How would we handle hot users?

• Some users upload a lot more photos, creating non-uniform distribution of storage

• What if we cannot store all pictures of a user on one shard?

• Unavailability of all of the user's data if that shard is down or higher latency if it's serving high load

b. Partitioning based on PhotoID

Generate unique PhotoID, and find a shard number through “PhotoID % 10”

How can we generate PhotoIDs?

One solution could be that we dedicate a separate database instance to generate auto-incrementing IDs.

Wouldn’t this key generating DB be a single point of failure?

Yes, it would be. A workaround for that could be defining two such databases with one generating even numbered IDs and the other odd numbered. For the MySQL, the following script can define such sequences:

KeyGeneratingServer1:
auto-increment-increment = 2
auto-increment-offset = 1

KeyGeneratingServer2:
auto-increment-increment = 2
auto-increment-offset = 2

Ranking and News Feed Generation

sorting/merging/ranking

Pre-generating the News Feed: (UserNewsFeed table)

We can have dedicated servers that are continuously generating users’ News Feeds and storing them in a ‘UserNewsFeed’ table. So whenever any user needs the latest photos for their News Feed, we will simply query this table and return the results to the user.