Design Photo Sharing App

Photo-sharing service like Instagram, Flickr, where users can upload photos to share them with other users

Data Sharding

a. Partitioning based on UserID

find the shard number by UserID % 10

Issues with this partition scheme:

  • How would we handle hot users?

  • Some users upload a lot more photos, creating non-uniform distribution of storage

  • What if we cannot store all pictures of a user on one shard?

  • Unavailability of all of the user's data if that shard is down or higher latency if it's serving high load

b. Partitioning based on PhotoID

Generate unique PhotoID, and find a shard number through “PhotoID % 10”

How can we generate PhotoIDs?

One solution could be that we dedicate a separate database instance to generate auto-incrementing IDs.

Wouldn’t this key generating DB be a single point of failure?

Yes, it would be. A workaround for that could be defining two such databases with one generating even numbered IDs and the other odd numbered. For the MySQL, the following script can define such sequences:

auto-increment-increment = 2
auto-increment-offset = 1

auto-increment-increment = 2
auto-increment-offset = 2

Ranking and News Feed Generation


Pre-generating the News Feed: (UserNewsFeed table)

We can have dedicated servers that are continuously generating users’ News Feeds and storing them in a ‘UserNewsFeedtable. So whenever any user needs the latest photos for their News Feed, we will simply query this table and return the results to the user.

Last updated