System Design
  • Introduction
  • System Design Process
  • System Design Systematic Approach
  • System Design Topics
  • System Design Interview Tips
  • Object Oriented Design
  • System Design Problems
    • Designing an API Rate Limiter
    • Design News Feed
    • Design Recommendation System
    • Design Photo Sharing App
    • Design Location Based App
    • Design Messenger App
    • Design Twitter
    • Design Uber Lyft
    • Design Surge Pricing
  • Architect's Toolbox
    • Cache Design
    • Database and Cache
    • Pull vs Poll
    • Geo Location
    • Storage Estimation
    • ID Generator
    • Latency Numbers
    • Encoding Decoding Encryption Decryption
  • Systems Design Glossary
    • Consistent Hashing
    • Sharding or Partitioning
    • Database Indexes
    • Proxies
    • Caching
    • Queues
    • SQL vs. NoSQL
    • CAP Theorem
    • Distributed Messaging System
    • Long-Polling vs WebSockets vs Server-Sent Events
    • Producer and Consumer
    • Latency, Bandwidth and Throughput
    • Microservices Architecture
    • RESTful API
    • Concurrent Programming
  • Distributed System Resources
    • Distributed System Notes
  • Reference
Powered by GitBook
On this page

Was this helpful?

Last updated 4 years ago

Photo-sharing service like Instagram, Flickr, where users can upload photos to share them with other users

Data Sharding

a. Partitioning based on UserID

find the shard number by UserID % 10

Issues with this partition scheme:

  • How would we handle hot users?

  • Some users upload a lot more photos, creating non-uniform distribution of storage

  • What if we cannot store all pictures of a user on one shard?

  • Unavailability of all of the user's data if that shard is down or higher latency if it's serving high load

b. Partitioning based on PhotoID

Generate unique PhotoID, and find a shard number through “PhotoID % 10”

How can we generate PhotoIDs?

One solution could be that we dedicate a separate database instance to generate auto-incrementing IDs.

Wouldn’t this key generating DB be a single point of failure?

Yes, it would be. A workaround for that could be defining two such databases with one generating even numbered IDs and the other odd numbered. For the MySQL, the following script can define such sequences:

Ranking and News Feed Generation

sorting/merging/ranking

Pre-generating the News Feed: (UserNewsFeed table)

We can have dedicated servers that are continuously generating users’ News Feeds and storing them in a ‘UserNewsFeed’ table. So whenever any user needs the latest photos for their News Feed, we will simply query this table and return the results to the user.

http://blog.gainlo.co/index.php/2016/03/01/system-design-interview-question-create-a-photo-sharing-app/
KeyGeneratingServer1:
auto-increment-increment = 2
auto-increment-offset = 1

KeyGeneratingServer2:
auto-increment-increment = 2
auto-increment-offset = 2
  • Data Sharding
  • Ranking and News Feed Generation
PreviousDesign Recommendation SystemNextDesign Location Based App
  1. System Design Problems

Design Photo Sharing App