|
@@ -159,6 +159,10 @@ Review the [Contributing Guidelines](https://github.com/donnemartin/system-desig
|
|
* [Appendix](#appendix)
|
|
* [Appendix](#appendix)
|
|
* [Powers of two table](#powers-of-two-table)
|
|
* [Powers of two table](#powers-of-two-table)
|
|
* [Latency numbers every programmer should know](#latency-numbers-every-programmer-should-know)
|
|
* [Latency numbers every programmer should know](#latency-numbers-every-programmer-should-know)
|
|
|
|
+ * [Additional system design interview questions](#additional-system-design-interview-questions)
|
|
|
|
+ * [Real world architectures](#real-world-architectures)
|
|
|
|
+ * [Company architectures](#company-architectures)
|
|
|
|
+ * [Company engineering blogs](#company-engineering-blogs)
|
|
* [Under development](#under-development)
|
|
* [Under development](#under-development)
|
|
* [Credits](#credits)
|
|
* [Credits](#credits)
|
|
* [Contact info](#contact-info)
|
|
* [Contact info](#contact-info)
|
|
@@ -339,154 +343,6 @@ Check out the following links to get a better idea of what to expect:
|
|
| Design a circular array | [Contribute](#contributing) |
|
|
| Design a circular array | [Contribute](#contributing) |
|
|
| Add an object-oriented design question | [Contribute](#contributing) |
|
|
| Add an object-oriented design question | [Contribute](#contributing) |
|
|
|
|
|
|
-## Additional system design interview questions
|
|
|
|
-
|
|
|
|
-> Common system design interview questions, with links to resources on how to solve each.
|
|
|
|
-
|
|
|
|
-| Question | Reference(s) |
|
|
|
|
-|---|---|
|
|
|
|
-| Design a file sync service like Dropbox | [youtube.com](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
|
|
|
-| Design a search engine like Google | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
|
|
|
-| Design a scalable web crawler like Google | [quora.com](https://www.quora.com/How-can-I-build-a-web-crawler-from-scratch) |
|
|
|
|
-| Design Google docs | [code.google.com](https://code.google.com/p/google-mobwrite/)<br/>[neil.fraser.name](https://neil.fraser.name/writing/sync/) |
|
|
|
|
-| Design a key-value store like Redis | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
|
|
|
-| Design a cache system like Memcached | [slideshare.net](http://www.slideshare.net/oemebamo/introduction-to-memcached) |
|
|
|
|
-| Design a recommendation system like Amazon's | [hulu.com](http://tech.hulu.com/blog/2011/09/19/recommendation-system.html)<br/>[ijcai13.org](http://ijcai13.org/files/tutorial_slides/td3.pdf) |
|
|
|
|
-| Design a tinyurl system like Bitly | [n00tc0d3r.blogspot.com](http://n00tc0d3r.blogspot.com/) |
|
|
|
|
-| Design a chat app like WhatsApp | [highscalability.com](http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html)
|
|
|
|
-| Design a picture sharing system like Instagram | [highscalability.com](http://highscalability.com/flickr-architecture)<br/>[highscalability.com](http://highscalability.com/blog/2011/12/6/instagram-architecture-14-million-users-terabytes-of-photos.html) |
|
|
|
|
-| Design the Facebook news feed function | [quora.com](http://www.quora.com/What-are-best-practices-for-building-something-like-a-News-Feed)<br/>[quora.com](http://www.quora.com/Activity-Streams/What-are-the-scaling-issues-to-keep-in-mind-while-developing-a-social-network-feed)<br/>[slideshare.net](http://www.slideshare.net/danmckinley/etsy-activity-feeds-architecture) |
|
|
|
|
-| Design the Facebook timeline function | [facebook.com](https://www.facebook.com/note.php?note_id=10150468255628920)<br/>[highscalability.com](http://highscalability.com/blog/2012/1/23/facebook-timeline-brought-to-you-by-the-power-of-denormaliza.html) |
|
|
|
|
-| Design the Facebook chat function | [erlang-factory.com](http://www.erlang-factory.com/upload/presentations/31/EugeneLetuchy-ErlangatFacebook.pdf)<br/>[facebook.com](https://www.facebook.com/note.php?note_id=14218138919&id=9445547199&index=0) |
|
|
|
|
-| Design a graph search function like Facebook's | [facebook.com](https://www.facebook.com/notes/facebook-engineering/under-the-hood-building-out-the-infrastructure-for-graph-search/10151347573598920)<br/>[facebook.com](https://www.facebook.com/notes/facebook-engineering/under-the-hood-indexing-and-ranking-in-graph-search/10151361720763920)<br/>[facebook.com](https://www.facebook.com/notes/facebook-engineering/under-the-hood-the-natural-language-interface-of-graph-search/10151432733048920) |
|
|
|
|
-| Design a content delivery network like CloudFlare | [cmu.edu](http://repository.cmu.edu/cgi/viewcontent.cgi?article=2112&context=compsci) |
|
|
|
|
-| Design a trending topic system like Twitter's | [michael-noll.com](http://www.michael-noll.com/blog/2013/01/18/implementing-real-time-trending-topics-in-storm/)<br/>[snikolov .wordpress.com](http://snikolov.wordpress.com/2012/11/14/early-detection-of-twitter-trends/) |
|
|
|
|
-| Design a random ID generation system | [blog.twitter.com](https://blog.twitter.com/2010/announcing-snowflake)<br/>[github.com](https://github.com/twitter/snowflake/) |
|
|
|
|
-| Return the top k requests during a time interval | [ucsb.edu](https://icmi.cs.ucsb.edu/research/tech_reports/reports/2005-23.pdf)<br/>[wpi.edu](http://davis.wpi.edu/xmdv/docs/EDBT11-diyang.pdf) |
|
|
|
|
-| Design a system that serves data from multiple data centers | [highscalability.com](http://highscalability.com/blog/2009/8/24/how-google-serves-data-from-multiple-datacenters.html) |
|
|
|
|
-| Design an online multiplayer card game | [indieflashblog.com](http://www.indieflashblog.com/how-to-create-an-asynchronous-multiplayer-game.html)<br/>[buildnewgames.com](http://buildnewgames.com/real-time-multiplayer/) |
|
|
|
|
-| Design a garbage collection system | [stuffwithstuff.com](http://journal.stuffwithstuff.com/2013/12/08/babys-first-garbage-collector/)<br/>[washington.edu](http://courses.cs.washington.edu/courses/csep521/07wi/prj/rick.pdf) |
|
|
|
|
-| Add a system design question | [Contribute](#contributing) |
|
|
|
|
-
|
|
|
|
-## Real world architectures
|
|
|
|
-
|
|
|
|
-> Articles on how real world systems are designed.
|
|
|
|
-
|
|
|
|
-<p align="center">
|
|
|
|
- <img src="http://i.imgur.com/TcUo2fw.png">
|
|
|
|
- <br/>
|
|
|
|
- <i><a href=https://www.infoq.com/presentations/Twitter-Timeline-Scalability>Source: Twitter timelines at scale</a></i>
|
|
|
|
-</p>
|
|
|
|
-
|
|
|
|
-**Don't focus on nitty gritty details for the following articles, instead:**
|
|
|
|
-
|
|
|
|
-* Identify shared principles, common technologies, and patterns within these articles
|
|
|
|
-* Study what problems are solved by each component, where it works, where it doesn't
|
|
|
|
-* Review the lessons learned
|
|
|
|
-
|
|
|
|
-|Type | System | Reference(s) |
|
|
|
|
-|---|---|---|
|
|
|
|
-| Data processing | **MapReduce** - Distributed data processing from Google | [research.google.com](http://static.googleusercontent.com/media/research.google.com/zh-CN/us/archive/mapreduce-osdi04.pdf) |
|
|
|
|
-| Data processing | **Spark** - Distributed data processing from Databricks | [slideshare.net](http://www.slideshare.net/AGrishchenko/apache-spark-architecture) |
|
|
|
|
-| Data processing | **Storm** - Distributed data processing from Twitter | [slideshare.net](http://www.slideshare.net/previa/storm-16094009) |
|
|
|
|
-| | | |
|
|
|
|
-| Data store | **Bigtable** - Distributed column-oriented database from Google | [harvard.edu](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/chang06bigtable.pdf) |
|
|
|
|
-| Data store | **HBase** - Open source implementation of Bigtable | [slideshare.net](http://www.slideshare.net/alexbaranau/intro-to-hbase) |
|
|
|
|
-| Data store | **Cassandra** - Distributed column-oriented database from Facebook | [slideshare.net](http://www.slideshare.net/planetcassandra/cassandra-introduction-features-30103666)
|
|
|
|
-| Data store | **DynamoDB** - Document-oriented database from Amazon | [harvard.edu](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/decandia07dynamo.pdf) |
|
|
|
|
-| Data store | **MongoDB** - Document-oriented database | [slideshare.net](http://www.slideshare.net/mdirolf/introduction-to-mongodb) |
|
|
|
|
-| Data store | **Spanner** - Globally-distributed database from Google | [research.google.com](http://research.google.com/archive/spanner-osdi2012.pdf) |
|
|
|
|
-| Data store | **Memcached** - Distributed memory caching system | [slideshare.net](http://www.slideshare.net/oemebamo/introduction-to-memcached) |
|
|
|
|
-| Data store | **Redis** - Distributed memory caching system with persistence and value types | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
|
|
|
-| | | |
|
|
|
|
-| File system | **Google File System (GFS)** - Distributed file system | [research.google.com](http://static.googleusercontent.com/media/research.google.com/zh-CN/us/archive/gfs-sosp2003.pdf) |
|
|
|
|
-| File system | **Hadoop File System (HDFS)** - Open source implementation of GFS | [apache.org](https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html) |
|
|
|
|
-| | | |
|
|
|
|
-| Misc | **Chubby** - Lock service for loosely-coupled distributed systems from Google | [research.google.com](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/chubby-osdi06.pdf) |
|
|
|
|
-| Misc | **Dapper** - Distributed systems tracing infrastructure | [research.google.com](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36356.pdf)
|
|
|
|
-| Misc | **Kafka** - Pub/sub message queue from LinkedIn | [slideshare.net](http://www.slideshare.net/mumrah/kafka-talk-tri-hug) |
|
|
|
|
-| Misc | **Zookeeper** - Centralized infrastructure and services enabling synchronization | [slideshare.net](http://www.slideshare.net/sauravhaloi/introduction-to-apache-zookeeper) |
|
|
|
|
-| | Add an architecture | [Contribute](#contributing) |
|
|
|
|
-
|
|
|
|
-### Company architectures
|
|
|
|
-
|
|
|
|
-| Company | Reference(s) |
|
|
|
|
-|---|---|
|
|
|
|
-| Amazon | [Amazon architecture](http://highscalability.com/amazon-architecture) |
|
|
|
|
-| Cinchcast | [Producing 1,500 hours of audio every day](http://highscalability.com/blog/2012/7/16/cinchcast-architecture-producing-1500-hours-of-audio-every-d.html) |
|
|
|
|
-| DataSift | [Realtime datamining At 120,000 tweets per second](http://highscalability.com/blog/2011/11/29/datasift-architecture-realtime-datamining-at-120000-tweets-p.html) |
|
|
|
|
-| DropBox | [How we've scaled Dropbox](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
|
|
|
-| ESPN | [Operating At 100,000 duh nuh nuhs per second](http://highscalability.com/blog/2013/11/4/espns-architecture-at-scale-operating-at-100000-duh-nuh-nuhs.html) |
|
|
|
|
-| Google | [Google architecture](http://highscalability.com/google-architecture) |
|
|
|
|
-| Instagram | [14 million users, terabytes of photos](http://highscalability.com/blog/2011/12/6/instagram-architecture-14-million-users-terabytes-of-photos.html)<br/>[What powers Instagram](http://instagram-engineering.tumblr.com/post/13649370142/what-powers-instagram-hundreds-of-instances) |
|
|
|
|
-| Justin.tv | [Justin.Tv's live video broadcasting architecture](http://highscalability.com/blog/2010/3/16/justintvs-live-video-broadcasting-architecture.html) |
|
|
|
|
-| Facebook | [Scaling memcached at Facebook](https://cs.uwaterloo.ca/~brecht/courses/854-Emerging-2014/readings/key-value/fb-memcached-nsdi-2013.pdf)<br/>[TAO: Facebook’s distributed data store for the social graph](https://cs.uwaterloo.ca/~brecht/courses/854-Emerging-2014/readings/data-store/tao-facebook-distributed-datastore-atc-2013.pdf)<br/>[Facebook’s photo storage](https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf) |
|
|
|
|
-| Flickr | [Flickr architecture](http://highscalability.com/flickr-architecture) |
|
|
|
|
-| Mailbox | [From 0 to one million users in 6 weeks](http://highscalability.com/blog/2013/6/18/scaling-mailbox-from-0-to-one-million-users-in-6-weeks-and-1.html) |
|
|
|
|
-| Pinterest | [From 0 To 10s of billions of page views a month](http://highscalability.com/blog/2013/4/15/scaling-pinterest-from-0-to-10s-of-billions-of-page-views-a.html)<br/>[18 million visitors, 10x growth, 12 employees](http://highscalability.com/blog/2012/5/21/pinterest-architecture-update-18-million-visitors-10x-growth.html) |
|
|
|
|
-| Playfish | [50 million monthly users and growing](http://highscalability.com/blog/2010/9/21/playfishs-social-gaming-architecture-50-million-monthly-user.html) |
|
|
|
|
-| PlentyOfFish | [PlentyOfFish architecture](http://highscalability.com/plentyoffish-architecture) |
|
|
|
|
-| Salesforce | [How they handle 1.3 billion transactions a day](http://highscalability.com/blog/2013/9/23/salesforce-architecture-how-they-handle-13-billion-transacti.html) |
|
|
|
|
-| Stack Overflow | [Stack Overflow architecture](http://highscalability.com/blog/2009/8/5/stack-overflow-architecture.html) |
|
|
|
|
-| TripAdvisor | [40M visitors, 200M dynamic page views, 30TB data](http://highscalability.com/blog/2011/6/27/tripadvisor-architecture-40m-visitors-200m-dynamic-page-view.html) |
|
|
|
|
-| Tumblr | [15 billion page views a month](http://highscalability.com/blog/2012/2/13/tumblr-architecture-15-billion-page-views-a-month-and-harder.html) |
|
|
|
|
-| Twitter | [Making Twitter 10000 percent faster](http://highscalability.com/scaling-twitter-making-twitter-10000-percent-faster)<br/>[Storing 250 million tweets a day using MySQL](http://highscalability.com/blog/2011/12/19/how-twitter-stores-250-million-tweets-a-day-using-mysql.html)<br/>[150M active users, 300K QPS, a 22 MB/S firehose](http://highscalability.com/blog/2013/7/8/the-architecture-twitter-uses-to-deal-with-150m-active-users.html)<br/>[Timelines at scale](https://www.infoq.com/presentations/Twitter-Timeline-Scalability)<br/>[Big and small data at Twitter](https://www.youtube.com/watch?v=5cKTP36HVgI)<br/>[Operations at Twitter: scaling beyond 100 million users](https://www.youtube.com/watch?v=z8LU0Cj6BOU) |
|
|
|
|
-| Uber | [How Uber scales their real-time market platform](http://highscalability.com/blog/2015/9/14/how-uber-scales-their-real-time-market-platform.html) |
|
|
|
|
-| WhatsApp | [The WhatsApp architecture Facebook bought for $19 billion](http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html) |
|
|
|
|
-| YouTube | [YouTube scalability](https://www.youtube.com/watch?v=w5WVu624fY8)<br/>[YouTube architecture](http://highscalability.com/youtube-architecture) |
|
|
|
|
-
|
|
|
|
-## Company engineering blogs
|
|
|
|
-
|
|
|
|
-> Architectures for companies you are interviewing with.
|
|
|
|
->
|
|
|
|
-> Questions you encounter might be from the same domain.
|
|
|
|
-
|
|
|
|
-* [Airbnb Engineering](http://nerds.airbnb.com/)
|
|
|
|
-* [Atlassian Developers](https://developer.atlassian.com/blog/)
|
|
|
|
-* [Autodesk Engineering](http://cloudengineering.autodesk.com/blog/)
|
|
|
|
-* [AWS Blog](https://aws.amazon.com/blogs/aws/)
|
|
|
|
-* [Bitly Engineering Blog](http://word.bitly.com/)
|
|
|
|
-* [Box Blogs](https://www.box.com/blog/engineering/)
|
|
|
|
-* [Cloudera Developer Blog](http://blog.cloudera.com/blog/)
|
|
|
|
-* [Dropbox Tech Blog](https://tech.dropbox.com/)
|
|
|
|
-* [Engineering at Quora](http://engineering.quora.com/)
|
|
|
|
-* [Ebay Tech Blog](http://www.ebaytechblog.com/)
|
|
|
|
-* [Evernote Tech Blog](https://blog.evernote.com/tech/)
|
|
|
|
-* [Etsy Code as Craft](http://codeascraft.com/)
|
|
|
|
-* [Facebook Engineering](https://www.facebook.com/Engineering)
|
|
|
|
-* [Flickr Code](http://code.flickr.net/)
|
|
|
|
-* [Foursquare Engineering Blog](http://engineering.foursquare.com/)
|
|
|
|
-* [GitHub Engineering Blog](http://githubengineering.com/)
|
|
|
|
-* [Google Research Blog](http://googleresearch.blogspot.com/)
|
|
|
|
-* [Groupon Engineering Blog](https://engineering.groupon.com/)
|
|
|
|
-* [Heroku Engineering Blog](https://engineering.heroku.com/)
|
|
|
|
-* [Hubspot Engineering Blog](http://product.hubspot.com/blog/topic/engineering)
|
|
|
|
-* [High Scalability](http://highscalability.com/)
|
|
|
|
-* [Instagram Engineering](http://instagram-engineering.tumblr.com/)
|
|
|
|
-* [Intel Software Blog](https://software.intel.com/en-us/blogs/)
|
|
|
|
-* [Jane Street Tech Blog](https://blogs.janestreet.com/category/ocaml/)
|
|
|
|
-* [LinkedIn Engineering](http://engineering.linkedin.com/blog)
|
|
|
|
-* [Microsoft Engineering](https://engineering.microsoft.com/)
|
|
|
|
-* [Microsoft Python Engineering](https://blogs.msdn.microsoft.com/pythonengineering/)
|
|
|
|
-* [Netflix Tech Blog](http://techblog.netflix.com/)
|
|
|
|
-* [Paypal Developer Blog](https://devblog.paypal.com/category/engineering/)
|
|
|
|
-* [Pinterest Engineering Blog](http://engineering.pinterest.com/)
|
|
|
|
-* [Quora Engineering](https://engineering.quora.com/)
|
|
|
|
-* [Reddit Blog](http://www.redditblog.com/)
|
|
|
|
-* [Salesforce Engineering Blog](https://developer.salesforce.com/blogs/engineering/)
|
|
|
|
-* [Slack Engineering Blog](https://slack.engineering/)
|
|
|
|
-* [Spotify Labs](https://labs.spotify.com/)
|
|
|
|
-* [Twilio Engineering Blog](http://www.twilio.com/engineering)
|
|
|
|
-* [Twitter Engineering](https://engineering.twitter.com/)
|
|
|
|
-* [Uber Engineering Blog](http://eng.uber.com/)
|
|
|
|
-* [Yahoo Engineering Blog](http://yahooeng.tumblr.com/)
|
|
|
|
-* [Yelp Engineering Blog](http://engineeringblog.yelp.com/)
|
|
|
|
-* [Zynga Engineering Blog](https://www.zynga.com/blogs/engineering)
|
|
|
|
-
|
|
|
|
-### Source(s) and further reading
|
|
|
|
-
|
|
|
|
-* [kilimchoi/engineering-blogs](https://github.com/kilimchoi/engineering-blogs)
|
|
|
|
-
|
|
|
|
## System design topics: start here
|
|
## System design topics: start here
|
|
|
|
|
|
New to system design?
|
|
New to system design?
|
|
@@ -1711,6 +1567,154 @@ Handy metrics based on numbers above:
|
|
* [Designs, lessons, and advice from building large distributed systems](http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf)
|
|
* [Designs, lessons, and advice from building large distributed systems](http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf)
|
|
* [Software Engineering Advice from Building Large-Scale Distributed Systems](https://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf)
|
|
* [Software Engineering Advice from Building Large-Scale Distributed Systems](https://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf)
|
|
|
|
|
|
|
|
+### Additional system design interview questions
|
|
|
|
+
|
|
|
|
+> Common system design interview questions, with links to resources on how to solve each.
|
|
|
|
+
|
|
|
|
+| Question | Reference(s) |
|
|
|
|
+|---|---|
|
|
|
|
+| Design a file sync service like Dropbox | [youtube.com](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
|
|
|
+| Design a search engine like Google | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
|
|
|
+| Design a scalable web crawler like Google | [quora.com](https://www.quora.com/How-can-I-build-a-web-crawler-from-scratch) |
|
|
|
|
+| Design Google docs | [code.google.com](https://code.google.com/p/google-mobwrite/)<br/>[neil.fraser.name](https://neil.fraser.name/writing/sync/) |
|
|
|
|
+| Design a key-value store like Redis | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
|
|
|
+| Design a cache system like Memcached | [slideshare.net](http://www.slideshare.net/oemebamo/introduction-to-memcached) |
|
|
|
|
+| Design a recommendation system like Amazon's | [hulu.com](http://tech.hulu.com/blog/2011/09/19/recommendation-system.html)<br/>[ijcai13.org](http://ijcai13.org/files/tutorial_slides/td3.pdf) |
|
|
|
|
+| Design a tinyurl system like Bitly | [n00tc0d3r.blogspot.com](http://n00tc0d3r.blogspot.com/) |
|
|
|
|
+| Design a chat app like WhatsApp | [highscalability.com](http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html)
|
|
|
|
+| Design a picture sharing system like Instagram | [highscalability.com](http://highscalability.com/flickr-architecture)<br/>[highscalability.com](http://highscalability.com/blog/2011/12/6/instagram-architecture-14-million-users-terabytes-of-photos.html) |
|
|
|
|
+| Design the Facebook news feed function | [quora.com](http://www.quora.com/What-are-best-practices-for-building-something-like-a-News-Feed)<br/>[quora.com](http://www.quora.com/Activity-Streams/What-are-the-scaling-issues-to-keep-in-mind-while-developing-a-social-network-feed)<br/>[slideshare.net](http://www.slideshare.net/danmckinley/etsy-activity-feeds-architecture) |
|
|
|
|
+| Design the Facebook timeline function | [facebook.com](https://www.facebook.com/note.php?note_id=10150468255628920)<br/>[highscalability.com](http://highscalability.com/blog/2012/1/23/facebook-timeline-brought-to-you-by-the-power-of-denormaliza.html) |
|
|
|
|
+| Design the Facebook chat function | [erlang-factory.com](http://www.erlang-factory.com/upload/presentations/31/EugeneLetuchy-ErlangatFacebook.pdf)<br/>[facebook.com](https://www.facebook.com/note.php?note_id=14218138919&id=9445547199&index=0) |
|
|
|
|
+| Design a graph search function like Facebook's | [facebook.com](https://www.facebook.com/notes/facebook-engineering/under-the-hood-building-out-the-infrastructure-for-graph-search/10151347573598920)<br/>[facebook.com](https://www.facebook.com/notes/facebook-engineering/under-the-hood-indexing-and-ranking-in-graph-search/10151361720763920)<br/>[facebook.com](https://www.facebook.com/notes/facebook-engineering/under-the-hood-the-natural-language-interface-of-graph-search/10151432733048920) |
|
|
|
|
+| Design a content delivery network like CloudFlare | [cmu.edu](http://repository.cmu.edu/cgi/viewcontent.cgi?article=2112&context=compsci) |
|
|
|
|
+| Design a trending topic system like Twitter's | [michael-noll.com](http://www.michael-noll.com/blog/2013/01/18/implementing-real-time-trending-topics-in-storm/)<br/>[snikolov .wordpress.com](http://snikolov.wordpress.com/2012/11/14/early-detection-of-twitter-trends/) |
|
|
|
|
+| Design a random ID generation system | [blog.twitter.com](https://blog.twitter.com/2010/announcing-snowflake)<br/>[github.com](https://github.com/twitter/snowflake/) |
|
|
|
|
+| Return the top k requests during a time interval | [ucsb.edu](https://icmi.cs.ucsb.edu/research/tech_reports/reports/2005-23.pdf)<br/>[wpi.edu](http://davis.wpi.edu/xmdv/docs/EDBT11-diyang.pdf) |
|
|
|
|
+| Design a system that serves data from multiple data centers | [highscalability.com](http://highscalability.com/blog/2009/8/24/how-google-serves-data-from-multiple-datacenters.html) |
|
|
|
|
+| Design an online multiplayer card game | [indieflashblog.com](http://www.indieflashblog.com/how-to-create-an-asynchronous-multiplayer-game.html)<br/>[buildnewgames.com](http://buildnewgames.com/real-time-multiplayer/) |
|
|
|
|
+| Design a garbage collection system | [stuffwithstuff.com](http://journal.stuffwithstuff.com/2013/12/08/babys-first-garbage-collector/)<br/>[washington.edu](http://courses.cs.washington.edu/courses/csep521/07wi/prj/rick.pdf) |
|
|
|
|
+| Add a system design question | [Contribute](#contributing) |
|
|
|
|
+
|
|
|
|
+### Real world architectures
|
|
|
|
+
|
|
|
|
+> Articles on how real world systems are designed.
|
|
|
|
+
|
|
|
|
+<p align="center">
|
|
|
|
+ <img src="http://i.imgur.com/TcUo2fw.png">
|
|
|
|
+ <br/>
|
|
|
|
+ <i><a href=https://www.infoq.com/presentations/Twitter-Timeline-Scalability>Source: Twitter timelines at scale</a></i>
|
|
|
|
+</p>
|
|
|
|
+
|
|
|
|
+**Don't focus on nitty gritty details for the following articles, instead:**
|
|
|
|
+
|
|
|
|
+* Identify shared principles, common technologies, and patterns within these articles
|
|
|
|
+* Study what problems are solved by each component, where it works, where it doesn't
|
|
|
|
+* Review the lessons learned
|
|
|
|
+
|
|
|
|
+|Type | System | Reference(s) |
|
|
|
|
+|---|---|---|
|
|
|
|
+| Data processing | **MapReduce** - Distributed data processing from Google | [research.google.com](http://static.googleusercontent.com/media/research.google.com/zh-CN/us/archive/mapreduce-osdi04.pdf) |
|
|
|
|
+| Data processing | **Spark** - Distributed data processing from Databricks | [slideshare.net](http://www.slideshare.net/AGrishchenko/apache-spark-architecture) |
|
|
|
|
+| Data processing | **Storm** - Distributed data processing from Twitter | [slideshare.net](http://www.slideshare.net/previa/storm-16094009) |
|
|
|
|
+| | | |
|
|
|
|
+| Data store | **Bigtable** - Distributed column-oriented database from Google | [harvard.edu](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/chang06bigtable.pdf) |
|
|
|
|
+| Data store | **HBase** - Open source implementation of Bigtable | [slideshare.net](http://www.slideshare.net/alexbaranau/intro-to-hbase) |
|
|
|
|
+| Data store | **Cassandra** - Distributed column-oriented database from Facebook | [slideshare.net](http://www.slideshare.net/planetcassandra/cassandra-introduction-features-30103666)
|
|
|
|
+| Data store | **DynamoDB** - Document-oriented database from Amazon | [harvard.edu](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/decandia07dynamo.pdf) |
|
|
|
|
+| Data store | **MongoDB** - Document-oriented database | [slideshare.net](http://www.slideshare.net/mdirolf/introduction-to-mongodb) |
|
|
|
|
+| Data store | **Spanner** - Globally-distributed database from Google | [research.google.com](http://research.google.com/archive/spanner-osdi2012.pdf) |
|
|
|
|
+| Data store | **Memcached** - Distributed memory caching system | [slideshare.net](http://www.slideshare.net/oemebamo/introduction-to-memcached) |
|
|
|
|
+| Data store | **Redis** - Distributed memory caching system with persistence and value types | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
|
|
|
+| | | |
|
|
|
|
+| File system | **Google File System (GFS)** - Distributed file system | [research.google.com](http://static.googleusercontent.com/media/research.google.com/zh-CN/us/archive/gfs-sosp2003.pdf) |
|
|
|
|
+| File system | **Hadoop File System (HDFS)** - Open source implementation of GFS | [apache.org](https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html) |
|
|
|
|
+| | | |
|
|
|
|
+| Misc | **Chubby** - Lock service for loosely-coupled distributed systems from Google | [research.google.com](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/chubby-osdi06.pdf) |
|
|
|
|
+| Misc | **Dapper** - Distributed systems tracing infrastructure | [research.google.com](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36356.pdf)
|
|
|
|
+| Misc | **Kafka** - Pub/sub message queue from LinkedIn | [slideshare.net](http://www.slideshare.net/mumrah/kafka-talk-tri-hug) |
|
|
|
|
+| Misc | **Zookeeper** - Centralized infrastructure and services enabling synchronization | [slideshare.net](http://www.slideshare.net/sauravhaloi/introduction-to-apache-zookeeper) |
|
|
|
|
+| | Add an architecture | [Contribute](#contributing) |
|
|
|
|
+
|
|
|
|
+### Company architectures
|
|
|
|
+
|
|
|
|
+| Company | Reference(s) |
|
|
|
|
+|---|---|
|
|
|
|
+| Amazon | [Amazon architecture](http://highscalability.com/amazon-architecture) |
|
|
|
|
+| Cinchcast | [Producing 1,500 hours of audio every day](http://highscalability.com/blog/2012/7/16/cinchcast-architecture-producing-1500-hours-of-audio-every-d.html) |
|
|
|
|
+| DataSift | [Realtime datamining At 120,000 tweets per second](http://highscalability.com/blog/2011/11/29/datasift-architecture-realtime-datamining-at-120000-tweets-p.html) |
|
|
|
|
+| DropBox | [How we've scaled Dropbox](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
|
|
|
+| ESPN | [Operating At 100,000 duh nuh nuhs per second](http://highscalability.com/blog/2013/11/4/espns-architecture-at-scale-operating-at-100000-duh-nuh-nuhs.html) |
|
|
|
|
+| Google | [Google architecture](http://highscalability.com/google-architecture) |
|
|
|
|
+| Instagram | [14 million users, terabytes of photos](http://highscalability.com/blog/2011/12/6/instagram-architecture-14-million-users-terabytes-of-photos.html)<br/>[What powers Instagram](http://instagram-engineering.tumblr.com/post/13649370142/what-powers-instagram-hundreds-of-instances) |
|
|
|
|
+| Justin.tv | [Justin.Tv's live video broadcasting architecture](http://highscalability.com/blog/2010/3/16/justintvs-live-video-broadcasting-architecture.html) |
|
|
|
|
+| Facebook | [Scaling memcached at Facebook](https://cs.uwaterloo.ca/~brecht/courses/854-Emerging-2014/readings/key-value/fb-memcached-nsdi-2013.pdf)<br/>[TAO: Facebook’s distributed data store for the social graph](https://cs.uwaterloo.ca/~brecht/courses/854-Emerging-2014/readings/data-store/tao-facebook-distributed-datastore-atc-2013.pdf)<br/>[Facebook’s photo storage](https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf) |
|
|
|
|
+| Flickr | [Flickr architecture](http://highscalability.com/flickr-architecture) |
|
|
|
|
+| Mailbox | [From 0 to one million users in 6 weeks](http://highscalability.com/blog/2013/6/18/scaling-mailbox-from-0-to-one-million-users-in-6-weeks-and-1.html) |
|
|
|
|
+| Pinterest | [From 0 To 10s of billions of page views a month](http://highscalability.com/blog/2013/4/15/scaling-pinterest-from-0-to-10s-of-billions-of-page-views-a.html)<br/>[18 million visitors, 10x growth, 12 employees](http://highscalability.com/blog/2012/5/21/pinterest-architecture-update-18-million-visitors-10x-growth.html) |
|
|
|
|
+| Playfish | [50 million monthly users and growing](http://highscalability.com/blog/2010/9/21/playfishs-social-gaming-architecture-50-million-monthly-user.html) |
|
|
|
|
+| PlentyOfFish | [PlentyOfFish architecture](http://highscalability.com/plentyoffish-architecture) |
|
|
|
|
+| Salesforce | [How they handle 1.3 billion transactions a day](http://highscalability.com/blog/2013/9/23/salesforce-architecture-how-they-handle-13-billion-transacti.html) |
|
|
|
|
+| Stack Overflow | [Stack Overflow architecture](http://highscalability.com/blog/2009/8/5/stack-overflow-architecture.html) |
|
|
|
|
+| TripAdvisor | [40M visitors, 200M dynamic page views, 30TB data](http://highscalability.com/blog/2011/6/27/tripadvisor-architecture-40m-visitors-200m-dynamic-page-view.html) |
|
|
|
|
+| Tumblr | [15 billion page views a month](http://highscalability.com/blog/2012/2/13/tumblr-architecture-15-billion-page-views-a-month-and-harder.html) |
|
|
|
|
+| Twitter | [Making Twitter 10000 percent faster](http://highscalability.com/scaling-twitter-making-twitter-10000-percent-faster)<br/>[Storing 250 million tweets a day using MySQL](http://highscalability.com/blog/2011/12/19/how-twitter-stores-250-million-tweets-a-day-using-mysql.html)<br/>[150M active users, 300K QPS, a 22 MB/S firehose](http://highscalability.com/blog/2013/7/8/the-architecture-twitter-uses-to-deal-with-150m-active-users.html)<br/>[Timelines at scale](https://www.infoq.com/presentations/Twitter-Timeline-Scalability)<br/>[Big and small data at Twitter](https://www.youtube.com/watch?v=5cKTP36HVgI)<br/>[Operations at Twitter: scaling beyond 100 million users](https://www.youtube.com/watch?v=z8LU0Cj6BOU) |
|
|
|
|
+| Uber | [How Uber scales their real-time market platform](http://highscalability.com/blog/2015/9/14/how-uber-scales-their-real-time-market-platform.html) |
|
|
|
|
+| WhatsApp | [The WhatsApp architecture Facebook bought for $19 billion](http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html) |
|
|
|
|
+| YouTube | [YouTube scalability](https://www.youtube.com/watch?v=w5WVu624fY8)<br/>[YouTube architecture](http://highscalability.com/youtube-architecture) |
|
|
|
|
+
|
|
|
|
+### Company engineering blogs
|
|
|
|
+
|
|
|
|
+> Architectures for companies you are interviewing with.
|
|
|
|
+>
|
|
|
|
+> Questions you encounter might be from the same domain.
|
|
|
|
+
|
|
|
|
+* [Airbnb Engineering](http://nerds.airbnb.com/)
|
|
|
|
+* [Atlassian Developers](https://developer.atlassian.com/blog/)
|
|
|
|
+* [Autodesk Engineering](http://cloudengineering.autodesk.com/blog/)
|
|
|
|
+* [AWS Blog](https://aws.amazon.com/blogs/aws/)
|
|
|
|
+* [Bitly Engineering Blog](http://word.bitly.com/)
|
|
|
|
+* [Box Blogs](https://www.box.com/blog/engineering/)
|
|
|
|
+* [Cloudera Developer Blog](http://blog.cloudera.com/blog/)
|
|
|
|
+* [Dropbox Tech Blog](https://tech.dropbox.com/)
|
|
|
|
+* [Engineering at Quora](http://engineering.quora.com/)
|
|
|
|
+* [Ebay Tech Blog](http://www.ebaytechblog.com/)
|
|
|
|
+* [Evernote Tech Blog](https://blog.evernote.com/tech/)
|
|
|
|
+* [Etsy Code as Craft](http://codeascraft.com/)
|
|
|
|
+* [Facebook Engineering](https://www.facebook.com/Engineering)
|
|
|
|
+* [Flickr Code](http://code.flickr.net/)
|
|
|
|
+* [Foursquare Engineering Blog](http://engineering.foursquare.com/)
|
|
|
|
+* [GitHub Engineering Blog](http://githubengineering.com/)
|
|
|
|
+* [Google Research Blog](http://googleresearch.blogspot.com/)
|
|
|
|
+* [Groupon Engineering Blog](https://engineering.groupon.com/)
|
|
|
|
+* [Heroku Engineering Blog](https://engineering.heroku.com/)
|
|
|
|
+* [Hubspot Engineering Blog](http://product.hubspot.com/blog/topic/engineering)
|
|
|
|
+* [High Scalability](http://highscalability.com/)
|
|
|
|
+* [Instagram Engineering](http://instagram-engineering.tumblr.com/)
|
|
|
|
+* [Intel Software Blog](https://software.intel.com/en-us/blogs/)
|
|
|
|
+* [Jane Street Tech Blog](https://blogs.janestreet.com/category/ocaml/)
|
|
|
|
+* [LinkedIn Engineering](http://engineering.linkedin.com/blog)
|
|
|
|
+* [Microsoft Engineering](https://engineering.microsoft.com/)
|
|
|
|
+* [Microsoft Python Engineering](https://blogs.msdn.microsoft.com/pythonengineering/)
|
|
|
|
+* [Netflix Tech Blog](http://techblog.netflix.com/)
|
|
|
|
+* [Paypal Developer Blog](https://devblog.paypal.com/category/engineering/)
|
|
|
|
+* [Pinterest Engineering Blog](http://engineering.pinterest.com/)
|
|
|
|
+* [Quora Engineering](https://engineering.quora.com/)
|
|
|
|
+* [Reddit Blog](http://www.redditblog.com/)
|
|
|
|
+* [Salesforce Engineering Blog](https://developer.salesforce.com/blogs/engineering/)
|
|
|
|
+* [Slack Engineering Blog](https://slack.engineering/)
|
|
|
|
+* [Spotify Labs](https://labs.spotify.com/)
|
|
|
|
+* [Twilio Engineering Blog](http://www.twilio.com/engineering)
|
|
|
|
+* [Twitter Engineering](https://engineering.twitter.com/)
|
|
|
|
+* [Uber Engineering Blog](http://eng.uber.com/)
|
|
|
|
+* [Yahoo Engineering Blog](http://yahooeng.tumblr.com/)
|
|
|
|
+* [Yelp Engineering Blog](http://engineeringblog.yelp.com/)
|
|
|
|
+* [Zynga Engineering Blog](https://www.zynga.com/blogs/engineering)
|
|
|
|
+
|
|
|
|
+#### Source(s) and further reading
|
|
|
|
+
|
|
|
|
+* [kilimchoi/engineering-blogs](https://github.com/kilimchoi/engineering-blogs)
|
|
|
|
+
|
|
## Under development
|
|
## Under development
|
|
|
|
|
|
Interested in adding a section or helping complete one in-progress? [Contribute](#contributing)!
|
|
Interested in adding a section or helping complete one in-progress? [Contribute](#contributing)!
|