mirror of
https://github.com/binhnguyennus/awesome-scalability.git
synced 2024-10-01 01:06:14 -04:00
63 KiB
63 KiB
High Scalability, High Availability, High Stability, High Performance, and High Intelligence Back-end Designs
An updated and curated list of selected readings to illustrate High Scalability, High Availability, High Stability, High Performance, and High Intelligence Back-end Designs. Concepts are explained in the articles of notable engineers (Werner Vogels, James Hamilton, Martin Kleppmann, etc) and credible references (highscalability.com, infoq.com, official engineering blogs). Case studies are taken from battle-tested systems those are serving millions to billions of users (Netflix, Alibaba, Flipkart, LINE, Spotify, etc).
What if your Back-end went slow?
Understand your problems: performance problem (slow for a single user) or scalability problem (fast for a single user but slow under heavy load) by reviewing design principles. You can also check some talks of elite engineers from tech giants (Google, Facebook, Instagram, etc) to see how they build and scale their systems.
What if your Back-end went down?
"Even if you lose all one day, you can build all over again if you retain your calm!" - Thuan Pham, CTO at Uber Technologies Inc.
For the future CTO of the next Uber :)
Checking out some interview notes and compeleted architectures to get a comprehensive view. Before designing Whatsapp or Twitter on whiteboard, you must understand thoroughly fundamental building blocks (IPC, OSI, TCP/IP, Gossip, etc). It is even better to take a course on Distributed Systems or Distributed Computing. Good luck!
Community Power
Contributions are greatly welcome! You may want to take a look at the contribution guidelines. If you find this project helpful, please help me share on Twitter or share_on_Weibo.
@Chinese friends 🇨🇳 : Can someone help me create a pull request with a Weibo share link at the "share_on_Weibo" position above? 🙌. Thank you very much 🙇
Contents
- Principles
- Scalability
- Availability
- Stability
- Performance
- Intelligence
- Architectures
- Ad-hoc
- Interview
- Talks
- Books
Principles
- On Efficiency, Reliability, Scaling - James Hamilton, VP at AWS
- Principles of Chaos Engineering
- Finding the Order in Chaos
- The Twelve-Factor App
- Clean Architecture
- High Cohesion and Low Coupling
- CAP Theorem and Trade-offs
- CP Databases and AP Databases
- Scale Up vs Scale Out
- Scale Up vs Scale Out: Hidden Costs
- Best Practices for Scaling Out
- ACID and BASE
- Blocking/Non-Blocking and Sync/Async
- Performance and Scalability of Databases
- Database Isolation Levels and Effects on Performance and Scalability
- SQL vs NoSQL
- SQL vs NoSQL - Lesson Learned from Salesforce
- How Sharding Works
- Consistent Hashing
- Uniform Consistent Hashing (used at Netflix)
- Eventually Consistent - Werner Vogels, CTO at Amazon
- Cache is King
- Anti-Caching
- Understand Latency
- Latency Numbers Every Programmer Should Know
- Architecture Issues When Scaling Web Applications: Bottlenecks, Database, CPU, IO
- Common Bottlenecks
- Life Beyond Distributed Transactions
- Relying on Software to Redirect Traffic Reliably at Various Layers
- Breaking Things on Purpose
- Avoid Over Engineering
- Scalability Worst Practices
- Use Solid Technologies - Don’t Re-invent the Wheel - Keep It Simple!
- Why Over-Reusing is Bad
- Performance is a Feature
- Make Performance Part of Your Workflow
- The Benefits of Server Side Rendering Over Client Side Rendering
- Writing Code that Scales
- Automate and Abstract: Lessons from Facebook on Engineering for Scale
- AWS Do's and Don'ts
- (UI) Design Doesn’t Scale - Stanley Wood, Design Director at Spotify
- Linux Performance
- How To Design A Good API and Why it Matters - Joshua Bloch
- Building Fast & Resilient Web Applications - Ilya Grigorik
- Design for Loose-coupling
- Design for Resiliency
- Design for Self-healing
- Design for Scaling Out
- Design for Evolution
- Learn from Mistakes
- Code Review Best Practices at Palantir
Scalability
- Microservices and Orchestration
- Microservices Resource Guide - Martin Fowler, Chief Scientist at ThoughtWorks
- Microservices Patterns
- Advantages and Drawbacks of Microservices
- Microservices Scale Cube
- Thinking Inside the Container (8 parts) at Riot Games
- Containerization at Pinterest
- Techniques for Splitting Up a Codebase into Microservices and Artifacts at LinkedIn
- The Evolution of Container Usage at Netflix
- Dockerizing MySQL at Uber
- Testing of Microservices at Spotify
- Organize Monolith Before Breaking it into Services at Weebly
- Lessons learned running Docker in production at Treehouse
- Inside a SoundCloud Microservice
- Microservices at BlaBlaCar
- Operate Kubernetes Reliably at Stripe
- Kubernetes Traffic Routing (2 parts) at Rakuten
- Agrarian-Scale Kubernetes (3 parts) at New York Times
- Mesos, Docker and Ochopod in Localization Services at Autodesk
- Nanoservices at BBC Online
- PowerfulSeal: Testing Tool for Kubernetes Clusters at Bloomberg
- Conductor: Microservices Orchestrator at Netflix
- Making 10x Improvement in Release Times with Docker and Amazon ECS at Nextdoor
- K8Guard: Auditing System for Kubernetes Clusters at Target.com
- Distributed Caching
- Read-Through, Write-Through, Write-Behind, and Refresh-Ahead Caching
- Eviction Policy and Expiration Policy
- EVCache: Caching for a Global Netflix
- Memsniff: Robust Memcache Traffic Analyzer at Box.com
- Caching with Consistent Hashing and Cache Smearing at Etsy
- Analysis of Photo Caching at Facebook
- Cache Efficiency Exercise at Facebook
- tCache: Scalable Data-aware Java Caching at Trivago
- Reduce Memcached Memory Usage by 50% at Trivago
- Caching Internal Service Calls at Yelp
- Distributed Tracking and Tracing
- Tracking Service Infrastructure at Scale at Shopify
- Distributed Tracing with Pintrace at Pinterest
- Analyzing Distributed Trace Data at Pinterest
- Distributed Tracing at Uber
- Data Checking at Dropbox
- Tracing Distributed Systems at Showmax
- Real-time Distributed Tracing at LinkedIn
- Zipkin: Distributed Systems Tracing at Twitter
- osquery Across the Enterprise at Palantir
- Distributed Logging
- The Problem with Logging - Jeff Atwood
- The Log: What Every Software Engineer Should Know
- Using Logs to Build a Solid Data Infrastructure - Martin Kleppmann
- Scalable and Reliable Log Ingestion at Pinterest
- Building DistributedLog at Twitter: High-performance replicated log service
- Logging Service with Spark at CERN Accelerator
- Logging and Aggregation at Quora
- BookKeeper: Distributed Log Storage at Yahoo
- LogDevice: Distributed Data Store for Logs at Facebook
- LogFeeder: Log Collection System at Yelp
- Distributed Messaging and Event Streaming
- When to use RabbitMQ or Kafka
- Should You Put Several Event Types in the Same Kafka Topic? - Martin Kleppmann
- Kafka at Scale at Linkedin
- Delaying Asynchronous Message Processing with RabbitMQ at Indeed
- Real-time Data Pipeline with Kafka at Yelp
- Building Reliable Reprocessing and Dead Letter Queues with Kafka at Uber
- Audit Kafka End-to-End at Uber (count each message exactly once, audit a message across tiers)
- Kafka for PaaS at Rakuten
- Publishing with Kafka at The New York Times
- Kafka Streams on Heroku
- Kafka in Platform Events Architecture at Salesforce
- Bullet: Forward-Looking Query Engine for Streaming Data at Yahoo
- Benchmarking Streaming Computation Engines at Yahoo
- Deduplication Techniques
- Distributed Searching
- Search Architecture of Instagram
- Search Architecture of eBay
- Improving Search Engine Efficiency by over 25% at eBay
- Elasticsearch Performance Tuning Practice at eBay
- Nautilus: Travel Search Engine of Expedia
- Galene: Search Architecture of LinkedIn (2014)
- Search Federation Architecture at LinkedIn (2018)
- Search at Slack
- Search Service (Half a Trillion Documents and Query Average Latency < 100ms) at Twitter (2014)
- Manas: High Performing Customized Search System at Pinterest
- Sherlock: Near Real Time Search Indexing at Flipkart
- Nebula: Storage Platform to Build Search Backends at Airbnb
- Elasticsearch at Kickstarter
- Distributed Troubleshooting Platform with ELK Stack at Target.com
- Distributed Storage
- In-memory Storage
- Introduction to In-memory Data - Viktor Gamov, Solutions Architect at Hazelcast
- MemSQL Architecture - The Fast (MVCC, InMem, LockFree, CodeGen) And Familiar (SQL)
- Optimizing Memcached Efficiency at Quora
- Real-Time Data Warehouse with MemSQL on Cisco UCS
- Moving to MemSQL (with Horizontally Scalable, ACID Compliant, MySQL Compatibility) at Tapjoy
- Durable Storage (Amazon S3)
- Reasons for Choosing S3 over HDFS at Databricks
- S3 in the Data Infrastructure at Airbnb
- Quantcast File System on Amazon S3
- Using S3 in Netflix Chukwa
- Yahoo Cloud Object Store - Object Storage at Exabyte Scale
- Ambry: Distributed Immutable Object Store at LinkedIn
- Hammerspace: Persistent, Concurrent, Off-heap Storage at Airbnb
- In-memory Storage
- NoSQL
- Key-Value Databases (DynamoDB, Voldemort, Manhattan)
- Scaling Mapbox infrastructure with DynamoDB Streams
- Manhattan: Twitter’s distributed key-value database
- Sherpa: Yahoo’s distributed NoSQL key-value store
- Riak inside Chat Service Architecture at Riot Games
- MPH: Fast and Compact Immutable Key-Value Stores at Indeed
- zBase: High Performance, Elastic, Distributed Key-Value Store at Zynga
- Column Databases (Cassandra, HBase, Vertica, Sybase IQ)
- Consistent Hashing in Cassandra
- Understanding Gossip (Cassandra Internals)
- When NOT to use Cassandra?
- Avoid Pitfalls in Scaling Cassandra Cluster: Lessons and Remedies at Walmart Labs
- Storing Images in Cassandra at Walmart Scale
- Cassandra at Instagram
- How Yelp Scaled Ad Analytics with Cassandra
- How Discord Stores Billions of Messages with Cassandra
- Scale to serve 100+ million reads/writes using Spark and Cassandra at Dream11
- Imgur Notification: From MySQL to HBASE at Imgur
- Moving Food Feed from Redis to Cassandra at Zomato
- Benchmarking Cassandra Scalability on AWS at Netflix
- ClickHouse - Open Source Distributed Column Database at Yandex
- Document Databases (MongoDB, SimpleDB, CouchDB)
- eBay: Building Mission-Critical Multi-Data Center Applications with MongoDB
- MongoDB at Baidu: Multi-Tenant Cluster Storing 200+ Billion Documents across 160 Shards
- The AWS and MongoDB Infrastructure of Parse (acquired by Facebook)
- Migrating Mountains of Mongo Data at Addepar
- Couchbase Ecosystem at LinkedIn
- SimpleDB at Zendesk
- Graph Databases
- Datastructure Databases (Redis, Hazelcast)
- Using Redis To Scale at Twitter
- Scaling Job Queue with Redis at Slack
- Moving persistent data out of Redis at Github
- Storing Hundreds of Millions of Simple Key-Value Pairs in Redis at Instagram
- Redis in Chat Architecture of Twitch (from 27:22)
- Learn Redis the hard way (in production) at Trivago
- Optimizing Session Key Storage in Redis at Deliveroo
- Optimizing Redis Storage at Deliveroo
- Key-Value Databases (DynamoDB, Voldemort, Manhattan)
- RDBMS (MySQL, MSSQL, PostgreSQL)
- Microsoft SQL versus MySQL
- SQL Database Performance Tuning
- Scaling PostgreSQL Using CUDA
- Scaling Distributed Joins
- MySQL System Design at Booking.com
- MySQL Parallel Replication (4 parts) at Booking.com
- Sharding MySQL at Pinterest
- Sharding MySQL at MailChimp
- Partitioning Main MySQL Database at Airbnb
- PostgreSQL at Twitch
- Scaling MySQL-based Financial Reporting System at Airbnb
- Scaling MySQL at Wix
- Switching from Postgres to MySQL at Uber
- Handling Growth with Postgres at Instagram
- Scaling the Analytics Database (Postgres) at TransferWise
- MySQL Sharding (3 parts) at Evernote
- Time Series Database (TSDB)
- Time Series Data: Why and How to Use a Relational Database instead of NoSQL
- Beringei: High-performance Time Series Storage Engine at Facebook
- Atlas: In-memory Dimensional Time Series Database at Netflix
- Heroic: Time Series Database at Spotify
- Roshi: Distributed Storage System for Time-Series Event at SoundCloud
- Building a Scalable Time Series Database on PostgreSQL
- Scaling Time Series Data Storage at Netflix
- HTTP Caching (Reverse Proxy, CDN)
- Reverse Proxy (Nginx, Varnish, Squid, rack-cache)
- Stop Worrying and Love the Proxy
- Playing HTTP Tricks with Nginx
- Using CDN to Improve Site Performance at Coursera
- Strategy: Caching 404s Saved 66% On Server Time at The Onion
- Increasing Application Performance with HTTP Cache Headers
- Zynga Geo Proxy: Reducing Mobile Game Latency at Zynga
- Google AMP at Condé Nast
- Running A/B Tests on Hosting Infrastructure (CDNs) at Deliveroo
- HAProxy with Kubernetes for User-facing Traffic at SoundCloud
- Bandaid: Service Proxy at Dropbox
- Load Balancing
- Introduction to Modern Network Load Balancing and Proxying
- Load Balancing infrastructure to support more than 1.3 billion users at Facebook
- DHCPLB: Open Source Load Balancer for DHCP at Facebook
- Load Balancing with Eureka at Netflix
- Load Balancing at Yelp
- Load Balancing at Github
- Consistent Hashing to Improve Load Balancing at Vimeo
- UDP Load Balancing at 500 pixel
- Autoscaling
- Concurrency
- Message-Passing Concurrency
- Software Transactional Memory
- Dataflow Concurrency
- Shared-State Concurrency
- Concurrency series by Larry Osterman (Principal SDE at Microsoft)
- Concurrency with Erlang
- Running Concurrent Queries in GoSocial (Go and Neo4j) at Medium
- The Secret To 10 Million Concurrent Connections
- Parallel Computing
- SPMD (Single Program Multiple Data): The Genetic Pattern
- Master/Worker Pattern
- Loop Parallelism Pattern: Extracting parallel tasks from loops
- Fork/Join Pattern: Good for recursive data processing
- Map-Reduce: Born for Simplified Data Processing on Large Clusters
- On the Death of Map-Reduce - Henry Robinson, Cloudera
- Server-side Optimization to Parallelize the Rendering of Web Pages at Yelp
- Event-Driven Architecture
- Distributed Source Code and Configuration Files Management
Availability
- Failover
- Replication
- NodeJS High Availability at Yahoo
- Every Day Is Monday in Operations (11 parts) at LinkedIn
- Practical Guide to Monitoring and Alerting with Time Series at Scale
- How Robust Monitoring Powers High Availability for LinkedIn Feed
- Architectural Patterns for High Availability - Adrian Cockcroft, Director of Architecture at Netflix
- Ensuring Resilience to Disaster at Quora
- Resiliency against Traffic Oversaturation at iHeartRadio
- Resiliency in Distributed Systems at GO-JEK
- Supporting Global Events at Facebook
Stability
- Circuit Breaker
- Always use timeouts (if possible)
- Let it Crash/Supervisors: Embrace Failure As Natural State
- Crash Early: Better Error Now Than Response Tomorrow
- Crash-safe Replication for MySQL at Booking.com
- Bulkheads: Partition and Tolerate Failure in One Part
- Steady State: Always Put Logs on Separate Disk
- Throttling: Maintain a Steady Pace
- Multi-Clustering: Improving Resiliency and Stability of a Large-scale Monolithic API Service at LinkedIn
Performance
- Improving Performance with Background Data Prefetching at Instagram
- Compression Techniques to Solve Network I/O Bottlenecks at eBay
- Optimizing Web Servers for High Throughput and Low Latency at Dropbox
- Boosting Site Speed Using Brotli Compression at LinkedIn
- Linux Performance Analysis in 60.000 Milliseconds at Netflix
- Optimizing 360 Photos at Scale at Facebook
- Reducing Image File Size in the Photos Infrastructure at Etsy
- Improving Video Thumbnails with Deep Neural Nets at YouTube
- Optimizing APIs through Dynamic Polyglot Runtime, Fully Asynchronous, and Reactive Programming at Netflix
- Optimizing Video Playback Performance at Pinterest
- Optimizing Video Stream for Low Bandwidth with Dynamic Optimizer at Netflix
- Reducing Video Loading Time by Prefetching during Preroll at Dailymotion
- Improving GIF Performance at Pinterest
- Performance Improvements (All Stacks) at Pinterest
- Server Side Rendering at Wix
- 30x Performance Improvements on MySQLStreamer at Yelp
- Performance Monitoring with Riemann and Clojure at Walmart
- Improving Homepage Performance at Zillow
- Decreasing RAM Usage by 40% Using jemalloc with Python & Celery at Zapier
- Using Java Large Heap (110 GB) for Boosting Site Perpormance at Expedia
- The Process of Optimizing for Client Performance at Expedia
Intelligence
- Scalable Deep Learning Platform On Spark In Baidu
- Horovod: Uber’s Open Source Distributed Deep Learning Framework for TensorFlow
- Scaling Gradient Boosted Trees for Click-Through-Rate Prediction at Yelp
- TensorFlowOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo
- CaffeOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo
- AIOps in Practice at Baidu
- Learning with Privacy at Scale - Differential Privacy Team, Apple
- Image Classification Experiment Using Deep Learning at Mercari
- Content-based Video Relevance Prediction at Hulu
- PaddlePaddle Fluid: Elastic Deep Learning on Kubernetes at Baidu
- Training ML Models with Airflow and BigQuery at WePay
- Improving Photo Selection With Deep Learning at TripAdvisor
- Machine Learning (2 parts) at Condé Nast
- Machine Learning Applications In The E-commerce Domain (4 parts) at Rakuten
- Venue Rating System at Foursquare
- Using Machine Learning to Improve Streaming Quality at Netflix
Architectures
- Architecture of Tripod (Flickr’s Backend)
- Architecture of SurveyMonkey
- Architecture of Data Platform at Flipkart
- Architecture of Data Infrastructure at GO-JEK
- Architecture of Stack Overflow Enterprise at Palantir
- Architecture of Distributed Cron at Quora
- Architecture of Real-Time Presence Platform at LinkedIn
- Architecture of LIVE's Encoder Layer at LINE
- Architecture of Back-end at BlaBlaCar
- Architecture of Simone: Distributed Simulation Service at Netflix
- Architecture of Seagull: Distributed System that Helps Running > 20 Million Tests Per Day at Yelp
- Architecture of Cloud Bouncer: Distributed Rate Limiting at Yahoo
- Architecture of Finance and Banking Systems
Ad-hoc
- Systems We Make (Academic Papers)
- Criteria for Selecting a Cloud Provider at Etsy
- Practical NoSQL Resilience Design Pattern for the Enterprise at eBay
- Basic Infrastructure Patterns at Zenefits
- Syscall Auditing at Scale at Slack
- Service Decomposition at Scale at Intuit QuickBooks
- Scalable Gaming Patterns on AWS
- Scaling Chat To 70 Million Players at League Of Legends
- Scaling Online Migrations at Stripe
- Scaling NodeJS at Alibaba
Interview
- My Scaling Hero - Jeff Atwood (Read it for Endorphins before Inteview, JK)
- Notes on Distributed Systems for Young Bloods
- Anatomy of a System Design Interview
- Top 10 System Design Interview Questions
- Top 10 Common Large-Scale Software Architectural Patterns in a Nutshell
- How NOT to design Netflix in your 45-minute System Design Interview?
- Netflix: What Happens When You Press Play?
- System Design - Hired in Tech
- How to Ace a Systems Design Interview - Palantir
- The Architecture Interview - Susan Fowler
- OSI and TCP/IP Cheat Sheet (Short but Sweet)
- The Precise Meaning of I/O Wait Time in Linux
Talks
- Distributed Systems in One Lesson - Tim Berglund, Senior Director of Developer Experience at Confluent
- Building Real Time Infrastructure at Facebook - Jeff Barber and Shie Erlich, Software Engineer at Facebook
- Building Reliable Social Infrastructure for Google - Marc Alvidrez, Senior Manager at Google
- Site Reliability Engineering at Dropbox - Tammy Butow, Site Reliability Engineering Manager at Dropbox
- How Google Does Planet-Scale for Planet-Scale Infra - Melissa Binde, SRE Director for Google Cloud Platform
- Netflix Guide to Microservices - Josh Evans, Director of Operations Engineering at Netflix
- Achieving Rapid Response Times in Large Online Services - Jeff Dean, Google Senior Fellow
- Architecture to Handle 80K RPS Celebrity Sales at Shopify - Simon Eskildsen, Engineering Lead at Shopify
- Lessons of Scale at Facebook - Bobby Johnson, Director of Engineering at Facebook
- Performance Optimization for the Greater China Region at Salesforce - Jeff Cheng, Enterprise Architect at Salesforce
- How GIPHY Delivers a GIF to 300 Millions Users - Alex Hoang and Nima Khoshini, Services Engineers at GIPHY
- High Performance Packet Processing Platform at Alibaba - Haiyong Wang, Senior Director at Alibaba
- Scaling Dropbox - Kevin Modzelewski, Back-end Engineer at Dropbox
- Scaling Reliability at Dropbox - Sat Kriya Khalsa, SRE at Dropbox
- Scaling with Performance at Facebook - Bill Jia, VP of Infrastructure at Facebook
- Scaling Live Videos to a Billion Users at Facebook - Sachin Kulkarni, Director of Engineering at Facebook
- Scaling Low-latency Live Streams at Facebook (Latencies for Real-time Interactions) - Saral Shodhan, SDE at Facebook
- Scaling Low-latency Live Streams at Facebook (End-to-End Considerations) - Federico Larumbe, SDE at Facebook
- Scaling Infrastructure at Instagram - Lisa Guo, Instagram Engineering
- Scaling Infrastructure at Twitter - Yao Yue, Staff Software Engineer at Twitter
- Scaling Infrastructure at Etsy - Bethany Macri, Engineering Manager at Etsy
- Scaling Real-time Infrastructure at Alibaba for Global Shopping Holiday - Xiaowei Jiang, Senior Director at Alibaba
- Scaling Data Infrastructure at Spotify - Matti (Lepistö) Pehrs, Spotify
- Scaling Pinterest - Marty Weiner, Pinterest’s founding engineer
- Scaling Slack - Bing Wei, Software Engineer (Infrastructure) at Slack
- Scaling Backend at Youtube - Sugu Sougoumarane, SDE at Youtube
- Scaling Backend at Uber - Matt Ranney, Chief Systems Architect at Uber
- Scaling Global CDN at Netflix - Dave Temkin, Director of Global Networks at Netflix
- Scaling Load Balancing Infra to Support 1.3 Billion Users at Facebook - Patrick Shuff, Production Engineer at Facebook
- Scaling (a NSFW site) to 200 Million Views A Day And Beyond - Eric Pickup, Lead Platform Developer at MindGeek
- Scaling Counting Infrastructure at Quora - Chun-Ho Hung and Nikhil Gar, SEs at Quora
- Scaling Git at Microsoft - Saeed Noursalehi, Principal Program Manager at Microsoft
Books
- Big Data, Web Ops & DevOps Ebooks - O'Reilly (Online - Free)
- Google Site Reliability Engineering (Online - Free)
- Distributed Systems for Fun and Profit (Online - Free)
- What Every Developer Should Know About SQL Performance (Online - Free)
- Beyond the Twelve-Factor App - Exploring the DNA of Highly Scalable, Resilient Cloud Applications (Free)
- Chaos Engineering - Building Confidence in System Behavior through Experiments (Free)
- The Art of Scalability
- Designing Data-Intensive Applications
- Web Scalability for Startup Engineers
- Scalability Rules: 50 Principles for Scaling Web Sites
Special Thanks
- Jonas Bonér, CTO at Lightbend, for the original inspiration
License
Copyright Benny (Quoc-Binh) Nguyen, 2018. Licensed under a Creative Commons Attribution 4.0 International License. This small work is dedicated to the wolves climbing the hill.