Awesome Scalability, Availability, and Stability Backend

A curated list of selected readings/case studies to illustrate Scalability, Availability, and Stability patterns in backend design.

What if your backend went slow?

Understand your problems: performance problem (slow for a single user) or scalability problem (fast for a single user but slow under heavy load) from basic.

What if your backend went down?

"Even if you lose all one day, you can build all over again if you retain your calm!" - Thuan Pham, CTO at Uber Technologies Inc.

Contributing

Please take a look at the contribution guidelines first. Contributions are always welcome!

Contents

• Basic
• Scalability
• Availability
• Stability

Basic

• CAP Theorem and the Trade-offs
• Scale up vs Scale out
• Latency vs Throughput: Striving for maximal throughput with acceptable latency
• ACID?
• Architecture Issues: Bottlenecks, Database, CPU, IO
• Immutability

Scalability

• Distributed Caching
  • Write-behind and Write-through
  • Eviction Policies
  • Peer-To-Peer Caching
• Storage
  • In-memory Storage
  • Durable Storage
• NoSQL
  • Key-Value Databases (Voldemort, Dynomite)
  • Document Databases (Cassandra, Vertica, Sybase IQ)
    • Consistent Hashing in Cassandra
  • Graph Databases (MongoDB, CouchDB, Neo4j)
  • Datastructure Databases (Redis, Hazelcast)
• RDBMS
  • Sharding
    • Partitioning
    • Replication
  • Denormalization
  • ORM + Rich Domain Model Antipattern
• HTTP Caching
  • Reverse Proxy (Nginx, Varnish, Squid, rack-cache)
  • CDN (Akamai)
• Concurrency
  • Message-Passing Concurrency
  • Software Transactional Memory
  • Dataflow Concurrency
  • Shared-State Concurrency
• Event-Driven Architecture
  • Messaging
    • Publish-Subscribe
    • Point-to-Point
    • Store-Forward
    • Request-Reply
  • Actors: Fire-forget and Fire-Receive-Eventually
  • Enterprise Service Bus
  • Domain Events
  • Event Stream Processing
  • Event Sourcing
  • Command & Query Responsibility Segregation (CQRS)
• Load-balancing
  • Round-robin Allocation
  • Random Allocation
  • Weighted Allocation
  • Dynamic Load-balancing
    • Work-stealing
  • Consistent Hashing
  • UDP Load-balancing
• Parallel Computing
  • SPMD (Single Program Multiple Data): The Genetic Pattern
  • Master/Worker Pattern
  • Loop Parallelism Pattern: Extracting parallel tasks from loops
  • Fork/Join Pattern: Good for recursive data processing
  • MapReduce Pattern: Born for Big Data

Availability

• Fail-over
• Replication
  • Master-Slave
  • Tree Replication
  • Master-Master
  • Buddy Replication

Stability

• Circuit Breaker
• Always use timeouts (if possible)
• Let it crash/Supervisors: Embrace failure as a natural state in the life-cycle of the application
• Crash early: An error now is better than a response tomorrow
• Bulkheads: Partition and tolerate failure in one part
• Steady state: Always put logs on separate disk
• Throttling: Maintain a steady pace

Special Thanks

• Jonas Bonér, CTO at Lightbend, for the awesome main reference source