Building Practical Distributed Systems

What happens when the application logic that worked perfectly on one machine suddenly has to work across multiple services, processes, databases, and machines?
A network call can fail without telling you whether the operation succeeded. A retry can create duplicate work. A database update can succeed while its event never reaches another service. Data can be correct in one place and stale in another. As systems grow, failures that were once simple errors become problems of uncertainty, timing, coordination, consistency, and recovery.
Building Practical Distributed Systems teaches you how to design applications that continue working when those failures occur.
Instead of treating distributed systems as a collection of abstract theories, this book takes a practical, problem-solving approach. You will build and evolve an order-processing system while encountering the kinds of failures that developers must account for in real distributed applications. Each new problem introduces a distributed-systems principle, applies it to the system, and then tests whether the protection actually works.
You will learn how to:

• Understand what changes when application components communicate across machines
• Design effective timeouts and deadlines for remote operations
• Use retries without creating retry storms or amplifying failures
• Make operations safe to repeat through idempotent design
• Reason about unknown outcomes when a remote operation times out
• Understand consistency, replication, and stale data
• Reliably coordinate database changes with asynchronous events
• Handle duplicate messages and recover failed background processing
• Design multi-step workflows when one global transaction is not available
• Distribute workloads without creating damaging hot spots
• Apply backpressure, resource limits, and workload isolation
• Contain cascading failures across dependent services
• Use logs, metrics, and distributed tracing to find the real source of a failure
• Test distributed systems against delays, duplication, overload, partial failure, and recovery scenarios