CQRS separates the write model (commands) from the read model (queries) into two distinct paths. The write side validates business invariants and persists events or state changes. The read side projects data into shapes optimised for queries — denormalized views, materialized aggregations, or search indexes. This separation lets you scale reads and writes independently: your write database can be a strongly consistent relational store while reads are served from Redis, Elasticsearch, or a purpose-built read replica.

In practice, CQRS surfaces wherever you see a system that writes to one store and reads from another. An e-commerce platform writes orders to a PostgreSQL database but projects order history into DynamoDB for fast customer lookups. A social network writes posts to a primary datastore but fans them out into per-user timeline caches. The two sides are kept in sync by an event pipeline — usually Kafka, Kinesis, or an internal change data capture (CDC) mechanism.

Client → [Command Bus] → Write Model → Event Store → [Projection Engine] → Read Model → Client

Event Sourcing stores every state change as an immutable event rather than overwriting the current state. Instead of a "users" table with the latest values, you have an append-only log: UserCreated, EmailChanged, AccountDeactivated. The current state is derived by replaying events from the beginning (or from a snapshot). This gives you a complete audit trail, the ability to rebuild state at any point in time, and natural compatibility with CQRS — events from the write side feed projections on the read side.

Kafka, EventStoreDB, and AWS DynamoDB Streams are common infrastructure choices. The pattern is used by financial systems (every transaction is an event), e-commerce order pipelines, and collaborative editing tools. The main costs are increased storage, the need for snapshotting (replaying millions of events on every read is impractical), and the complexity of handling schema evolution as event shapes change over time.

A saga is a sequence of local transactions where each step publishes an event or sends a command that triggers the next step. If any step fails, compensating transactions undo the work of prior steps. Sagas replace the heavyweight two-phase commit (2PC) protocol, which requires locking resources across services and becomes a scalability bottleneck. There are two coordination styles: orchestration and choreography.

In orchestration, a central saga coordinator (orchestrator) tells each service what to do and what to compensate if things fail. In choreography, each service listens for events and decides independently what to do next — no central coordinator. Orchestration is easier to reason about and debug; choreography is more decoupled but can become a tangled web of implicit dependencies.

The circuit breaker pattern prevents a failing downstream dependency from cascading failures across the entire system. It wraps remote calls in a state machine with three states: Closed (normal operation — requests pass through), Open (the dependency is considered down — requests fail immediately without attempting the call), and Half-Open (a trial period where a limited number of requests are allowed through to test if the dependency has recovered). Netflix popularised this pattern with Hystrix; modern implementations include Resilience4j for Java and Polly for .NET.

When in the Closed state, the circuit breaker tracks failure rates. Once failures exceed a threshold (e.g., 50% of the last 100 requests within a 10-second window), the breaker trips to Open. After a configurable timeout, it transitions to Half-Open and allows a probe request. If the probe succeeds, the breaker resets to Closed; if it fails, the breaker returns to Open.

CircuitBreakerConfig.custom()
  .failureRateThreshold(50)
  .waitDurationInOpenState(Duration.ofSeconds(30))
  .slidingWindowSize(100)
  .permittedNumberOfCallsInHalfOpenState(3)
  .build()

Named after the watertight compartments in a ship hull, the bulkhead pattern isolates components so that a failure in one does not sink the entire system. In software, this means giving each downstream dependency (or tenant, or workload class) its own dedicated pool of resources — thread pools, connection pools, rate limiters, or even separate infrastructure. If the recommendation service exhausts its thread pool, the checkout service continues unaffected because it has its own isolated pool.

Bulkheads come in two flavours: thread-pool isolation and semaphore isolation. Thread-pool isolation gives each dependency its own fixed-size thread pool and a bounded queue. This provides strong isolation but adds overhead from context switching and thread management. Semaphore isolation limits concurrency with a counter (no separate threads), offering lower overhead but less protection against slow calls that tie up the calling thread.

Transient failures — network blips, brief overloads, leader elections — are inevitable in distributed systems. A naive immediate retry floods the recovering service with a thundering herd. Exponential backoff increases the delay between retries exponentially (e.g., 100ms, 200ms, 400ms, 800ms), giving the downstream time to recover. Adding jitter (randomness) to the delay decorrelates retries from multiple callers, preventing synchronized retry storms.

There are three common jitter strategies. Full jitter picks a random delay between 0 and the exponential ceiling. Equal jitter uses half the exponential value plus a random component up to the other half. Decorrelated jitter makes each delay independent of the previous delay. AWS recommends full jitter for most cases because it produces the broadest spread and the fewest collisions.

An operation is idempotent if performing it multiple times produces the same result as performing it once. In distributed systems where retries, duplicate messages, and at-least-once delivery are the norm, idempotency is the mechanism that prevents double charges, duplicate orders, and corrupted state. The standard approach is an idempotency key: the client generates a unique identifier (UUID) for each logical operation and sends it with every request. The server stores the key alongside the result; if it sees the same key again, it returns the cached result instead of re-executing.

Stripe, PayPal, and most payment APIs require an Idempotency-Key header. Kafka achieves exactly-once semantics via producer IDs and sequence numbers. DynamoDB conditional writes (PutItem with attribute_not_exists) provide idempotent inserts. The pattern applies anywhere messages might be delivered more than once.

A write-ahead log (WAL) is an append-only file where every mutation is recorded before it is applied to the main data structure. If the system crashes mid-operation, the WAL provides the information needed to recover: either replay uncommitted entries to complete the operation or discard them to roll back. WAL is the foundation of durability in PostgreSQL, MySQL (InnoDB redo log), SQLite, etcd, and Kafka (the commit log itself is a WAL).

The mechanism is straightforward: (1) write the intended change to the WAL on disk, (2) fsync the WAL to guarantee persistence, (3) apply the change to the in-memory data structure and/or main data files, (4) acknowledge success to the client. Because sequential appends to a log are far faster than random writes to a B-tree or LSM-tree, WAL allows high write throughput while guaranteeing durability.

Sharding (horizontal partitioning) splits data across multiple nodes so that no single node holds the entire dataset. Each shard is responsible for a subset of the data, determined by a partitioning strategy. The three main strategies are range-based, hash-based, and directory-based partitioning. The choice affects query patterns, hotspot risk, and operational complexity.

Range partitioning assigns contiguous key ranges to shards (e.g., users A-M on shard 1, N-Z on shard 2). It supports efficient range scans but risks hotspots if access is skewed. Hash partitioning applies a hash function to the key and maps the result to a shard; this distributes load uniformly but destroys key ordering, making range queries expensive. Directory-based partitioning uses a lookup table to map keys to shards, offering maximum flexibility but adding the lookup service as a dependency and potential bottleneck. Consistent hashing (used by DynamoDB, Cassandra, and many caches) is a specialisation of hash partitioning that minimises data movement when nodes are added or removed.

Leader election designates a single node as the coordinator (leader) for a task while others act as followers or standby replicas. The leader makes decisions, assigns work, or manages writes, avoiding the split-brain problems that arise when multiple nodes believe they are in charge. When the leader fails, the remaining nodes detect the failure and elect a new leader. Raft (used by etcd, Consul, CockroachDB) and ZAB (used by ZooKeeper) are the two most widely deployed consensus algorithms for leader election.

In Raft, nodes start as followers. If a follower does not receive a heartbeat from the leader within a randomised timeout, it transitions to candidate and requests votes from peers. If it receives a majority of votes, it becomes leader and begins sending heartbeats. The randomised timeout prevents split votes. ZooKeeper uses ephemeral sequential znodes: each node creates a znode and the node with the lowest sequence number becomes the leader. When the leader crashes, its ephemeral znode disappears, and the next node in sequence takes over.

Follower --[timeout]--> Candidate --[majority vote]--> Leader --[heartbeat]--> Followers
Leader --[crash]--> Followers detect timeout --> New election round

This pattern addresses how to deliver content (posts, updates, notifications) from a producer to many consumers. Fan-out-on-write (push model) pre-computes the result at write time: when a user posts, the system immediately writes that post into every follower timeline cache. Fan-out-on-read (pull model) defers the work to read time: when a follower opens their feed, the system gathers and merges posts from all followed users on the fly. Most real-world systems use a hybrid approach.

Fan-out-on-write delivers fast reads (the timeline is pre-built) but is expensive for users with millions of followers — writing a single celebrity post to 50 million timeline caches is prohibitively slow and wasteful. Fan-out-on-read saves write amplification for high-follower users but pushes latency to read time, which can be slow if a user follows many accounts. The hybrid approach (used by Twitter) fans out on write for regular users and fans out on read for celebrities, merging the two at read time.

The sidecar pattern deploys a helper process alongside your main application in the same host or pod. The sidecar handles cross-cutting concerns — networking (service mesh proxies like Envoy), observability (log shippers, metric collectors), security (mTLS termination, authentication proxies), or configuration (secret injection). The application itself remains unaware of the sidecar; it communicates with it over localhost or a shared Unix socket. This lets you add capabilities to any service regardless of its language, framework, or age.

Service meshes like Istio and Linkerd are the most prominent example: an Envoy sidecar proxy intercepts all inbound and outbound traffic, handling load balancing, circuit breaking, retries, mTLS, and distributed tracing without any code changes to the application. Other sidecar examples include Fluentd or Filebeat for log collection, Vault Agent for secret injection, and OPA (Open Policy Agent) for policy enforcement.

No pattern exists in isolation. A real system design composes multiple patterns to address different requirements simultaneously. Understanding which patterns complement each other — and which create tension — is a key differentiator in senior-level interviews.