Chat systems are deceptively simple on the surface — users send text messages to each other, and messages appear on screens. But underneath that simplicity lies one of the most demanding distributed systems problems in all of software engineering. A chat system must simultaneously guarantee real-time delivery (sub-second latency), persistent storage (messages must never be lost), ordering guarantees (messages appear in the correct sequence), presence tracking (who is online right now), and delivery confirmation (was the message received and read). No other system design question requires you to solve all five of these constraints at once.

Consider the scale of production chat systems. WhatsApp handles over 100 billion messages per day from 2 billion monthly active users. Slack serves over 30 million daily active users across millions of workspaces with rich threading, search, and file sharing. Discord supports servers with hundreds of thousands of concurrent users in a single channel, all receiving messages in real time. Facebook Messenger processes 100 billion messages per day across 1.3 billion monthly users. Each of these platforms has made fundamentally different architectural choices, yet they all grapple with the same core engineering challenges.

The fundamental challenge is that chat combines two conflicting paradigms: real-time streaming and durable persistence. A real-time system optimizes for latency — get the message to the recipient as fast as possible, ideally under 100 milliseconds. A durable storage system optimizes for reliability — never lose a message, even if servers crash. In most systems, you pick one paradigm or the other. Chat demands both simultaneously, and the interaction between them creates cascading complexity.

This entry walks through the complete design from requirements to production architecture. We cover connection management with WebSockets, the end-to-end message flow from sender to recipient, message storage schema design, group chat fan-out architecture, presence and typing indicators, delivery semantics with receipt tracking, and scaling strategies for billions of daily messages. Each section is written at the depth expected in a FAANG L5-L6 system design interview.

Before drawing any architecture diagrams, a strong system design answer begins with clarifying requirements and running back-of-the-envelope calculations. For a chat system, the functional requirements define the messaging capabilities, while the non-functional requirements establish the latency, durability, and scale targets that drive every architectural decision.

Now let us run the estimation math. These numbers determine whether a single server, a handful of servers, or a massive distributed fleet is required for each component.

These estimation numbers reveal the architecture. We need a fleet of 2,000+ stateful WebSocket connection servers, a distributed message store capable of sustained millions of writes per second, a message routing layer that can look up which connection server each recipient is connected to, and an offline queue for users who are not currently connected. Each of these components has its own scaling challenges, which we address in the following sections.

The transport layer is the foundation of a real-time chat system. Unlike traditional HTTP request-response APIs, chat requires the server to push messages to clients at any time without the client asking. This demands a persistent, bidirectional communication channel between every online client and the server infrastructure. The choice of transport protocol has profound implications for server resource consumption, latency, battery life on mobile devices, and operational complexity.

WebSocket is the dominant transport for production chat systems. A WebSocket connection starts as an HTTP request (the handshake) and then upgrades to a persistent, full-duplex TCP connection. Once established, both the client and server can send frames at any time with minimal overhead — just 2-14 bytes of framing per message, compared to hundreds of bytes of HTTP headers for each request. WhatsApp, Slack, Discord, and Messenger all use WebSockets as their primary transport.

graph TD
    A[Client] -->|"1. HTTP Upgrade Request"| B[Load Balancer L4/L7]
    B -->|"2. Route to Connection Server"| C[Connection Server]
    C -->|"3. Upgrade to WebSocket"| A
    C -->|"4. Register connection"| D[Connection Registry / Redis]
    A <-->|"5. Bidirectional messages"| C
    C -->|"6. Heartbeat every 30s"| A
    A -->|"7. Pong response"| C
    C -->|"8. No pong in 90s = disconnect"| E[Cleanup: Remove from Registry]

    style A fill:#e1f5fe
    style C fill:#f3e5f5
    style D fill:#fff3e0

The connection server fleet is the most stateful component in the entire chat architecture. Each connection server maintains tens of thousands of open TCP connections, each representing an active user session. When a connection server receives a message destined for a user connected to that server, it writes the message frame directly to the appropriate TCP socket. This statefulness creates operational challenges: you cannot simply restart a connection server without disconnecting all its users, and load balancing must be sticky (route the same client to the same server for reconnections).

Long polling is the fallback transport for environments where WebSockets are blocked (some corporate firewalls, older proxy servers). In long polling, the client sends an HTTP request that the server holds open for up to 30 seconds. If a message arrives during that window, the server responds immediately with the message. If no message arrives, the server responds with an empty body after the timeout, and the client immediately sends a new request. The disadvantage is higher latency (messages wait for the next poll cycle), higher server resource usage (holding open HTTP connections consumes more memory than WebSocket frames), and higher battery drain on mobile (frequent HTTP roundtrips).

Reconnection logic is critical for mobile clients. Mobile devices constantly switch between WiFi and cellular networks, enter tunnels, or go through periods of poor connectivity. The client must detect disconnection (via heartbeat failure or TCP reset), exponentially back off reconnection attempts (1s, 2s, 4s, 8s, capped at 30s), and upon reconnection, sync any messages missed during the disconnect window. The sync protocol sends the client's last-seen message ID, and the server responds with all messages after that ID. This is the foundation of the message sync protocol discussed in Section 9.

The end-to-end message flow is the heart of any chat system. When a user types a message and taps send, that message must traverse a pipeline of services, be persisted durably, and be delivered to the intended recipient — all within a few hundred milliseconds. Understanding this flow in detail is critical for a system design interview because it reveals the interaction between stateless services, stateful connection servers, persistent storage, and asynchronous delivery mechanisms.

Let us trace the complete lifecycle of a single 1-on-1 message from sender to recipient. This is the critical path that must be optimized for latency.

sequenceDiagram
    participant S as Sender Client
    participant CS1 as Connection Server A
    participant GW as API Gateway
    participant MS as Message Service
    participant DB as Message Store
    participant MQ as Message Queue
    participant CS2 as Connection Server B
    participant R as Recipient Client

    S->>CS1: 1. Send message via WebSocket
    CS1->>GW: 2. Forward to API Gateway
    GW->>MS: 3. Route to Message Service
    MS->>DB: 4. Persist message (write-ahead)
    DB-->>MS: 5. Write acknowledged
    MS-->>CS1: 6. ACK to sender (message stored)
    CS1-->>S: 7. Show sent checkmark
    MS->>MQ: 8. Publish delivery event
    MQ->>CS2: 9. Route to recipient connection server
    CS2->>R: 10. Push message via WebSocket
    R-->>CS2: 11. Client ACK (delivered)
    CS2-->>MS: 12. Update status: delivered
    MS-->>CS1: 13. Notify sender
    CS1-->>S: 14. Show delivered checkmark

Step by step, here is what happens. The sender client writes the message to its local WebSocket connection (step 1). The connection server receives the WebSocket frame and forwards it to the API gateway (step 2), which routes to the message service (step 3). The message service assigns a globally unique message ID (using a Snowflake-style ID generator for chronological sortability), validates the message (sender is a participant in this conversation, message size is within limits), and writes the message to the durable message store (step 4). Once the write is acknowledged by the store (step 5), the message service sends an acknowledgment back to the sender via the connection server (steps 6-7). The sender sees a single checkmark indicating the message has been stored safely.

After persisting, the message service publishes a delivery event to a message queue or event bus (step 8). A delivery worker consumes this event, looks up the recipient in the connection registry to find which connection server holds their WebSocket, and routes the message to that connection server (step 9). The connection server pushes the message to the recipient via their WebSocket connection (step 10). The recipient client receives the message, displays it, and sends a client-level ACK back through the WebSocket (step 11). This ACK propagates back to the message service (step 12), which updates the message status to "delivered" and notifies the sender (steps 13-14). The sender sees a double checkmark.

The entire critical path from send to delivered confirmation has a latency budget of roughly 300-500ms. The breakdown is approximately: client to connection server (10-30ms network), connection server to message service (1-5ms internal), message service to database write (5-20ms depending on storage), message service to queue publish (1-5ms), queue to delivery worker (1-5ms), delivery worker to recipient connection server (1-5ms), and connection server to recipient client (10-30ms network). The dominant factor is the two network hops to/from the clients, especially on mobile networks with higher latency.

Message storage is the most write-intensive component of a chat system. At WhatsApp scale, the storage layer must handle 1-3 million writes per second sustained, with daily ingestion of 25+ TB of new message data. The choice of storage technology, table schema, partition strategy, and index design determines whether the system can scale to billions of daily messages or collapses under write pressure.

The message table schema must support two primary access patterns efficiently: (1) insert a new message for a conversation, and (2) retrieve the most recent N messages for a conversation (the "load chat history" query). Both operations must be fast — writes in single-digit milliseconds and reads returning the last 50 messages in under 10ms.

The critical insight for message storage is the partition key. Messages must be partitioned by conversation_id, not by user_id. The reason: the most frequent query is "get the last 50 messages in conversation X." If messages are partitioned by conversation_id, all messages for a conversation are stored on the same partition (same node or same SSTable in Cassandra), and the query is a simple sequential read of the last 50 rows. If messages were partitioned by user_id, loading a conversation would require reading from multiple partitions (one for each participant), then merging and sorting — much slower.

Choosing between Cassandra and MySQL for message storage is a critical architectural decision that depends on scale, operational expertise, and access patterns.

Time-series optimization is essential for chat storage. Messages are inherently time-ordered — users almost always read recent messages, and older messages are accessed exponentially less frequently. This creates a natural hot/cold data pattern: the last 30 days of messages are "hot" (frequently accessed), while older messages are "cold" (rarely accessed). Production systems exploit this pattern with tiered storage: hot messages live in fast storage (SSD-backed Cassandra or MySQL on NVMe), while cold messages are archived to cheaper storage (S3 or HDD-backed clusters). A background archival process moves messages older than 30-90 days from the hot tier to the cold tier, keeping the hot tier small and fast.

For search functionality (searching message content within a conversation or across all conversations), the primary message store is not suitable — Cassandra does not support full-text search, and SQL LIKE queries are too slow at scale. Instead, message content is asynchronously indexed into Elasticsearch or a similar search engine. A Kafka consumer reads new messages from the message pipeline and writes them to the search index. Search queries hit Elasticsearch, which returns message IDs, and the chat client then fetches the full messages from the primary store. This separation keeps the write-optimized message store simple while providing rich search capabilities.

Group chat introduces a fan-out problem that fundamentally changes the message delivery architecture. In a 1-on-1 conversation, sending a message means delivering it to exactly one recipient. In a group conversation with N members, sending a message means delivering it to N-1 recipients. The complexity and cost of this fan-out varies dramatically depending on group size, and production systems handle small groups and large channels with entirely different mechanisms.

Consider the fan-out math. A message to a group of 10 requires 9 deliveries. A message to a group of 100 requires 99 deliveries. A message to a Discord server channel with 100,000 members viewing the channel requires 100,000 deliveries. If each delivery takes 1ms of processing, a 10-person group adds 9ms of fan-out latency, while a 100K-member channel adds 100 seconds — obviously unacceptable. This is why the fan-out strategy must differ by group size.

graph TD
    A[Sender sends message to Group] --> B{Group Size?}
    B -->|"Small Group ≤ 200"| C[Direct Fan-Out]
    B -->|"Large Channel > 200"| D[Pub/Sub Fan-Out]

    C --> C1[Lookup all member connection servers]
    C1 --> C2[Send message to each member individually]
    C2 --> C3[Each member gets push via WebSocket]

    D --> D1[Publish to channel topic]
    D1 --> D2[Connection servers subscribe to channels]
    D2 --> D3[Each server pushes to its local members]

    style C fill:#e8f5e9
    style D fill:#fff3e0
    style B fill:#e1f5fe

For small groups (up to about 200 members), direct fan-out works well. When a message arrives at the message service, it looks up the group membership list, queries the connection registry for each member, and sends the message individually to each member's connection server. The message is stored once in the messages table with the group's conversation_id, but delivered N-1 times. Each recipient sees the same message_id. This approach is simple, provides consistent ordering (all members get messages in the same order), and allows per-member delivery tracking (you know exactly which members received and read the message).

For large channels (hundreds or thousands of concurrent viewers), direct fan-out is too slow and too expensive. Instead, use a pub/sub model. Each large channel is represented as a topic in a pub/sub system (Redis Pub/Sub, or a dedicated message broker). Connection servers that have users subscribed to that channel subscribe to the channel topic. When a message is sent to the channel, it is published to the topic once. Each subscribed connection server receives the message and fans it out locally to the users on that server who are in the channel. This reduces the fan-out from N messages to M messages, where M is the number of connection servers with active subscribers (typically orders of magnitude smaller than N).

Message ordering in group chats is more complex than in 1-on-1 conversations. In a 1-on-1 chat, only two users are sending, and the server assigns monotonically increasing Snowflake IDs, so ordering is straightforward. In a group chat, multiple users may send messages simultaneously. The server must establish a total order for all messages in the group. The simplest approach (and what most systems use) is server-side ordering: the message service assigns Snowflake IDs sequentially, and all clients display messages ordered by message_id. Since Snowflake IDs are timestamp-based, this gives a rough chronological order with tie-breaking by machine_id and sequence number.

Online presence (showing whether a user is online, offline, or idle) and typing indicators (showing when someone is composing a message) are secondary features that seem simple but create surprisingly complex engineering challenges at scale. These features are inherently real-time, ephemeral, and high-frequency — the exact combination that stresses distributed systems the most.

The presence service tracks the online/offline status of every user in the system. At 500M DAU with 200M concurrent connections, the presence service must manage 200 million active presence entries and handle millions of status changes per minute (users connecting, disconnecting, and timing out). The naive approach — broadcasting every status change to every user who cares — creates an explosion of messages that can dwarf the actual chat traffic.

graph TD
    A[Client connects] -->|"Register: user online"| B[Presence Service]
    B -->|"Store in Redis"| C["Presence Store<br/>user_id → {status, last_seen, server_id}"]
    D[Client heartbeat every 30s] -->|"Update last_seen"| B
    B -->|"Publish status change"| E[Pub/Sub Channel per User]
    E -->|"Notify subscribed friends"| F[Connection Servers]
    F -->|"Push presence update"| G[Online Friends]

    H[No heartbeat for 90s] -->|"Mark offline"| B
    B -->|"Publish offline event"| E

    style B fill:#f3e5f5
    style C fill:#fff3e0
    style E fill:#e8f5e9

The heartbeat protocol is the foundation of presence detection. When a user connects, the connection server reports the event to the presence service, which stores the entry in Redis: user_id maps to a hash containing status (online/idle/offline), last_seen timestamp, and connection_server_id. The client sends a heartbeat every 30 seconds through the WebSocket connection. The connection server forwards these heartbeats to the presence service, which updates the last_seen timestamp. If no heartbeat is received for 90 seconds (3 missed heartbeats), the presence service transitions the user to offline status.

The presence store in Redis uses a simple key-value structure with TTL-based expiry as a safety net. Each entry is a Redis hash: HSET presence:{user_id} status "online" last_seen 1704067200 server_id "ws-server-42". The key has a TTL of 120 seconds, which is reset on every heartbeat. If the presence service crashes or the heartbeat pipeline fails, the TTL ensures stale presence entries are automatically cleaned up. This is a common pattern: use active updates for normal operation and TTL as a passive safety net.

Typing indicators are even more ephemeral than presence. When a user starts typing in a conversation, the client sends a "typing" event to the server. The server forwards this event to the other participants in the conversation (for 1-on-1) or to the conversation's pub/sub topic (for groups). Critically, typing events are never persisted — they are purely real-time, fire-and-forget messages. If the recipient is offline, the typing event is simply dropped.

At scale, presence is one of the most expensive features in a chat system relative to its perceived importance. WhatsApp intentionally limits presence visibility: you only see last-seen times, not real-time online status, and users can disable even that. This dramatically reduces the presence fan-out volume. Slack shows real-time presence but limits it to workspace members (much smaller than a global friend graph). Discord shows presence but only within servers you share with the user. Each platform makes different trade-offs between presence richness and system cost.

Delivery semantics define the contract between the chat system and its users: what does it mean for a message to be "sent," "delivered," and "read"? Getting these semantics right is critical for user trust — users need confidence that their messages are reaching the intended recipient. The engineering challenge is implementing these guarantees reliably across unreliable networks, intermittent connectivity, and millions of concurrent sessions.

The three delivery states correspond to specific system events. "Sent" (single checkmark) means the server has persisted the message and acknowledged receipt to the sender. "Delivered" (double checkmark) means the recipient device has received the message and sent a client-level ACK back to the server. "Read" (blue double checkmark or equivalent) means the recipient has actually viewed the message in the chat window. Each state transition requires a round-trip confirmation, and each can fail independently.

The system uses at-least-once delivery semantics. This means the system guarantees that every message will be delivered at least once to the recipient, but in rare cases (network retries, server failovers), a message might be delivered twice. The alternative, exactly-once delivery, is essentially impossible in a distributed system without prohibitive coordination overhead. At-least-once is the pragmatic choice made by WhatsApp, Slack, Discord, and every other production chat system.

The client ACK mechanism is what enables the transition from "sent" to "delivered." When the recipient device receives a message via WebSocket, it must send an explicit acknowledgment back to the server. This ACK is a small message containing the message_id and a timestamp. The server updates the message status in the database and notifies the sender. If the server does not receive a client ACK within a timeout window (30 seconds), it retries delivery by pushing the message again. After 3 failed retry attempts, the server falls back to push notification delivery.

Read receipts require careful handling of privacy and performance. On the privacy side, many users prefer not to send read receipts (WhatsApp allows disabling them). On the performance side, read receipts generate a high volume of status update messages. When a user opens a conversation and scrolls through 50 unread messages, the client should NOT send 50 individual read receipt events. Instead, it sends a single "read up to message_id X" event, which marks all messages up to that ID as read in a single operation.

Push notifications serve as the delivery fallback for offline users. When the delivery system cannot reach the recipient via WebSocket (no active connection in the registry) and the message has been sitting in the offline queue for more than a configurable delay (typically 5-30 seconds), a push notification is sent via APNs (iOS) or FCM (Android). The push notification contains a truncated message preview and metadata to identify the conversation. When the user taps the notification, the app opens, establishes a WebSocket connection, syncs all pending messages, and the user sees the full conversation. Push notifications are rate-limited per user to prevent notification flooding — at most one push notification per conversation per minute.

Scaling a chat system from millions to billions of daily messages requires careful attention to sharding, replication, cross-region deployment, message synchronization, and encryption. Each of these topics could fill an entire system design session on its own, but an L5-L6 candidate is expected to discuss the key strategies and trade-offs at a high level with specific architectural choices.

Sharding the message store is the first scaling challenge. As discussed in the storage section, messages are partitioned by conversation_id. But how are conversations distributed across database nodes? The standard approach is consistent hashing: conversation_id is hashed, and the hash determines which shard (database partition) stores that conversation's messages. Consistent hashing ensures that adding or removing shards only redistributes a small fraction of conversations, avoiding a full data migration.

Cross-region replication is essential for a global chat service. Users in Asia should not experience 200ms+ latency because the chat servers are in North America. The standard approach is to deploy chat infrastructure in multiple regions (e.g., US-East, EU-West, Asia-Pacific) and route each user to the nearest region. Messages between users in the same region stay local. Messages between users in different regions require cross-region routing.

graph LR
    subgraph "US-East Region"
        A1[Connection Servers] --> B1[Message Service]
        B1 --> C1[Message Store Shard]
    end

    subgraph "EU-West Region"
        A2[Connection Servers] --> B2[Message Service]
        B2 --> C2[Message Store Shard]
    end

    subgraph "Cross-Region Bus"
        D[Kafka / Event Bridge]
    end

    B1 -->|"Cross-region message"| D
    D -->|"Replicate to EU"| B2
    B2 -->|"Cross-region message"| D
    D -->|"Replicate to US"| B1

    style D fill:#fff3e0

The cross-region message flow works as follows. User A in US-East sends a message to User B in EU-West. User A's message is received by a US-East connection server, processed by the US-East message service, and stored in a US-East message shard. The message service also publishes the message to a cross-region event bus (Kafka with cross-region replication, or a dedicated message relay). The EU-West message service consumes the event, stores a replica in an EU-West shard (for fast local reads when User B loads the conversation), and delivers the message to User B through an EU-West connection server. Cross-region latency adds 100-200ms, but this is a one-time cost per message, and subsequent reads by User B are served from the local EU-West replica.

The message sync protocol ensures clients always have a consistent view of their conversations, even after periods of disconnection. When a client reconnects, it sends its sync state to the server: the last message_id it has seen for each active conversation (or a single global sync cursor if using a sequential event log). The server computes the delta — all messages after the client's last known position — and sends them in batches. This protocol must be efficient: a user who was offline for a week in 100 active group chats could have thousands of unread messages, and the sync must complete within seconds.

End-to-end encryption (E2EE) adds a layer of security where messages are encrypted on the sender device and can only be decrypted on the recipient device. The server stores ciphertext and cannot read message content. WhatsApp, Signal, and iMessage all implement E2EE. The key insight for a system design interview is that E2EE fundamentally changes the architecture: the server cannot index, search, filter, or moderate message content, because it only sees encrypted blobs. This means server-side search, spam filtering, and content moderation must either operate on metadata only or be implemented on the client device.

Finally, operational scaling concerns at billions of messages per day include: monitoring message delivery latency at every hop (sender to server, server to storage, storage ACK, queue to delivery, delivery to recipient), alerting on delivery failure rates per region, capacity planning for storage growth (25 TB/day means 9 PB/year — plan for multi-year retention), and automated scaling of connection server fleets based on concurrent connection counts. Chat systems at this scale require dedicated SRE teams for each major subsystem: connection management, message storage, delivery pipeline, and presence.