MidSystem Design

Design a log and metrics ingestion pipeline for a platform with about 200 services. Engineers need to search logs and alert on metrics.

What they are really testing: Whether you separate the two very different workloads, logs and metrics, instead of dumping both into one expensive store.

A real interview question

Design a log and metrics ingestion pipeline for a platform with about 200 services. Engineers need to search logs and alert on metrics.

What most people say

drag me

I would send everything to Elasticsearch and build Kibana dashboards on top of it.

It names one product instead of sizing the load, and it treats metrics and logs as one problem, which is exactly how teams end up with a storage bill nobody can explain.

The follow-ups they ask next

  • Log volume triples overnight because someone turned on debug logging. What breaks first?

    Name the ingestion buffer filling and the index tier throttling. Talk about per service rate limits and sampling as the control.

  • Why not just use CloudWatch Logs for all of it?

    Say you would for a small estate. At 2 terabytes a day the per gigabyte ingestion charge dominates, so give the rough math rather than a taste opinion.

  • How do you keep the metrics path alive when the logs path is on fire?

    Separate agents, separate transport, separate quotas. Alerting must never depend on the log store.

What the interviewer is listening for

  • Sizes the load in gigabytes per day before naming a single service
  • Separates metrics from logs and gives each a different retention
  • Uses a local agent buffer as the anti data loss mechanism
  • Names the hot versus cold storage trade-off in cost terms

What sinks the answer

  • Puts everything in one search cluster and never mentions cost
  • Cannot say roughly how much data the system handles
  • Assumes the ingestion path never fails
  • Lists five AWS services without connecting them into a flow

If you genuinely do not know

Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.

I have not run a pipeline at this size, but here is how I would reason about it. First I would ask how many gigabytes a day and how far back people actually search, because that decides almost everything else.

Keep going with system design

All 336 cloud engineer questions

Knowing the answer is not the same as recalling it under pressure

Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.

Start free