When to go serverless, when not to, cold starts, state management, and the real cost model behind functions-as-a-service.
When to go serverless, when not to, cold starts, state management, and the real cost model behind functions-as-a-service.
Lesson outline
"Serverless" is a terrible name. There are absolutely servers. You just do not manage them.
The mental model that actually helps: serverless is an execution model where you provide code, and the platform handles provisioning, scaling, and billing at the granularity of individual function invocations. You pay per millisecond of execution, not per hour of a running server.
What serverless actually means
Serverless = no server management (provisioning, patching, capacity planning). The platform auto-scales from zero to millions, handles infrastructure failures, and bills per invocation. AWS Lambda, Google Cloud Functions, Azure Functions, and Cloudflare Workers are the major platforms.
The key insight: serverless flips the cost model. Traditional compute charges you for idle time (an EC2 instance costs the same whether it handles 1 req/s or 1,000 req/s). Serverless charges you for actual work done.
| Use case | Serverless? | Why |
|---|---|---|
| Event-driven processing (S3 upload → resize image) | Yes ✅ | Sporadic, stateless, short-duration — perfect fit |
| API with unpredictable bursty traffic | Yes ✅ | Auto-scales from 0 to 10k concurrent instantly, no pre-provisioning |
| Scheduled batch jobs (nightly report, cleanup) | Yes ✅ | No idle cost between runs; cron triggers built in |
| Steady high-traffic API (>1M req/day constant) | Maybe ❌ | Reserved EC2 + container may be 70% cheaper at sustained load |
| Long-running jobs (>15 min) | No ❌ | Lambda max timeout is 15 min; use ECS Fargate or batch instead |
| Real-time WebSocket connections | Tricky ❌ | Lambda WebSocket via API Gateway works but cold starts hurt UX |
| ML model inference (large models) | No ❌ | Cold start loading a 2GB model is 30+ seconds — use provisioned containers |
The serverless sweet spot
Event-driven, stateless, variable-load workloads with clear invocation boundaries. Think: webhooks, image/video processing, API backends for mobile apps, data transformation pipelines, scheduled tasks.
A cold start happens when a Lambda function is invoked but no warm instance exists. The platform must: download the deployment package, start a container, initialize the runtime, and run your init code — before it can process the request.
| Runtime | Typical cold start | Notes |
|---|---|---|
| Node.js (zip) | ~200–400ms | Fast. Minimal init overhead. |
| Python (zip) | ~200–500ms | Fast. Popular for data processing. |
| Java (zip) | ~1–3s | JVM startup is slow. GraalVM native image helps. |
| Container image | ~1–5s | Image pull adds significant overhead on first cold start |
| Node.js with Provisioned Concurrency | <10ms | Pre-warmed — eliminates cold starts at a cost (you pay for idle instances) |
Strategies to minimize cold start impact
Your Lambda function processes user authentication requests and needs sub-100ms P99 latency. Cold starts are causing 2-3s spikes. What is the best solution?
Lambda functions are stateless by design. Each invocation gets a fresh execution context (except for the optimization where AWS reuses a warm instance). You cannot store state in memory between invocations.
BAD: Storing state in Lambda memory
let requestCount = 0; // This resets on every cold start and is not shared across concurrent instances. At scale, you have 500 Lambda instances each with requestCount = 1. The real count is lost.
GOOD: Externalize all state
User sessions → ElastiCache (Redis). Counters/rate limits → DynamoDB atomic increments. File uploads → S3. Job state → SQS/Step Functions. The function only transforms data; state lives in managed services.
State storage patterns for serverless
The promise: "Pay only for what you use." The reality: at high sustained throughput, serverless can cost 5–10× more than equivalent container capacity.
The breakeven point calculation
Lambda: 1M requests/month × 100ms average × 512MB = ~$2.08. EC2 t3.small: $15/month handles the same load easily. At low volume, Lambda wins. At 100M requests/month, that same Lambda costs $208 vs $15 for EC2. Do the math before choosing.
Rule of thumb: serverless is economically optimal for workloads with significant idle time or unpredictable burst patterns. For steady, predictable high-volume traffic, containers or reserved compute beat serverless on cost.
Cloud and backend architecture interviews — often used to assess whether candidates understand execution models beyond just "it scales automatically."
Common questions:
Key takeaways
A Lambda function serving your homepage has occasional 3-second latency spikes. What is likely causing this and how do you fix it?
Cold starts — when no warm instance exists, the platform initializes a new one. Fix with Provisioned Concurrency (pre-warmed instances) for user-facing latency-sensitive functions.
Ready to see how this works in the cloud?
Switch to Career Paths for structured paths (e.g. Developer, DevOps) and provider-specific lessons.
View role-based pathsSign in to track your progress and mark lessons complete.
Questions? Discuss in the community or start a thread below.
Join DiscordSign in to start or join a thread.