From 20,000 to 20 million daily users on EKS

The business grew from 20,000 daily users to 20 million while we moved it from one large application to microservices on Amazon EKS. That's a 1,000× increase. Kubernetes didn't create any of that demand. The platform's job was to keep checkout, accounts and the release process working while it arrived.

The starting point

Every change shipped in one large coordinated release. A slow dependency in one feature could eat capacity meant for everything else, and scaling meant scaling the whole application whether or not the busy part needed it. That works at 20,000 users a day. It doesn't hold at a thousand times that.

We split out only the busiest capabilities, so they could scale and deploy on their own. Swapping one big release for a distributed system nobody owned would have been a bad trade.

Turning users into a workload contract

A daily-user count doesn't size anything by itself. So the first job was turning the audience number into something we could size against. We measured the request mix, cache behavior, payload sizes, write volume, hot tenants and retries.

At the final audience, 30 requests per user-day works out to about 6,944 requests a second on average. Busy intervals ran at 8× that, so the peak we planned for was about 55,556 requests a second. At 250 requests a second per Ready API replica, that peak needs at least 223 Ready replicas before any headroom.

The 223 was a floor for the API tier, not a node count or a benchmark result. The database, the network, external providers' quotas and failure behavior all still needed load tests with the real request mix.

Splitting services where it paid off

We didn't split everything. We put a routing boundary in front of the existing application and pulled services out one at a time, picking the ones where independent change, failure isolation or scaling justified the cost of running another service. Each got an owner, a versioned contract and a recovery path. A service that still reads and writes the old application's tables isn't separate, so data ownership moved with the code.

Old and new paths ran side by side through staged traffic cohorts, and schema changes used expand-and-contract so both versions could read the data. Anything with side effects needed explicit idempotency. A retried payment or order must never become a second one.

Making EKS follow demand

We treated scaling as four separate stages, because each one fails differently. The HPA sets desired replicas. The scheduler has to find room for them. Node provisioning adds room when there isn't any. Startup and readiness decide when a new pod can take traffic. The dashboards showed all four, so a gap between desired and Ready replicas was visible instead of averaged away.

We picked one node-scaling mechanism and owned it rather than running competing controllers. Critical services kept a capacity floor and spread across availability zones. Queues were bounded. Spot capacity ran only work that could be interrupted and safely retried.

Keeping failures local

Concurrency and retries were capped between services, so one slow dependency couldn't amplify into a retry storm. Caches had explicit freshness and authorization rules. Under pressure the platform shed optional work first. Recommendations could drop off a page during an outage; a failed payment authorization must never be reported as a success.

We tested a bad release, a slow dependency, the loss of an availability zone and a data restore. PodDisruptionBudgets, spare replicas and a healthy load balancer helped in every case. None of them proved that a customer's order went through, so we measured the transactions themselves.

Smaller releases

Releases moved from one coordinated event to immutable artifacts promoted per service, through contract checks and a bounded rollout, with errors and latency tied to the release that caused them. At about 40 releases a week, manual work per release fell from 45 minutes to 10. That's 101.03 hours a month of engineering time back.

What the extra releases bought was shorter feedback loops for fixes and experiments, and a rollback that restored a compatible release instead of assuming an older binary could undo every schema change.

Results

The cost comparison holds the final audience and workload constant: $1.2M against $780K a month for the same demand. That's 35% less, $420K a month or $5.04M a year. Both figures price the 20-million-user workload; neither is scaled up from the early bill. The difference came from better resource, storage and transfer efficiency at scale.

The availability figures are objectives, not measured uptime. Moving from 99.5% to 99.95% cuts the monthly downtime allowance from 216 minutes to 21.6, but the design has to earn that through drills. A short outage at peak can also hurt more than a longer one overnight, which a percentage won't show.

Business-level transaction checks belonged in place before the first service split, not alongside the later ones. A service can look healthy on CPU and error rate while orders fail somewhere downstream, and resource dashboards won't tell you.

Client name and identifying details are left out.