Planning launch-day capacity for a checkout API

A marketplace was getting ready for a launch, and its checkout API had to handle three times its usual request load. On paper the answer was short: go from 8 Ready replicas to 24. Most of the work was making sure those 24 replicas would exist when the traffic showed up, and that the rest of the checkout path could keep up with them.

The planning number

We planned around 375 requests a second per Ready replica. That figure was a sizing input, not a measured guarantee. With it, the arithmetic fits in a table:

The word "Ready" carries the table. A replica that's been requested, or scheduled, or has started its container, doesn't serve anything yet. Only pods passing readiness get traffic, so they're the only ones that count toward capacity.

What the autoscaler does

The HorizontalPodAutoscaler changes exactly one thing: the desired replica count on the checkout Deployment. It doesn't resize the other services, add nodes or make a new pod useful any sooner. Between the HPA wanting 24 replicas and 24 pods serving traffic there are three more stages, and we planned each one separately.

Scheduling. The scheduler has to find room on existing nodes that fits each pod's requests and placement rules. If the requests are wrong, pods either don't fit or they fit and then get throttled under load.

Node provisioning. When there isn't room, the node autoscaler adds nodes. That takes time and depends on quota and on the instance types being available. We checked quota well before the launch instead of finding the limit during it.

Startup and readiness. A new pod has to pull its image, start, warm up and pass its readiness probe before the load balancer sends it anything. A probe that passes too early is worse than a slow one, because it routes traffic to a pod that can't handle it yet.

We also kept these stages apart on the dashboards. During a ramp, desired, scheduled and Ready replicas diverge, and a single replicas line hides exactly the gap you need to see.

The parts autoscaling can't touch

A checkout doesn't end at the API tier. Behind it sit a database and an external payment provider, and neither gets faster because there are more API pods. Tripling the replicas triples the number of clients that can hit the database at once. Without a per-replica limit, the extra pods would have moved the overload onto the database.

So we bounded concurrency and queue depth at the API tier. Each replica accepted a limited number of in-flight requests and held a short queue behind them. Past that bound it rejected new work quickly instead of piling it up. A fast, clear rejection under overload is a better experience than a checkout that hangs and then fails, and it keeps the database out of a retry storm.

Database throughput and the payment provider's capacity were separate limits with their own owners. The API tier's job was to stay inside them, not assume them away. That's also why I'd describe this as a 3× increase in request-serving capacity at the API, and nothing more. It isn't a latency guarantee, and it says nothing about how many payments per second the provider would accept.

Load testing set the real ceiling

The 375 figure produced a plan. End-to-end load testing told us whether the plan held. We drove the checkout path through the API, the database and the payment integration together, with a realistic request mix, and watched where it bent first.

We read the results for where the bottleneck sat and whether the concurrency and queue bounds held once traffic reached it. A test against the API tier alone could have looked fine at 9,000 requests a second and told us nothing about the database behind it.

This plan was a separate piece of work from any cost reduction. Capacity for a launch costs money, and some of the headroom we planned for would sit idle on a normal day. That was an accepted tradeoff, owned by the team running the launch.

What I'd tell the next team

Write down which numbers are planning inputs and which are measured. 375 requests a second per replica was the first kind. It was the right number for sizing nodes and checking quota, and the wrong one to quote as capacity until a load test agreed with it.

The API scaled by a factor of three. The database and the payment provider didn't, so the concurrency and queue bounds mattered as much as the extra replicas did.

And test readiness probes under real load before the day. A probe that's honest at idle can still pass too early when a node is busy pulling images and every new pod is starting at once.

Client name and identifying details are left out.