Taking a $2M monthly cloud bill to $400K

The client ran a B2B SaaS platform: 24 services in two production regions, about 240M useful requests a month, and a cloud bill of $2M a month. Nobody on the team thought the platform was wasteful. Every extra replica and every retained object had a good reason when someone added it. The reasons had just stopped being true.

We took the bill to $400K a month at the same traffic. That's an 80% cut, $1.6M a month, or $19.2M a year if demand and prices stay where they were. Most of the money came out of idle compute. The rest came from storage nobody read and network transfer nobody owned.

Where the money went

Interactive APIs, exports and maintenance jobs all shared the same capacity assumptions, even though their deadlines had nothing in common. An export that can finish any time in the next hour was sized like a request that has to answer while a user waits.

When a queue built up, someone raised the replica count. Nobody lowered it afterwards, so yesterday's safety margin became today's baseline. Historical objects sat in hot storage because no one was sure what still read them. Data crossed availability zones constantly, and the transfer charge landed in a shared account that no team looked at.

Finance could see the spending categories. Product teams couldn't connect any of them to a customer request. So before cutting anything, we had to tie each line of spend to a service and a request.

What we did

A baseline with an unallocated bucket

We joined the billing exports to service ownership, environment and workload class. Anything we couldn't attribute went into an explicit unallocated bucket instead of being spread around until the numbers looked tidy. That bucket was big at the start, and shrinking it was a small project of its own.

Then we lined the cost data up with request counts, latency, errors, CPU throttling, memory pressure and queue age over busy periods. We defined a useful request once and left synthetic retries out of the count. Otherwise duplicate work from retries would have made cost per request look better than it was.

Autoscaling per Deployment, separate from nodes

Each Deployment got its own HorizontalPodAutoscaler and explicit CPU requests. That sounds basic, and it was. Without requests the HPA has nothing sensible to compare usage against, and the scheduler packs nodes on guesses.

We kept pod scaling and node scaling as separate loops. The HPA changes a Deployment's replica count; it doesn't add nodes. Pods that can't be scheduled trigger the node autoscaler, and new capacity only counts once those pods pass readiness. GitOps owned the pod spec and the HPA policy but not the replica count, so Git and the autoscaler never fought over it. Where CPU didn't describe the deadline, as with batch workers, we scaled on queue depth or backlog age instead.

Lifecycle rules after a dry run

Storage changes started with an inventory and a dry-run report, never with deletion. Owners confirmed that restores and retention obligations still worked before any lifecycle rule went live. For large objects we changed producers to pass references through queues instead of copying the payload at every hop. That cut memory pressure and transfer at the same time.

One workload class at a time

We started with noncritical batch jobs and compared completed work, queue age and spend against an unchanged control group. Interactive services came later, through canaries with a fixed replica floor, because a good batch result tells you nothing about tail latency on the request path.

If a guardrail tripped, the previous capacity policy came back and the next reduction waited. Missing telemetry counted as a tripped guardrail. We kept the old worker pool around until restores had been tested against the new setup.

A monthly bridge finance could check

Every month we published a bridge from the baseline bill to the current one. Volume changes, rate changes and architecture changes sat in separate columns, with one-off migration costs annotated. That stopped anyone from booking a low-traffic month as a saving, or counting spend that had moved to another account. Engineering and finance worked from the same sheet.

What changed

The bill went from $2M to $400K a month with useful traffic flat at 240M requests.

Compute was most of it, which matches what the baseline showed: replicas sized for a peak that had passed, on nodes sized for those replicas. Storage fell once cold history moved off hot tiers. Network fell once cross-zone traffic had an owner who could ask why it existed.

Some costs stayed on purpose. Critical interactive traffic kept an on-demand capacity floor that's paid for whether it's busy or not, and archive retrieval got slower. A named owner signed off on each of those tradeoffs. A spend reduction also isn't the same as booked cash: existing commitments and migration labor sit outside these numbers.

What I'd keep in mind next time

Apply commitment discounts after right-sizing, not before. If you buy reserved capacity for the inflated footprint, you've locked in the waste and made every later cut look smaller than it is. We folded any purchasing effect into the compute figure and didn't count it a second time.

Don't stack percentages against the original bill either. If right-sizing takes a share off compute and a discount then takes a share off what's left, you can't add the two percentages and quote the total against where you started. Each saving has to be measured against the bill as it stood after the previous one, which is why the monthly bridge kept those changes in separate columns.

Client name and identifying details are left out.