LLM inference
Open models on GPUs with vLLM or SGLang. I measure first, then tune speculative decoding, precision, batching and node shape.
AI infrastructure engineer · San Francisco
I make LLM inference faster, and the platforms around it boring.
I'm Zac Peterson. Since 2019 I've built and run cloud platforms for companies in healthcare, energy, retail and marketing, nearly all of it on Kubernetes.
I also consult. Most projects land in one of three places: a model that's too slow or too expensive to serve, an ML platform that needs building or untangling, or coding agents that need somewhere safe to work.
Numbers from real work
22.25 tok/sGLM-5.1 on eight H200s, one stream, after tuning speculative decoding$2M → $400Ka client's monthly cloud bill at the same traffic, after we automated its Kubernetes workloads51 → 17 minthe same 38 CI jobs, run one after another versus on four workers in my k3s build fleet4,752model-and-task cells in my coding-agent benchmark, all run as Kubernetes jobs
What I work on
ConsultingML platforms
GPU node pools, job queues, MLflow and GitOps on GKE, EKS or bare metal, kept small enough that your team can run it.
Agent sandboxing
Credential-free sandboxes for coding agents, plus the benchmarks that tell you whether they did the job.
Selected work
All workClient work
Taking a $2M monthly cloud bill to $400K
Same traffic, a fifth of the bill. Most of the money was idle capacity, hot storage nobody read and data crossing zones for no reason.
- Kubernetes
- HorizontalPodAutoscaler
- Metrics Server
- Node autoscaling
- GitOps
Client work
From 20,000 to 20 million daily users on EKS
A thousand times more people, and a platform where one part can change without stopping the rest.
- Amazon EKS
- Kubernetes
- HorizontalPodAutoscaler
- Node autoscaling
- PodDisruptionBudgets
My own project
Running GLM-5.1 on a rented 8× H200 node
One node, one model, one user. The setup that made it repeatable.
- vLLM 0.19
- GLM-5.1 FP8
- DeepGEMM
- Prometheus
- Pushgateway
My own project
A Kubernetes operator for coding-agent benchmarks
CRDs for profiles and runs, a scheduler that checks headroom, and a database that remembers every attempt.
- Go
- kubebuilder
- controller-runtime
- Kubernetes
- PostgreSQL
Writing
All writing-
Tuning GLM-5.1 on eight H200s: three speculative tokens beat two by 29%
Two speculative-decoding settings and a CUDA graphs run, measured on one stream. Three draft tokens won by 29%, and the acceptance rates I didn't save are the gap in the data.
-
Running 38,150 coding-agent benchmark attempts on Kubernetes
Most of what I learned running coding-agent benchmarks at scale was about bookkeeping, not GPUs or prompts.
-
Our dev database backups were zero bytes for four days
The job ran on time, the files had the right names, and none of them could be restored.
-
Git hooks change PATH, and our gate cache missed 97% of the time
A validation cache keyed on the environment looked fine when I tested it by hand. Inside a Git hook, the environment was different.
Background
Full experience- Healthcare company · Staff DevOps Engineer, AI/ML infrastructure2025–present
- Global energy company · Staff DevOps Architect, AI/ML infrastructure2024
- Consumer apparel company · Senior Cloud Architect2023–2024
- Digital marketing company · Senior DevOps Engineer2021–2022
- Cloud consultancy · Managed Services Practice Lead (joined as Cloud Engineer)2019–2021
Get in touch
If something here sounds like your problem, send me a few lines about what you're running.