ML platforms your team stops thinking about

What goes into one

• GPU and CPU node pools with autoscaling.
• Queues and capacity-aware scheduling, so a training job waits for room instead of evicting someone else's.
• Experiment tracking and a model registry (MLflow), with pipelines in Airflow or Kubeflow.
• GitOps with Argo CD or Flux, and Terraform for the cloud underneath.
• Observability: Prometheus, Grafana, Loki and OpenTelemetry.
• Cost controls: requests and limits that match real use, idle capacity that goes away, and storage tiers for data nobody reads.

I start from what people run each week, not from a reference architecture. A team that trains twice a month on one GPU node needs a different platform from one serving models to production traffic. Build the second for the first and that team ends up maintaining controllers nobody uses.

Where I've done it

At a global energy company I took model deployment from weeks to hours with Kubernetes, Kubeflow and MLflow, and cut infrastructure costs by 40% with a Google Cloud and Kubernetes redesign.

For clients, we moved a data-science team from Azure VMs to Kubernetes and MLflow on two clouds, grew an online business to 20 million daily users on EKS, took a $2M monthly cloud bill to $400K at the same traffic, and planned launch-day capacity for a checkout API.

On my own time I run CI for a product on a five-node BuildKit fleet in bare-metal k3s, which is where I try scheduling and caching ideas before I suggest them to anyone else.