Kubernetes cost optimization: where cost comes from and what to do first

Where the cost comes from

You can set CPU and memory requests and limits for each container. The Kubernetes docs say that the kube-scheduler uses the request to decide which node to place a Pod on. The kubelet also reserves at least the request amount for that container.

So a request holds capacity, even when the container does not use it. A container may use more than its request if the node has spare capacity. But the scheduler only counts the request.

A limit is a ceiling. The kernel enforces it. A container that reaches its CPU limit is throttled. A container that goes over its memory limit may be killed.

This is why cost work starts with requests. High requests can leave capacity unused. Low CPU requests can leave a container with too little CPU when the node is busy. Low memory requests can add to node memory pressure and Pod eviction. CPU limits cause throttling. Memory limits can cause OOM kills.

Step 1: See the cost per namespace and team

You cannot lower a cost you cannot see. Start by giving every workload labels. The Kubernetes docs describe labels as key/value pairs that let you map your own structure, such as environments or tiers, onto objects.

Pick a small set of labels, for example a team label and an app label. Apply them the same way everywhere. Namespaces per team or per environment also help.

OpenCost is an open source project that measures and allocates cloud infrastructure and container costs. It is a CNCF incubating project. Its Allocation API has an aggregate parameter. You can group by namespace, controller, pod, container or a label you name, such as label:app. You can combine groups with commas, for example namespace,label:app.

Write down the top namespaces and workloads. This is your baseline for all later steps.

Step 2: Right-size requests with the Vertical Pod Autoscaler

The Vertical Pod Autoscaler (VPA) computes recommended requests from real usage. Its update mode "Off" never changes Pod resources. The recommender still writes its recommendations into the VPA object. The API reference says this can be used for a "dry run".

First install a VPA version that fits your cluster. Install its CRDs and controllers too. Check that Metrics Server is available. The manifest below assumes the installation is complete.

Create one VPA per Deployment, with updateMode set to "Off". Let it observe the workload over a normal period, including peaks. Then read the result with kubectl describe vpa.

Compare the recommendation with the current request. Change the request in your own manifests, in a normal review, one workload at a time. The largest workloads from step 1 come first.

VPA has other modes (Initial, Recreate, InPlaceOrRecreate, InPlace) that change Pods for you. Start with "Off". Move to another mode only when you have checked what it does to your workload, because some modes restart Pods.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-app-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Off"

Step 3: Scale Pods with the Horizontal Pod Autoscaler

The Horizontal Pod Autoscaler (HPA) changes the number of Pods in a Deployment or StatefulSet to match demand. It adds Pods when load rises and removes them when load falls, but never below the minimum you set. By default the control loop runs every 15 seconds.

For CPU and memory resource metrics, HPA uses the metrics.k8s.io API. Metrics Server usually provides it. Utilization targets compare usage with requests. They need the relevant requests to be set. Absolute-value targets use measured values directly.

HPA can also use custom or external metrics. Review request changes together with any utilization-based HPA targets.

Step 4: Let the nodes follow the Pods

Fewer or smaller Pods only help when the nodes shrink too. Two open source tools do this.

Cluster Autoscaler adds nodes when Pods cannot be scheduled because of insufficient resources. It removes a node that is unneeded: it has low utilization and its important Pods can move elsewhere. By default, "low" means the Pod requests on the node are under 50% of its allocatable resources. A node must stay unneeded for more than 10 minutes before it is removed.

Karpenter watches for Pods that the scheduler marked as unschedulable. It provisions nodes that fit them, and removes nodes that are no longer needed. Use NodePool requirements to constrain instance types, zones and capacity type. Use spec.limits to cap total resources. The current Karpenter docs list the consolidation policies WhenEmpty, Balanced and WhenEmptyOrUnderutilized. Check which ones your installed version supports. Scheduling needs and disruption rules can block a consolidation. consolidateAfter sets how long Karpenter waits before it looks at a node.

Check the disruption settings against your workloads before you turn on consolidation, because consolidation can evict running Pods.

Step 5: Spot nodes and idle environments

Spot (or preemptible) nodes use spare cloud capacity. The cloud can take them back. AWS documents this for EC2 Spot Instances: they suit work that can be interrupted, such as batch jobs and background processing. EC2 sends interruption notices on a best-effort basis. Before a stop or termination, the notice normally comes two minutes ahead. Hibernation starts immediately. Do not rely on getting a warning. Other clouds have their own rules, so read your provider's page.

Put only interruptible work on spot nodes. Keep databases and single-replica services on regular nodes. Allow more than one instance type, so the scheduler has choices.

Idle environments are the second case. Test, staging and preview environments often run all day with no users. Scale their Deployments down when nobody uses them, with kubectl scale and --replicas. Once the Pods are gone, the node autoscaler from step 4 can remove the nodes. Delete preview environments that are no longer needed.

Measure before and after

Change one thing at a time. Before each change, write down the cost per namespace from step 1, the Pod requests, and the node count. After the change, look at the same numbers over a similar period.

Also watch for harm: CPU throttling, OOM kills, Pending Pods, restarts and slow responses. A lower number is not a win if the service got worse.

If a change does not show in the numbers, undo it or leave it. Keep the baseline for the next round.