Karpenter for EKS Cost Optimization: A Deep Dive with Real Config

1. The Baseline Problem

The cluster ran on three static m5.xlarge On-Demand nodes with no autoscaling. Nodes were sized for peak-year traffic 24/7, keeping baseline CPU utilization at 18% to 22%.

Legacy Cluster Autoscaler could not be enabled safely because its 4- to 5-minute provisioning delay caused checkout pods to queue in Pending during burst surges. To prevent customer-facing dropouts, the team carried permanent idle over-provisioning.

2. What Karpenter Actually Changes

Karpenter is not just a tool for buying cheaper compute. It replaces static Auto Scaling Groups with a direct control loop:

• Direct EC2 Fleet integration: Evaluates pending pod requests and launches exact instance types in under 90 seconds without waiting for ASG scaling activities.
• Intelligent bin-packing & multi-family fallback: Diversifies across m5.xlarge, m6i.large, and c6i.xlarge across multiple availability zones.
• Aggressive idle cleanup: With ttlSecondsAfterEmpty: 30, nodes terminate within 30 seconds of becoming empty, eliminating 24/7 idle waste.

3. The Production Config

Here is the Karpenter provisioner configuration deployed for the cluster:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot", "on-demand"]
    - key: node.kubernetes.io/instance-type
      operator: In
      values: ["m5.xlarge", "m6i.large", "c6i.xlarge"]
    - key: topology.kubernetes.io/zone
      operator: In
      values: ["us-east-1a", "us-east-1b"]
  weight: 100
  limits:
    resources:
      cpu: "64"
      memory: 256Gi
  providerRef:
    name: default
  ttlSecondsAfterEmpty: 30

To pin critical authentication and payment lock workloads to non-interruptible capacity, deployment manifests define an explicit nodeSelector:

# deployment-payment-service.yaml (excerpt)
spec:
  template:
    spec:
      nodeSelector:
        karpenter.sh/capacity-type: on-demand

4. Cost Math: Line-by-Line Breakdown

The combined infrastructure remediation produced the following measured cost impact:

Note: Spot prices reflect us-east-1 6-month trailing average at time of engagement; actual savings vary based on spot pool depth and AZ availability.

5. What This Does Not Solve

Karpenter is not a universal fix for every workload:

• Spot interruptions on stateful workloads: Stateful services with persistent volume attachments should never run on Spot without strict replication architectures.
• GPU & ML model inference: LLM serving requires dedicated accelerator instances, specialized device drivers, and different provisioner constraints, as detailed in my Custom LLM Serving on Bare-Metal A100s breakdown.
• Applications with slow graceful shutdown: Workloads requiring more than 2 minutes to drain will get forcibly terminated by AWS Spot notices unless handled with pre-drain hooks.

6. Is This "Just Reading the Manual"?

Karpenter documentation is thorough and well-written. The challenge was never copy-pasting a provisioner manifest; it was diagnosing which manuals were relevant across multiple overlapping failure modes:

The team was dealing simultaneously with JVM off-heap memory exhaustion (512MB heap in a 512MB container limit), misconfigured liveness probes that killed restarting pods during 45s warmup windows, and network egress surcharges from dead NAT gateway routes. Making Karpenter effective required concurrent remediation across the JVM memory model, Kubernetes probe lifecycles, and AWS VPC routing under production SLA constraints.

7. Frequently Asked Questions

Q: How does Karpenter reduce EKS scaling latency?
A: Karpenter provisions nodes on demand from a weighted, multi-AZ instance pool — under 90 seconds here vs 4–5 minutes with a managed node group, eliminating queue/Pending build-up on spikes.

Q: Is Karpenter cheaper than EKS Cluster Autoscaling?
A: Primarily yes — Karpenter bin-packs + favors spot + terminates idle nodes (ttlSecondsAfterEmpty) rather than treating whole node groups as units. In this engagement it cut the compute bill ~70%.

Q: Does spot autoscaling hurt reliability?
A: Not by itself if you pin stateful/critical services (payment, auth) to On-Demand via node-selector and use Spot only for bursty, interruptible work. Mixed provisioner covers both.