Pitfalls to Avoid When Using Spot VMs in GKE for Cost Reduction

Google Kubernetes Engine (GKE) offers on-demand and spot VMs for running user workloads (pods, deployments, and daemon sets), providing a low-cost solution to run specific workloads within your Kubernetes cluster. Spot VMs are unused Compute Engine instances within a cloud provider's data center available at a significant discount. Spot VMs are an excellent way to save money, and this ensures that business operations continue uninterrupted. Pods that are scheduled to run in an on-demand instance due to spot unavailability at a specific point in time will continue to operate there even if spot VMs become available later.

What are Spot VMs in GKE?

Spot VMs are Compute Engine virtual machine (VM) instances that are less expensive than conventional Compute Engine VMs but offer no guarantee of availability. Spot VMs provide the same machine types and configurations as normal VMs. Spot VMs can be utilized in your clusters and node pools to execute stateless, batch, or fault-tolerant workloads that can withstand disruptions due to their transitory nature.

How do Spot VMs work in GKE?

• When you create a cluster or node pool using Spot VMs, GKE generates underlying Compute Engine Spot VMs that act similarly to managed instance groups (MIGs).
• Nodes that use Spot VMs act similarly to normal GKE nodes but with no guarantee of availability. When the resources used by Spot VMs are required to run normal VMs, Compute Engine terminates them so that the resources can be used elsewhere.
• When Compute Engine wants to reclaim the resources utilized by Spot VMs, it sends a termination notification to GKE.
• Spot VMs will terminate 30 seconds after getting a termination notice.

Why are Spot VMs risky if misused?

Using Spot VMs is like booking a cheap flight ticket with a condition if the airline gets full, you’re the first to lose your seat. Spot VMs are low-cost virtual machines that Google can take back anytime when others need the resources. They help save money but don’t guarantee availability.

If your app is running on a Spot VM and it suddenly disappears, it can cause problems. Your app might crash, lose data, or stop working. That’s why Spot VMs should only be used for tasks that can handle interruptions, like temporary jobs or apps that can restart without issues.

Pitfalls to Avoid When Using Spot VMs in GKE for Cost Reduction

Below are some pitfalls to avoid when using spot VMs in GKE for cost reduction

1. Choose your spot VMs

When choosing a VM location, choose the slightly less popular options. It's simple: they're less likely to be interrupted. Once you've decided on one, check the frequency of interruptions. This is the rate at which this instance reclaimed capacity over the last month.

2. Unpredictable Availability

Google Cloud can preempt spot VMs at any moment as a result you need to design fault-tolerant workloads. Employ workload types—such as stateless apps or those with fast failure recovery that can withstand interruptions.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: my-app-pdb
spec:
  minAvailable: 50%
  selector:
    matchLabels:
      app: my-app

3. Check your storage transfer limitations

Data storage is an important part of GKE cost optimization because each application has different storage requirements. Check that the machine you chose has the storage throughput your workloads require. Also, avoid expensive drive solutions like premium SSDs unless you intend to use them to their full potential.

4. Use groups

To increase your chances of acquiring the spot machines you require, create groups of spot instances that request various machine kinds at the same time (managed instance group). When more resources become available, managed instance groups generate or add new spot virtual machines.

5. Consider instance weighted scores

When autoscaling, use instance weighting to determine how much of your resource pool to allocate to a specific task. This is how you ensure that the machines you design are optimal for the task at hand.

6. Ensure HPA and VPA policies don’t clash

The Vertical Pod Autoscaler automatically changes the number of pods and restrictions depending on a target average CPU use, minimizing overhead and lowering costs. The Horizontal Pod Autoscaler seeks to scale out rather than up.

So, make sure the VPA and HPA policies don't conflict with one another. When creating clusters for the business or purpose-class tier of service, double-check your binning and packing density settings.

Deploy Workloads on both Spot and On-Demand

Step 1: Create Node Pools

First, Utilizing Spot VMs, run the below command to establish a node pool:

gcloud container node-pools create spot-pool \
    --cluster=your-cluster-name \
    --zone=your-cluster-zone \
    --num-nodes=1 \
    --preemptible \
    --node-taints=spot=true:NoSchedule \
    --node-labels=type=spot

Output:

Step 2: Create a Node Pool with On-Demand VMs

To construct a node pool with On-Demand virtual machines, use the following command.

gcloud container node-pools create on-demand-pool \
    --cluster=your-cluster-name \
    --zone=your-cluster-zone \
    --num-nodes=1 \
    --node-labels=type=on-demand

Output:

Deploy in Standard when Spot is Unavailable

Step 1: Deploy Workloads

Make the following file, deployment. yaml, and save it.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      tolerations:
      - key: "spot"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
      containers:
      - name: my-container
        image: gcr.io/your-project-id/my-image:latest

Step 2: Apply the Configuration

Use kubectl to apply the PDB configuration and deployment.

kubectl apply -f deployment.yaml

Output:

Step 3: Testing the Setup

Lastly, you must make sure the pods are scheduled to execute on spot VMs while available and run on on-demand VMs if spot VMs become unavailable.

kubectl get pods -o wide

Output:

Best Practices for Spot VMs in GKE

• Before adding a GPU node pool that uses Spot VMs, be sure that your cluster already has at least one non-GPU node pool that uses regular VMs.
• Spot VMs provide no availability guarantees. Design your systems with the expectation that GKE may reclaim some or all of your Spot VMs at any time, with no certainty on when new instances will become available.
• To ensure that your workloads and Jobs are processed even when no Spot VMs are available, split your clusters into node pools that use Spot VMs and node pools that use normal Compute Engine VMs.
• Use node taints and tolerations to avoid scheduling essential Pods on node pools that use Spot VMs.

Conclusion

In this article, we have learned about Spot VMs in GKE for Cost Reduction. Spot VMs are ideal for fault-tolerant workloads since Compute Engine can preemptively stop them at any time. As a result, they have no availability guarantee and may be paused in favor of more important requests, such as when another GCP user asks for an on-demand instance.