Getting started

Developing a Locust Pod Autoscaler is designed to be simple and utilise the existing and mature Locust load testing framework.

In this guide we will create a Locust Pod Autoscaler that will be responsible for scaling a Fibonacci calculating service.
The Fibonacci service will expose an endpoint /fibonacci which accepts an integer query parameter n and returns the Fibonacci number at that position.
For example GET /fibonacci?n=7 would return 13.

The autoscaler will call this /fibonacci endpoint with the value n=10 with 10 users for 5 seconds. The latency values will be collected and evaluated.
The autoscaler will use these latency values to compare to an evaluation configuration, which will try to keep the mean latency below 7 ms.
The autoscaler will use a decay, causing the replica count to be reduced by 1 replica every 3 runs in which there is no scaling change.

The finished and full code can be found in the example in the locust-pod-autoscaler repository.

Set up the development environment

Dependencies required to follow this guide:

• Python 3
• Docker
• Minikube or another Kubernetes cluster
• kubectl

Enable Custom Pod Autoscalers on your cluster

Using this Locust Autoscaler requires Custom Pod Autoscalers (CPAs) to be enabled on your kubernetes cluster, follow this guide to set up CPAs on your cluster.

Create the project

Create a new directory for the project locust-example-autoscaler and begin working out the project directory.

mkdir locust-example-autoscaler

Create the Fibonacci application

Create an app folder.
Create the following files inside the app folder:
api.py:

from flask import Flask, abort, request
import json
app = Flask(__name__)

@app.route("/fibonacci")
def fibonacci_endpoint():
    nVal = request.args.get("n")
    try:
        n = int(nVal)
        return str(fibonacci(n))
    except ValueError:
        abort(400, f"Value '{nVal}' is not an integer")

def fibonacci(n): 
    # First Fibonacci number is 0 
    if n==1: 
        return 0
    # Second Fibonacci number is 1 
    elif n==2: 
        return 1
    else: 
        return fibonacci(n-1)+fibonacci(n-2) 

if __name__ == "__main__":
    app.run(debug=True, host="0.0.0.0")

Dockerfile:

FROM python:3.6-slim
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
# Install dependencies
RUN pip install -r requirements.txt
# Copy in source files
COPY . /app
ENTRYPOINT [ "python" ]
CMD [ "api.py" ]

requirements.txt:

Flask==1.0

deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fibonacci
  labels:
    app: fibonacci
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fibonacci
  template:
    metadata:
      labels:
        app: fibonacci
    spec:
      containers:
      - name: fibonacci
        image: fibonacci:latest
        ports:
        - containerPort: 5000
        imagePullPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: fibonacci
spec:
  ports:
  - port: 5000
    protocol: TCP
    targetPort: 5000
  selector:
    app: fibonacci
  type: ClusterIP

Build and deploy the Fibonacci application

• Switch to the Minikube Docker registry:
  eval $(minikube docker-env)
• Inside the app folder build the Docker image of the application:
  docker build -t fibonacci .
• Deploy the application using kubectl:
  kubectl apply -f deployment.yaml

Write the Locust test

Now we have an application running we can manage, lets write our load test in Python using Locust:

from locust import HttpLocust, TaskSet, task, between

class UserBehavior(TaskSet):
    @task(1)
    def profile(self):
        self.client.get("/fibonacci?n=10")

class WebsiteUser(HttpLocust):
    task_set = UserBehavior
    wait_time = between(5, 9)

This simple Locust test will call the GET /fibonacci?n=10 endpoint.

Write the evaluation configuration

Lets write our evaluation configuration:

targets:
  - method: "GET"
    endpoint: "/fibonacci?n=10"
    type: "mean"
    target: 7
decay: 
  replicas: 1
  unchangedRuns: 3

This configuration is in two parts; targets and decay.
The targets configuration defines the GET /fibonacci?n=10 endpoint, and states that the result mean average latency should be kept below 7ms.
The decay configuration defines that the decay will occur every 3 runs without scaling change, and the replica count will be reduced by 1.

Write the Dockerfile

Lets pull the code we have written together into a Docker image with our Dockerfile:

FROM jthomperoo/locust-pod-autoscaler:latest

ENV locustFilePath=/locustfile.py evaluationConfigFilePath=/evaluation_config.yaml

ENV locustRunTime=5 locustUsers=10 locustHatchRate=10

ADD locustfile.py evaluation_config.yaml /

This Dockerfile:
* Sets file paths of Locust test file and the evaluation configuration. * Sets some Locust values; the load testing will run for 5 seconds, with 10 users that are spawned at a rate of 10 a second. * Adds in locust file and evaluation configuration we have created.

Write the deployment YAML

Now we have our Docker image and all our autoscaler created, we just need to create a deployment file for the autoscaler:

apiVersion: custompodautoscaler.com/v1alpha1
kind: CustomPodAutoscaler
metadata:
  name: locust-autoscaler-example
spec:
  template:
    spec:
      containers:
      - name: locust-autoscaler-example
        image: locust-pod-autoscaler-example:latest
        imagePullPolicy: Always
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: fibonacci
  config: 
    - name: interval
      value: "10000"
    - name: locustHost
      value: "http://fibonacci:5000"

This YAML outlines the name of our autoscaler instance, the image/pod definition for our autoscaler using template, the resource we are targeting to manage using scaleTargetRef and some basic extra configuration with interval set to 10000 milliseconds, meaning the autoscaler will run every 10 seconds and the locustHost defined as http://fibonacci:5000.

Build and deploy the autoscaler

• Target the Minikube registry for building the image:
  eval $(minikube docker-env)
• Build the example image.
  docker build -t locust-pod-autoscaler-example .
• Deploy the Locust Autoscaler using the image just built.
  kubectl apply -f locust_autoscaler.yaml
  Now the Locust Autoscaler should be running on your cluster, managing the app we previously deployed.

Test the autoscaler

Watch the autoscaler's logs with:

kubectl logs locust-autoscaler-example --follow

Increase the load with:

kubectl run --generator=run-pod/v1 -it --rm load-generator --image=busybox /bin/sh

Once a prompt appears:

while true; do wget -q -O- http://fibonacci:5000/fibonacci?n=23; done

This will repeatedly call the fibonacci service with the value 23; this should increase load and the number of replicas should increase.
Once the replica count increases, try stopping the increased load and check the decay works, it should wait for 3 unchanged load tests before scaling down by 1, it should repeat this until it reaches a stable state.

Clean up

Run these commands to remove any resources created during this guide:

kubectl delete -f app/deployment.yaml

Removes our managed deployment.

kubectl delete -f locust_autoscaler.yaml

Removes our autoscaler.

VERSION=v0.5.0
curl -L "https://github.com/jthomperoo/custom-pod-autoscaler-operator/releases/download/${VERSION}/cluster.tar.gz" | tar xvz --to-command 'kubectl delete -f -'

Removes the Custom Pod Autoscaler operator.

Conclusion

Congratulations! You have now successfully created a Locust Pod Autoscaler.