Otel Keda Example Application

This repository contains an example application demonstrating the use of OpenTelemetry with KEDA for Kubernetes-based autoscaling.

Dependencies

• Helm
• kubectl
• kind
• grpcurl
• Nix (Optional)
• direnv (Optional)

Setup

Environment Setup with Nix and direnv (Optional)

This repository makes use of Nix and direnv to manage the development environment and dependencies. Follow these steps to set them up:

1. Install Nix by following the instructions here.
2. Install direnv by following the instructions here.
3. Navigate to the project directory and allow direnv to load the environment:

direnv allow

Add Helm Repositories

helm repo add grafana https://grafana.github.io/helm-charts
helm repo add kedacore https://kedacore.github.io/charts
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

Create Kubernetes Cluster

kind create cluster --config=kind.yaml --name otel-keda-example

Install supporting Helm Charts

helm upgrade grafana grafana/grafana --namespace grafana --create-namespace --install --values grafana-values.yaml
helm upgrade mimir grafana/mimir-distributed --namespace mimir --create-namespace --install --values mimir-values.yaml
helm upgrade keda kedacore/keda --namespace keda --create-namespace --install --values keda-values.yaml
helm upgrade opentelemetry-collector-daemonset open-telemetry/opentelemetry-collector --namespace opentelemetry-collector --create-namespace --install --values opentelemetry-collector-daemonset-values.yaml
helm upgrade opentelemetry-collector-deployment open-telemetry/opentelemetry-collector --namespace opentelemetry-collector --create-namespace --install --values opentelemetry-collector-deployment-values.yaml

Install sample-app Helm Chart

From local repository

helm upgrade sample-app sample-app/helm --namespace sample-app --create-namespace --install --values sample-app-values.yaml

From GHCR

helm upgrade sample-app oci://ghcr.io/joaoestrela/otel-keda-example/helm-charts/sample-app --version 0.1.1 --namespace sample-app --create-namespace --install --values sample-app-values.yaml

Create Keda Scaled Object

kubectl apply -f sampleAppScaledObject.yaml

Retrieve Grafana Admin Password

kubectl get secret --namespace grafana grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Port Forward Grafana

kubectl --namespace grafana port-forward svc/grafana 3000:9090

Port Forward Sample Application Pods

POD_N=0
kubectl --namespace sample-app get pods -o jsonpath="{.items[$POD_N].metadata.name}" | xargs -I {} kubectl --namespace sample-app port-forward {} $((8080 + POD_N)):8080

Interact with the Sample Application

To trigger an autoscaling above 1 pod, make sure the counter average is above what is set in sampleAppScaledObject.yaml. If you have not changed anything, it should be 10.

POD_N=0
grpcurl -plaintext -d '{"value": 5}' localhost:$((8080 + POD_N)) counter.CounterService/IncreaseCounter
grpcurl -plaintext -d '{"value": 3}' localhost:$((8080 + POD_N)) counter.CounterService/DecreaseCounter

Afterwards you can check the pod count

kubectl get pods --namespace sample-app --no-headers | wc -l

Note

For simplicity's sake, the configured metric is just a basic average at a point in time. It can be changed to any valid Prometheus query.

Project Structure

• grafana-values.yaml: Configuration for Grafana.
• keda-values.yaml: Configuration for KEDA.
• mimir-values.yaml: Configuration for Mimir.
• opentelemetry-collector-daemonset-values.yaml: Configuration for OpenTelemetry Collector DaemonSet.
• opentelemetry-collector-deployment-values.yaml: Configuration for OpenTelemetry Collector Deployment.
• sample-app/: Sample application.
• sample-app/helm/: Helm chart for deploying the sample application.
• sample-app-values.yaml: Configuration for the sample application.