Helm

Understanding and Building Helm Chart Tests

Mumshad Mannambeth

Jan 31, 2022 • 7 min read

No matter how good the chart is, successful installation in a new cluster is not guaranteed. Additionally, a chart that works in one cluster may fail to install correctly in another because a necessary feature is lacking or there are not enough resources. That’s why we need chart tests.

In this blog, you’ll learn what chart tests are, how to create them, and how to run them.

What is a Helm Chart Test?

A Helm chart test, located in the templates directory, is an implementation that validates the functionality and correctness of a Helm chart. Chart testing involves running a series of tests on the chart to ensure that all the components are working as expected and that the chart can be successfully deployed to the desired environment. This type of testing is important for ensuring that the Helm chart is production-ready and can be safely used in a live environment.

How Helm Chart Tests Work

A test file describes a container with a given command, which returns an exit code after being executed. If the container exits with (exit 0), the test is considered a success, meaning the charts are ready for installation. The test is considered failed if the code returned is any other number.

Below are some of the scenarios we can test with chart tests:

A test that tries to log in to the database server with the username and password supplied in the values.yaml file. This way, we verify that both the username and password were set correctly and also that the database server is fully functional.
Test if the web server is working correctly by trying to establish a connection to it.
Make a specific request to a web application and check that it returns a valid answer. For example, say you installed some website that has a special API. You can run a command to send a specially crafted HTTP POST request requesting the website’s version. If the API is working, it will return a valid answer. If it is not working, there will be no answer. We can, for instance, use a curl command for such a purpose.

Building a Helm Chart Test

In this section, we shall install a Nginx chart, create a test for it and run the test.

Prerequisite

To follow along in this tutorial, you'll need access to a running Kubernetes cluster. If you don’t have access to one, you can use a tool such as minikube to set up a Kubernetes cluster. Also, you’ll need to have kubectl and Helm installed on your local machine.

You can also use KodeKloud's Helm Playground. With this playground, you won't need to go through the hassle of installing any additional software— everything you need is already set up and ready to use.

Pull a Helm Chart

We'll use of Nginx chart in this section. We'll pull it, then install it and create tests for it. To pull the chart, first, add a repo using this command.

helm repo add bitnami https://charts.bitnami.com/bitnami

Let's now pull the chart we'll be using. Navigate to the directory where you want to place the project and then run this command:

helm pull bitnami/nginx

Extract the content of the nginx.tgz file downloaded and navigate to the directory containing the values.yaml. From there, run the helm install as shown below:

helm install -f values.yaml my-website .

The command above creates a release, my-website, from the nginx chart.

Create a Chart Test

The test we’ll write for our chart is a regular template file placed in the templates/ directory. To ensure that Helm doesn’t treat the test like a regular template file, we add the following line in the file’s contents in the metadata section:

helm.sh/hook: test

You can build one or multiple tests and place them anywhere in the templates/ directory. But for more clarity and better structure, you should create a subdirectory called tests and place them all in templates/tests. Grouping them this way makes it easier to differentiate between your regular template files and your tests.

Let’s create this subdirectory.

mkdir ~/nginx/templates/tests

And now, let’s create our first test. Create a file named test-nginx.yaml in the subdirectory created in the previous step using this command:

nano ~/nginx/templates/tests/test-nginx.yaml

Paste the following code into the file.

apiVersion: v1
kind: Pod
metadata:
  name: "{{ .Release.Name }}-nginx-test"
  labels:
    {{- include "nginx.labels" . | indent 4 }}
  annotations:
    "helm.sh/hook": test
spec:
  containers:
    - name: wget
      image: busybox
      command: ['wget']
      args: ['{{ .Release.Name }}-nginx:80']
  restartPolicy: Never

We can once again see why the named template we defined in the _helpers.tpl file, in a previous blog, is useful. We used the nginx.labels named template to produce the same labels we used in our deployment object. And we can see the test itself in the containers: section, which is rather simple.

We used busybox as a container image, which is a very lightweight image that gives us access to a few useful commands, usually used for debugging/fixing stuff. In this case, we use the wget command provided by busybox, and we pass some command-line arguments to wget, in the args: section.

wget is a simple download utility, and we instruct it in the arguments to connect to the Nginx web server that our chart installs. We make sure we use the same name we used in our deployment.yaml file, {{ .Release.Name }}-nginx and tell wget to connect to port: 80 that Nginx listens to (for incoming connections) by default.

Running Helm Chart Tests

To see this in action, we’ll first need to install our chart again. The objects our chart will create in Kubernetes are few and simple so this gets up and running fast. But for more complex charts, you might need to wait for a few minutes to make sure everything is ready, before running the tests.

Generically, Helm tests are executed with the following command: helm test NAME_OF_RELEASE.

So, with the name of our release, my-website we can run our tests with this command:

helm test my-website

And this output confirms that the test(s) is(are) successful.

NAME: my-website
LAST DEPLOYED: Mon Jun 14 00:16:35 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE:     my-website-nginx-test
Last Started:   Mon Jun 14 00:19:47 2021
Last Completed: Mon Jun 14 00:19:50 2021
Phase:          Succeeded
NOTES:
Please wait a few seconds until the nginx chart is fully installed.

After installation is complete, you can access your website by running this command:

kubectl get --namespace default service my-website-nginx

and then entering the IP address you get, into your web browser's address bar.

But, as usual, we shouldn’t be convinced that this is working correctly until we see the other scenario. What if our Nginx web server is down? Would this detect the anomaly? We can simulate our Nginx server failing by setting the number of replicas to zero, so no Nginx pods will be running anymore.

kubectl scale --replicas=0 deployment/my-website-nginx

Now if we rerun the command helm test my-website, we’ll see a different result, indicating that the test has failed.

NAME: my-website
LAST DEPLOYED: Mon Jun 14 00:16:35 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE:     my-website-nginx-test
Last Started:   Mon Jun 14 00:23:35 2021
Last Completed: Mon Jun 14 00:23:39 2021
Phase:          Failed
NOTES:
Please wait a few seconds until the nginx chart is fully installed.

After installation is complete, you can access your website by running this command:

kubectl get --namespace default service my-website-nginx

and then entering the IP address you get, into your web browser's address bar.
Error: pod my-website-nginx-test failed

We can now give our chart users a quick way to verify that everything has deployed correctly in their clusters.

Since tests are Helm hooks, it’s worth noting that if you want to create multiple tests for a chart, if required, you need can specify the order of execution using helm.sh/hook-weight annotations.
Also, remember that hooks create Kubernetes objects that can be left behind. For example, in this case, the tests create some pods that, although they run once, still show up in kubectl get pods commands.

NAME                    READY   STATUS      RESTARTS   AGE
my-website-nginx-test   0/1     Completed   0          103s
test-nginx-test         0/1     Error       0          28m

We may use helm.sh/hook-delete-policy annotations to ensure these objects get deleted after they run once. We can ensure that these pod resources get deleted:

Before a new (repeated) test is executed.
After the test passes.
After the test fails.

If we group all policies together, we can ensure that the Kubernetes objects created by the tests always get cleaned up when the test finishes.

Example test with all three hook delete policies enabled:

apiVersion: v1
kind: Pod
metadata:
  name: "{{ .Release.Name }}-nginx-test"
  labels:
    {{- include "nginx.labels" . | indent 4 }}
  annotations:
    "helm.sh/hook": test
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed
spec:
  containers:
    - name: wget
      image: busybox
      command: ['wget']
      args: ['{{ .Release.Name }}-nginx:80']
  restartPolicy: Never

But, remember, this may not always be desired. Sometimes, you may want those objects to stick around.

ENROLL in our Helm For Beginner's Course to learn other concepts, such as creating Helm Charts, adding dependencies in them, and distributing them.

Conclusion

In this article, you've learned what chart tests are and how to create them. It is important to thoroughly test Helm charts to catch potential issues before they cause problems in production environments. By taking the time to test Helm charts, you can further improve the reliability and consistency of your infrastructure.