Lesson 15 Hooks & Tests
Theme:

Theory

Reading time: ~15 minutes  ·  Prerequisites: Helm templates, Jobs, Pods

1. What Are Helm Hooks?

Helm hooks are a mechanism that lets you intervene at specific points in a release lifecycle. When you run helm install, helm upgrade, helm delete, or helm rollback, Helm follows a well-defined sequence of steps. Hooks let you inject custom actions — typically Kubernetes Jobs or Pods — at precise moments within that sequence.

The key insight is that hook resources are not part of the release. Helm creates them, waits for them to complete, and then optionally deletes them. They never appear in helm get manifest alongside your Deployments and Services. This separation is intentional: hooks perform one-time operational tasks (migrations, backups, notifications) that should not persist as long-running workloads.

2. Lifecycle Events

Helm defines eight hook events, organized around four operations:

Operation Pre-Hook Post-Hook
Install pre-install post-install
Upgrade pre-upgrade post-upgrade
Delete pre-delete post-delete
Rollback pre-rollback post-rollback

The execution flow for an upgrade, for example, looks like this:

  1. Helm renders all templates (including hook templates)
  2. Helm creates pre-upgrade hook resources, sorted by weight
  3. Helm waits until each hook resource reaches "Ready" (Job completes, Pod succeeds)
  4. Helm applies the main release resources (Deployments, Services, etc.)
  5. Helm creates post-upgrade hook resources, sorted by weight
  6. Helm waits until post-upgrade hooks complete
  7. The upgrade is marked as successful

Important

If a pre-upgrade hook fails (the Job exits with a non-zero code), Helm aborts the entire upgrade. The main resources are never updated. This makes pre-hooks an excellent place for validation gates — if your migration fails, the rollout stops.

3. Hook Annotations

Three metadata annotations control how hooks behave. All are placed in the resource's metadata.annotations block.

helm.sh/hook

This is the primary annotation that marks a resource as a hook and specifies when it runs. You can assign multiple events by comma-separating them:

annotations:
  "helm.sh/hook": pre-install,pre-upgrade

This is common for database migrations that need to run both on initial install and on every subsequent upgrade.

helm.sh/hook-weight

When multiple hooks target the same event, Helm executes them in ascending weight order. Weights are strings representing integers (positive or negative):

annotations:
  "helm.sh/hook": pre-upgrade
  "helm.sh/hook-weight": "-5"

A hook with weight -5 runs before a hook with weight 0 (the default), which runs before weight 10. If two hooks have the same weight, they are sorted alphabetically by resource kind and name. Use negative weights for tasks that must happen first (like backups) and positive weights for tasks that follow (like cache warming).

helm.sh/hook-delete-policy

Hook resources are not automatically removed. Without a delete policy, completed Jobs and their Pods accumulate in the namespace. The delete policy annotation tells Helm when to clean up:

Policy When the Hook Resource Is Deleted
before-hook-creation The previous hook resource is deleted before the new one is created. Prevents "already exists" errors.
hook-succeeded Deleted immediately after the hook completes successfully.
hook-failed Deleted if the hook fails. Useful combined with hook-succeeded to always clean up.

You can combine multiple policies:

annotations:
  "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded

Best Practice

The most widely used policy is before-hook-creation. It ensures that each upgrade starts fresh by deleting the old hook Job before creating the new one. This avoids name collisions and keeps the namespace clean over time. If you also want to remove the hook immediately after success, combine it with hook-succeeded.

4. Hook Resource Types

While any Kubernetes resource can technically be a hook, the vast majority of hooks are Jobs or Pods:

Less commonly, you might use a ConfigMap or Secret as a hook when you need to create a configuration resource before the main install proceeds. For example, a pre-install Secret that holds database credentials generated by an external tool.

5. Common Hook Patterns

Database Migration (pre-upgrade Job)

The most common hook pattern runs schema migrations before the new application version starts. The migration Job uses the same image as the main application but runs a different command:

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "mychart.fullname" . }}-migrate
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-weight": "0"
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          command: ["python", "manage.py", "migrate", "--noinput"]
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: {{ include "mychart.fullname" . }}-db
                  key: url

Because the hook is pre-upgrade, if the migration fails, the new Deployment image is never updated, keeping the running application compatible with the current database schema.

Backup Before Upgrade

Pair a backup hook (weight -10) with a migration hook (weight 0). The backup runs first, creating a database dump. If the subsequent migration fails, you have a restore point. This two-hook pattern is a reliable safety net for stateful applications.

Post-Deploy Notification

A post-upgrade hook that sends a Slack message or updates a deployment tracker. Use hook-succeeded as the delete policy so the notification pod disappears after sending.

6. Helm Tests

Helm tests are a special category of hook. A test is a Pod annotated with helm.sh/hook: test. Unlike install/upgrade hooks that run automatically, tests only run when you explicitly invoke helm test <release>.

How Tests Work

  1. You run helm test my-release
  2. Helm creates all pods annotated with helm.sh/hook: test
  3. Helm waits for each pod to complete
  4. If all pods exit 0, the test passes; any non-zero exit fails the test
  5. Delete policies clean up the test pods

Writing Test Pods

Test pods typically live in templates/tests/ within your chart. Common test patterns include:

apiVersion: v1
kind: Pod
metadata:
  name: {{ include "mychart.fullname" . }}-test-health
  annotations:
    "helm.sh/hook": test
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  restartPolicy: Never
  containers:
    - name: health
      image: curlimages/curl:8.5.0
      command: ["curl", "--fail", "--silent", "--max-time", "10",
                "http://{{ include \"mychart.fullname\" . }}:{{ .Values.service.port }}/healthz"]

Test Timeouts

By default helm test waits 5 minutes. Override this with --timeout 10m0s. Use the --logs flag to print container logs when a test fails, which is invaluable for debugging: helm test my-release --logs.

7. Putting It All Together

A well-structured chart for a database-backed web application might include four hook templates:

  1. pre-upgrade backup Job (weight -10) — dumps the database
  2. pre-upgrade migration Job (weight 0) — runs schema migrations
  3. test: connectivity Pod — verifies the service responds to HTTP
  4. test: data Pod (weight 5) — verifies migration results are queryable

This gives you a safe upgrade path (backup, then migrate, then deploy) with post-deploy validation (connectivity, then data). If any step fails, you know exactly where the problem occurred.

Further Reading

Next lesson: Lesson 16 — continuing your Helm journey.