Lab 11 — Real domain + HTTPS in 15 minutes

Lesson 11 · Cloud Infrastructure & DevOps · ~45 min

This lab extends the shortener Helm chart from Lesson 10 with Ingress, cert-manager, and ExternalDNS. Two paths are supported: a cloud path (real domain, real TLS) and a kind path (local cluster, DNS faked via /etc/hosts). Both validate the chart rendering and ingress routing; only the cloud path completes the http-01 cert-manager flow end-to-end.

1. Prerequisites

Complete Lesson 10 — the Helm chart must install and all pods must reach Ready. Then confirm your chosen path:

Cloud path: a real domain you control; a Cloudflare or Route53 API token; a managed Kubernetes cluster (EKS, GKE, AKS, DigitalOcean, or Linode). The ACME server must be able to reach your cluster on port 80.
kind path: kind installed; Go toolchain for cloud-provider-kind (go install sigs.k8s.io/cloud-provider-kind@latest). Confirm with cloud-provider-kind --version. No real domain needed.

2. Step 1 — Install `ingress-nginx`

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  --set controller.publishService.enabled=true \
  --wait

Verify the controller is ready and has an external IP:

kubectl -n ingress-nginx get pods
kubectl -n ingress-nginx get svc ingress-nginx-controller

Cloud path: the EXTERNAL-IP column shows a hostname or IP assigned by the cloud provider’s load balancer. Copy it — ExternalDNS will point your domain here automatically in Step 4.

kind path: the EXTERNAL-IP stays <pending> until cloud-provider-kind is running. In a separate terminal:

sudo cloud-provider-kind &

Within 10–20 seconds kubectl -n ingress-nginx get svc ingress-nginx-controller shows an IP in the 127.0.0.x range. Note that IP — you will need it in Step 4.

3. Step 2 — Install cert-manager

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true \
  --wait

Verify CRDs registered and all three controller pods are Ready:

kubectl get crd | grep cert-manager
kubectl -n cert-manager get pods

Expected: three pods (cert-manager, cert-manager-cainjector, cert-manager-webhook), all 1/1 Running.

4. Step 3 — Apply the chart with `values-l11.yaml`

From the repo root:

helm upgrade --install shortener cloud-devops/sample-app/helm \
  -f cloud-devops/sample-app/helm/values-dev.yaml \
  -f cloud-devops/sample-app/helm/values-l11.yaml \
  --namespace shortener --create-namespace --wait --timeout 8m

This single command creates:

The Ingress resource (because ingress.enabled: true in values-l11.yaml).
The ClusterIssuer named letsencrypt-staging (because manageIssuer: true).
cert-manager auto-creates a Certificate resource from the Ingress tls: block + annotation.

Confirm the objects exist:

kubectl -n shortener get ingress,certificate
kubectl get clusterissuer

5. Step 4 — Make DNS resolve to the Ingress

Cloud path: install ExternalDNS and it will create DNS records automatically. First edit cloud-devops/sample-app/helm/externaldns-values.yaml — replace REPLACE_ME_CLOUDFLARE_API_TOKEN with your real token and update domainFilters and host values in values-l11.yaml to match your domain. Then:

helm repo add external-dns https://kubernetes-sigs.github.io/external-dns
helm upgrade --install external-dns external-dns/external-dns \
  --namespace external-dns --create-namespace \
  -f cloud-devops/sample-app/helm/externaldns-values.yaml \
  --wait

Within ~30 seconds the DNS record appears at your provider. Confirm propagation:

dig shortener.<your-domain>

kind path: add a line to /etc/hosts using the IP from Step 2:

echo "<EXTERNAL-IP> shortener.local.test" | sudo tee -a /etc/hosts

Confirm: ping shortener.local.test should resolve to the EXTERNAL-IP.

6. Step 5 — Watch cert-manager issue the cert

kubectl -n shortener get certificate
kubectl -n shortener describe certificate shortener-tls
kubectl -n shortener get challenge,order

Expected progression (typically 60–120 seconds on a warm cluster with working DNS):

Certificate Ready: False — challenge in progress.
Order Pending → Challenge Valid → Order Valid.
Certificate Ready: True — cert stored in Secret shortener-tls.

kind note: on kind the Let’s Encrypt staging ACME server cannot reach shortener.local.test over the internet, so the Certificate will stay in Ready: False. The chart renders correctly and ingress routing works — only the http-01 challenge cannot complete without a publicly reachable hostname. This is the expected kind-path limitation; see the Troubleshooting section below.

7. Step 6 — Hit the URL

Cloud path (staging cert — browser will warn; that is expected):

curl -v https://shortener.<your-domain>/healthz

kind path (skip cert verification because the staging cert is not browser-trusted):

curl -vk https://shortener.local.test/healthz

Expected response: HTTP 200, JSON body {"status":"ok"}. In a browser, open https://shortener.<host> — you will see the “your connection is not private” warning from the staging issuer; click through to confirm the API responds.

8. Step 7 — Promote staging to prod issuer

Do this only when staging is stable for several hours and you are confident in the setup. Edit values-l11.yaml:

certIssuer: "letsencrypt-prod"
acmeServer: "https://acme-v02.api.letsencrypt.org/directory"

Then upgrade and delete the old staging Secret to force re-issue:

helm upgrade shortener cloud-devops/sample-app/helm \
  -f cloud-devops/sample-app/helm/values-dev.yaml \
  -f cloud-devops/sample-app/helm/values-l11.yaml \
  -n shortener --wait
kubectl -n shortener delete secret shortener-tls   # force re-issue from prod
kubectl -n shortener get certificate -w

Expected: a new Certificate is issued from letsencrypt-prod within ~60 seconds. The browser warning disappears.

9. Step 8 — Tear down

helm uninstall shortener -n shortener
helm uninstall ingress-nginx -n ingress-nginx
helm uninstall cert-manager -n cert-manager
helm uninstall external-dns -n external-dns 2>/dev/null || true
kind delete cluster --name shortener 2>/dev/null || true

10. What you should leave with

Real-domain TLS, automated DNS, automated cert renewal — all driven by chart values. To stand up a new environment you add a values-<env>.yaml with the right host and re-run helm upgrade --install. The cluster does the rest: ExternalDNS writes the DNS record, cert-manager issues and renews the cert, ingress-nginx routes and terminates TLS. Zero manual certificate steps, zero manual DNS entries.

11. Troubleshooting

Common issues

Certificate stuck in Ready: False with reason: Failed — describe the Certificate; look for “challenge failed,” then describe the Challenge resource. Most common cause: DNS doesn’t resolve at the ACME server yet. Fix DNS first, then delete the stale Challenge/Order to retry: kubectl delete challenge,order -n shortener --all.
cert-manager errors with 404 Not Found on the http-01 challenge URL — ingress-nginx isn’t routing /.well-known/acme-challenge/* to the cert-manager solver Pod. Confirm ingressClassName: nginx in the Ingress spec matches the controller’s installed class (kubectl get ingressclass).
Browser warning persists even on prod issuer — the cert renewed but ingress-nginx may still serve the old Secret. Restart the controller: kubectl -n ingress-nginx rollout restart deploy/ingress-nginx-controller.
kind path: EXTERNAL-IP stays <pending> — cloud-provider-kind is not running. Start it: sudo cloud-provider-kind &. Requires sudo because it binds privileged ports.
RateLimitedError from Let’s Encrypt — you exhausted the prod quota (50 certs/domain/week). Stay on staging until everything works for 24h, then promote once. If you are locked out, wait for the weekly reset.