Lab 07 — Ship a real URL on a real VM

Lesson 07 · Cloud Infrastructure & DevOps · ~60 min

Goal: by the end of this lab you have a public HTTPS URL (or a tls internal local one) running the URL-shortener API and worker from the CI-built GHCR images, fronted by Caddy, on a single VM you provisioned by hand. The lab covers AWS EC2 as the primary path with sidebar notes for GCP, Azure, and DigitalOcean.

Prerequisites

• Plan B’s Lesson 06 lab done: the GHCR pipeline has pushed at least one tag of ghcr.io/<your-username>/url-shortener-api and the worker.
• A free-tier AWS account (or GCP / Azure / DigitalOcean).
• A domain you can edit DNS on (optional — the lab includes a tls internal fallback for localhost).
• An SSH key pair already loaded in the cloud console.
• Local tools: ssh, scp, dig, curl.

Step 1 — Provision the VM (web console)

Use your cloud provider’s web console to create a single VM. Instructions per provider:

• AWS EC2: Launch a t3.micro running Ubuntu 24.04 LTS in us-east-1. Security group rules: TCP 22 from your IP, TCP 80 and 443 from 0.0.0.0/0. Assign a public IP. Capture the public IP — you’ll need it in Step 3.
• GCP: e2-micro in any free-tier region; allow HTTP + HTTPS via the firewall presets. Note the ephemeral external IP.
• Azure: Standard_B1s Ubuntu, open ports 22, 80, and 443.
• DigitalOcean: s-1vcpu-1gb droplet, Ubuntu 24.04, enable IPv4 firewall with rules for ports 22, 80, and 443.

Expected output: a public IPv4 you can ssh ubuntu@<ip> into. Note: ubuntu is the default user on AWS Ubuntu images; GCP defaults to your SSH username; DigitalOcean defaults to root. The rest of this lab assumes you are logged in as a user with sudo rights.

Step 2 — Install Docker engine + the compose plugin

ssh ubuntu@<vm_ip>

sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
  -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \
  https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
  | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io \
  docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker

docker compose version

Expected: Docker Compose version v2.x.x. The crucial detail is docker-compose-plugin (the v2 plugin) — not the legacy hyphenated docker-compose apt package. This course uses docker compose (space) everywhere.

Step 3 — Copy sample-app files to the VM

From your laptop, in the repository root:

ssh ubuntu@<vm_ip> "sudo mkdir -p /srv/url-shortener/caddy && sudo chown -R ubuntu:ubuntu /srv/url-shortener"
scp cloud-devops/sample-app/docker-compose.prod.yml ubuntu@<vm_ip>:/srv/url-shortener/
scp cloud-devops/sample-app/caddy/Caddyfile ubuntu@<vm_ip>:/srv/url-shortener/caddy/

docker-compose.prod.yml (also at cloud-devops/sample-app/docker-compose.prod.yml):

# Production compose file for the URL shortener.
# Reader-facing copy lives at cloud-devops/lesson07-vm-deploy/lab_guide.html —
# if you edit this file, update the inline copy in the lab guide too.

services:
  caddy:
    image: caddy:2.7
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    environment:
      - DOMAIN
      - ACME_EMAIL
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      - api

  api:
    image: ghcr.io/${GHCR_OWNER}/url-shortener-api:latest
    restart: unless-stopped
    env_file: .env
    environment:
      - DATABASE_URL=postgres://urls:${DB_PASSWORD}@postgres:5432/urls
      - REDIS_URL=redis://redis:6379/0
    depends_on:
      - postgres
      - redis

  worker:
    image: ghcr.io/${GHCR_OWNER}/url-shortener-worker:latest
    restart: unless-stopped
    env_file: .env
    environment:
      - DATABASE_URL=postgres://urls:${DB_PASSWORD}@postgres:5432/urls
      - REDIS_URL=redis://redis:6379/0
      - WORKER_INTERVAL=${WORKER_INTERVAL:-60}
    depends_on:
      - postgres
      - redis

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      - POSTGRES_USER=urls
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=urls
    volumes:
      - pg_data:/var/lib/postgresql/data

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    volumes:
      - redis_data:/data

volumes:
  caddy_data:
  caddy_config:
  pg_data:
  redis_data:

caddy/Caddyfile (also at cloud-devops/sample-app/caddy/Caddyfile):

# Caddy v2 reverse proxy for the URL shortener.
# Reader-facing copy lives at cloud-devops/lesson07-vm-deploy/lab_guide.html —
# if you edit this file, update the inline copy in the lab guide too.

{
    email {$ACME_EMAIL}
}

{$DOMAIN} {
    encode zstd gzip
    reverse_proxy api:8000

    @health path /healthz
    handle @health {
        reverse_proxy api:8000
    }

    log {
        output stdout
        format console
    }
}

Step 4 — Write the production .env on the VM

On the VM, generate strong passwords first, then write them into .env:

cd /srv/url-shortener

# Generate strong random values first:
DB_PASS=$(openssl rand -hex 16)
SECRET=$(openssl rand -hex 32)

# Write the .env file:
cat > .env <<EOF
GHCR_OWNER=<your-github-username>
DOMAIN=urls.example.com
ACME_EMAIL=you@example.com
DB_PASSWORD=${DB_PASS}
SECRET_KEY=${SECRET}
WORKER_INTERVAL=60
EOF

chmod 600 .env

Step 5 — Point DNS (optional)

If you want a real Let’s Encrypt certificate, add an A record at your DNS provider pointing your domain to the VM’s public IP, then verify propagation:

# On your laptop:
dig +short urls.example.com

Wait until dig returns the correct IP. DNS TTLs typically take 1–5 minutes with a new A record, longer for modifications. If you skip this step, set DOMAIN=localhost in .env and Caddy will issue a locally-trusted internal certificate — useful for the lab but not reachable from the public internet.

Step 6 — Deploy

On the VM:

cd /srv/url-shortener
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d
docker compose -f docker-compose.prod.yml ps
docker compose -f docker-compose.prod.yml logs --tail=50 caddy

Expected from ps: 5 services (caddy, api, worker, postgres, redis), all in state running. Expected in Caddy logs: a line containing certificate obtained successfully (or, if DOMAIN=localhost, using internal certificate).

Step 7 — Smoke-test

From your laptop:

curl -fsS https://urls.example.com/healthz
curl -fsS -X POST -d '{"url":"https://example.com"}' \
  -H 'Content-Type: application/json' https://urls.example.com/shorten

Expected: {"status":"ok"} from /healthz, then a JSON response with a short URL from /shorten. Click the short URL in a browser — it should redirect to https://example.com. If you used DOMAIN=localhost, replace urls.example.com with localhost and accept the locally-signed certificate warning.

Step 8 — Verify reboot survival

# On the VM:
sudo reboot

# Wait ~30 seconds, then SSH back in:
ssh ubuntu@<vm_ip>
docker compose -f /srv/url-shortener/docker-compose.prod.yml ps

Expected: all 5 services back up automatically. This works because restart: unless-stopped combined with Docker daemon’s systemd unit restarts containers on boot.

Step 9 — Re-deploy on a new image

Push any trivial change to main in your fork; wait for the GHCR pipeline from Lesson 06 to publish a new :latest. Then on the VM:

cd /srv/url-shortener
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d

Expected: pull shows new layers downloading; up -d recreates only the changed services (e.g., Recreating url-shortener-api-1); curl /healthz shows the updated behaviour. This is the entire deploy loop. Memorise the two-command rhythm — Module 5 automates it, but the shape is identical.

What you should leave with

• A real (or tls internal) HTTPS URL serving the URL shortener
• The mental model: image registry → VM → compose pull → Caddy fronts → request arrives at the API
• Awareness of what hurts: manual provision, single point of failure, data and runtime on the same disk. Lesson 08 fixes “manual provision” first.

Troubleshooting