VM Deploy with docker-compose

Lesson 07 · Cloud Infrastructure & DevOps · ~12 min read

1. Why a single-server lesson is worth a whole module

Most “production” introductions skip the single-VM stage and jump straight to Kubernetes. That’s a mistake for the reader’s mental model. There are five separable problems being collapsed into one when you go straight to K8s: get the image to a host, get the host to the internet, get a name pointed at the host, get a TLS cert, and survive a reboot. Solving them one VM at a time makes each one nameable and debuggable in isolation. On Kubernetes all five hide behind a single kubectl apply and when something goes wrong you have five candidate explanations instead of one.

By the end of this lesson you have a public URL for under $10/month (or $0 on the free tier). More importantly, you can draw on a whiteboard the five layers a request crosses to reach the API: user → DNS → Caddy (TLS) → compose network → API container → Postgres. Module 4’s Kubernetes lessons will keep referring back to “the thing Caddy did in Lesson 07” when they introduce Ingress controllers — knowing the single-VM version first makes the K8s analogue immediately comprehensible rather than opaque.

2. docker-compose.prod.yml vs. docker-compose.yml

The development compose file from Module 1 is built around three assumptions that do not belong in production. First, build: ./api compiles the image from your local source tree — but the production VM has no source tree and no Python toolchain. Second, volumes: ["./api:/app"] mounts the local directory over the image content for live reload — on the VM there is nothing useful at that path, and mounting it would silently overwrite the well-built image with an empty directory. Third, ports: ["8000:8000"] on localhost exposes the API directly — on the production VM “localhost” is the public internet, so this would expose an unauthenticated service without TLS.

docker-compose.prod.yml strips all three and adds the production-appropriate alternatives:

• image: instead of build: — the runtime never has the source tree
• No code mounts — only Caddy config (./caddy/Caddyfile:/etc/caddy/Caddyfile:ro) and named volumes for Caddy’s data dir and Postgres data
• restart: unless-stopped on every service — services come back after a reboot or crash
• ports: only on Caddy (80:80 and 443:443) — the API and worker are reachable only on the internal compose network
• env_file: .env for secrets; environment: only for non-secret config like WORKER_INTERVAL

Both files share the same services: names, so in principle they could be merged with -f docker-compose.yml -f docker-compose.prod.yml. We deliberately avoid this in production: running with the dev file accidentally would mount your local code over the image and expose unprotected ports. The lab calls this out explicitly as a common mistake in Step 4.

3. Caddy and automatic TLS

Caddy 2 is a single Go binary that does reverse proxying and automatic ACME certificate management. Point it at a domain that resolves to the VM, give it ports 80 and 443, and it negotiates a Let’s Encrypt certificate on first run, then auto-renews before expiry — no separate certbot cron job, no NGINX configuration reload dance, no manual certificate PEM juggling.

The production Caddyfile is small enough to fit on one slide:

{
    email {$ACME_EMAIL}
}

{$DOMAIN} {
    encode zstd gzip
    reverse_proxy api:8000
}

The top block contains global options. Caddy uses an ACME contact email so Let’s Encrypt can warn about renewal failures. The site block routes all traffic at {$DOMAIN} to the API container. Both {$ACME_EMAIL} and {$DOMAIN} are environment variable references that Caddy expands at startup. This means the same Caddyfile serves a real domain (DOMAIN=urls.example.com) and a developer’s local machine (DOMAIN=localhost).

The DOMAIN=localhost path is the “tls internal” fallback: when Caddy sees localhost, it issues a locally-trusted certificate from its own built-in CA rather than talking to Let’s Encrypt. The developer can visit https://localhost without browser security warnings (after trusting Caddy’s CA once with caddy trust). Switching from internal to public is a one-line .env change plus a DNS A record.

4. The deploy loop

The production deploy is three commands, repeated for every release:

docker compose -f docker-compose.prod.yml pull        # fetch new image(s)
docker compose -f docker-compose.prod.yml up -d       # recreate changed containers
docker compose -f docker-compose.prod.yml logs --tail=50 api

You run these by hand in this lesson. Module 5 (GitOps with ArgoCD) automates the same loop on Kubernetes. The crucial observation is that this is the same shape as a Kubernetes deploy: fetch new artifact, reconcile desired state with actual state, observe. Compose’s up -d is the reconciler; it reads the compose file, inspects the running containers, and applies the minimum delta — recreating services whose image or configuration has changed, leaving unchanged services untouched. Volumes and networks are never removed by up; only by docker compose down. This means the database data persists across every release.

5. Secrets, env, and the disk

Secrets live in one file on the VM: /srv/url-shortener/.env, mode 0600, owned by root (or the dedicated deploy user the lab creates). The compose file references it with env_file: .env. The five secrets and configuration values the URL shortener needs at this stage:

• GHCR_OWNER — the owner segment of the ghcr.io/<owner>/url-shortener-api image names
• DOMAIN — the public hostname, or localhost for the internal-cert fallback
• DB_PASSWORD — Postgres password (generate with openssl rand -hex 16)
• SECRET_KEY — Flask/FastAPI session secret (generate with openssl rand -hex 32)
• WORKER_INTERVAL — non-secret polling interval in seconds, but lives in the same file to keep the deploy surface small

6. Survival, rollback, and the boundaries of one-VM ops

Understanding what breaks on a single VM — and how to recover — is as important as understanding the happy path.

API crashes. restart: unless-stopped in the compose file tells Docker to restart the container whenever it exits with a non-zero code. If the API crash-loops, docker compose logs api shows the stack trace; docker compose down && docker compose up -d resets the compose state and gives the containers a clean start.

Bad image deployed. Pin a known-good tag in .env (API_TAG=v1.2.0) and re-run docker compose pull && up -d. Rollback is exactly the same as a forward deploy; only the tag value differs. This is why keeping tags in .env rather than hard-coded in the YAML is worth the extra indirection.

VM rebooted. Docker’s systemd unit starts automatically on boot. Compose services with restart: unless-stopped restart with it. The reader verifies this in Step 8 of the lab with a deliberate sudo reboot followed by a reconnect and docker compose ps.

VM destroyed. Game over for the data. Postgres lives on a Docker named volume on the VM’s root disk. One terraform destroy in Lesson 08 deletes the volume along with the instance. This is exactly why Module 4 moves Postgres to a managed database (RDS) that lives outside the compute tier — the VM becomes stateless and replaceable.

The honest scope of single-VM: it is the right shape for a demo, a staging slot, a personal blog, or a low-traffic side project. It is the wrong shape the moment any of the following become true: you need zero-downtime deploys, you need horizontal scaling, more than one engineer touches production, or your data is more valuable than your VM’s uptime SLA (which is ~99.5% for a single EC2 instance).

References

• Docker Compose file reference — docs.docker.com/compose/compose-file/
• Caddy v2 reverse proxy quick-start — caddyserver.com/docs/quick-starts/reverse-proxy
• Let’s Encrypt ACME challenge types — letsencrypt.org/docs/challenge-types/
• AWS EC2 t3.micro pricing & free-tier — aws.amazon.com/ec2/instance-types/t3/
• cloud-devops/lesson06-push-to-deploy/theory.html — the upstream piece of this pipeline (CI pushing the image)
• cloud-devops/lesson08-terraform-basics/theory.html — the next lesson, where the VM becomes Terraform-managed