Lesson 07 VM Deploy with docker-compose
Theme:

Solutions — VM Deploy with docker-compose

Lesson 07 · Cloud Infrastructure & DevOps · Worked solutions

Solution to Task 1 — Pin a specific image tag and roll back

Replace the hard-coded :latest tag with an environment-variable reference in docker-compose.prod.yml:

# docker-compose.prod.yml — change to:
services:
  api:
    image: ghcr.io/${GHCR_OWNER}/url-shortener-api:${API_TAG:-latest}
  worker:
    image: ghcr.io/${GHCR_OWNER}/url-shortener-worker:${WORKER_TAG:-latest}

Set the initial tags in your .env file on the VM:

# .env (production)
API_TAG=v0.1.1
WORKER_TAG=v0.1.1

The deploy / rollback rhythm is always the same three commands — only the variable value changes:

# Forward to v0.1.1:
sed -i 's/^API_TAG=.*/API_TAG=v0.1.1/' .env
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d

# Rollback to v0.1.0:
sed -i 's/^API_TAG=.*/API_TAG=v0.1.0/' .env
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d

Verify after each operation:

curl /healthz
docker compose -f docker-compose.prod.yml images   # API row must show the expected tag

Common mistake

Putting the tag directly in the YAML instead of in .env. That forces you to edit the compose file and ship it back to the VM on every rollback — error-prone and involves two touch-points instead of one. Keep mutable config in .env; keep structural shape in the YAML. The :-latest default also acts as a safe fallback when API_TAG is not set at all.

Solution to Task 2 — Add a fail2ban-style SSH lockout

Install and start fail2ban:

sudo apt-get install -y fail2ban
sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd     # confirm sshd jail is active

The default Debian configuration in /etc/fail2ban/jail.d/defaults-debian.conf already enables the sshd jail with bantime = 10m and maxretry = 5. To tighten these defaults, create an override file:

# /etc/fail2ban/jail.d/sshd.local
[sshd]
enabled = true
bantime = 1h
maxretry = 3

Apply the override:

sudo systemctl reload fail2ban

To test, fail authentication five times from a second terminal (wrong password or wrong key). Then confirm and clean up:

sudo fail2ban-client status sshd          # IP appears in 'Banned IP list'
sudo fail2ban-client set sshd unbanip 198.51.100.42   # substitute your test IP

Document the steps, the jail configuration, and the unban command in /srv/url-shortener/OPS.md committed to your fork.

Common mistake

Banning your own laptop’s IP during the test and then finding yourself locked out. Always run the failed attempts from a second IP (a different machine, a cloud shell, or a mobile hotspot), or be ready to use your cloud provider’s serial-console / out-of-band rescue path to unban yourself. The fail2ban ban is per-jail and per-IP, so accessing the VM over its cloud console bypasses SSH entirely.

Solution to Task 3 — Externalise Postgres data to a dedicated bind mount

Create the host directory with the correct ownership before bringing the stack up. The official Postgres image runs as uid 999:

sudo mkdir -p /var/lib/url-shortener/postgres
sudo chown 999:999 /var/lib/url-shortener/postgres

Replace the named volume with a bind mount in docker-compose.prod.yml:

# docker-compose.prod.yml — postgres service (after):
services:
  postgres:
    volumes:
      - /var/lib/url-shortener/postgres:/var/lib/postgresql/data
# Remove `pg_data` from the top-level `volumes:` map.

Migrate the existing data from the named volume to the bind mount before restarting:

docker compose -f docker-compose.prod.yml down
# Copy from the named volume's host path to the new bind mount:
sudo cp -a /var/lib/docker/volumes/url-shortener_pg_data/_data/. \
           /var/lib/url-shortener/postgres/
docker compose -f docker-compose.prod.yml up -d

Verify the data survived:

docker compose -f docker-compose.prod.yml exec postgres \
  ls /var/lib/postgresql/data   # should list PG files (PG_VERSION, base/, etc.)

Take a manual snapshot via the AWS CLI (substitute your EBS volume ID):

aws ec2 create-snapshot --volume-id vol-0abcd --description "url-shortener pre-L08"

For daily automated snapshots, add a cron entry:

echo '0 2 * * * root aws ec2 create-snapshot --volume-id vol-0abcd --description "daily $(date +\%F)"' \
  | sudo tee /etc/cron.d/url-shortener-snap

Common mistake

Forgetting to chown 999:999 before mounting. The Postgres container starts as uid 999 and refuses to initialise the data directory if it is owned by root or ubuntu. The symptom is a log line reading “permission denied on data directory” and the container exiting immediately with code 1. Always set ownership on the host path before the first up -d.

In Lesson 08 we replace the entire “click around in the cloud console” workflow with terraform apply. The compose + Caddy + .env story stays exactly the same — only the VM lifecycle moves into code.