Multi-Service Builds

Lesson 03 · Cloud Infrastructure & DevOps · ~12 min read

1. From one image to a system

Lessons 01 and 02 thought about a single image: one Dockerfile, one docker build, one docker run. Lesson 03 changes the frame. A real application is a system — multiple containers running together, sharing a network, sharing configuration, and depending on each other in a defined order. The URL-shortener becomes a real-ish app: an api service (FastAPI), a worker service (expiry sweeper), Postgres for persistence, and Redis as the worker’s queue. Each piece runs as its own container; docker-compose is the conductor.

The lesson’s premise: docker-compose lets you describe the whole stack in one YAML file, bring it up with a single command, and develop against it locally — without writing any custom orchestration code. No shell scripts, no hard-coded IPs, no manual docker network create. Just one file checked into the repo and one command to run it.

By the end of this lesson the URL-shortener sample app has a docker-compose.yml at the root of cloud-devops/sample-app/ that stands up all four services. Every subsequent lesson builds on top of this compose stack.

2. docker-compose.yml anatomy

A Compose file has a small number of top-level keys. Understanding each one unlocks the rest of the file format:

• services: — the containers in the stack, keyed by name. The name also becomes the DNS hostname on the implicit network.
• volumes: — named persistent volumes. Compose creates them lazily the first time they’re needed; they survive compose down but not compose down -v.
• networks: — optional. If omitted, Compose creates a default bridge network named <projectdir>_default and attaches every service to it.
• (legacy) version: — no longer required in Compose v2 / Docker Compose CLI. If you see it in older files, you can remove it safely.

For each service, the most important keys are:

3. The implicit network and service discovery

When Compose brings the stack up, it creates a bridge network and attaches every service to it. This is the feature that makes multi-service development feel effortless. Inside the network, each service is reachable by its service name as the hostname: api, worker, postgres, redis. DNS is provided by Docker’s embedded DNS server; no /etc/hosts editing, no hard-coded IPs.

The ports: mapping is only for host-to-container traffic. "8000:8000" means “forward host port 8000 to container port 8000”. Without that mapping, the container is still reachable from inside the network — from worker, the API is at http://api:8000/health — just not from your laptop’s browser. Only the api service needs to expose a port to the host; Postgres and Redis are reached only by the api and worker on the internal network, with no host-side port mapping needed in normal development.

4. depends_on and healthchecks

depends_on: by itself only orders start, not readiness. If you write depends_on: [postgres], Compose starts Postgres first — but it does not wait for Postgres to accept connections. Your API container starts and tries to connect to a not-yet-ready Postgres, gets ECONNREFUSED, and crashes. This is one of the most common sources of confusion for people using Compose for the first time.

Three solutions, in order of increasing robustness:

• Application-level retry. The cleanest solution. Your app retries the DB connection on startup with exponential backoff. Works everywhere — in compose, in Kubernetes, on bare metal — not just in local dev.
• condition: service_healthy with a healthcheck. Define a healthcheck on the dependency; Compose blocks the dependent service until the healthcheck passes. Shown below.
• A “wait-for-it” entrypoint script. A small shell script (wait-for-it.sh postgres:5432) that loops until the port responds. Older pattern; less needed since healthchecks landed.

The healthcheck + service_healthy pattern for Postgres:

postgres:
  image: postgres:16-alpine
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U urls -d urls"]
    interval: 5s
    timeout: 3s
    retries: 5
api:
  depends_on:
    postgres:
      condition: service_healthy

Now the api literally waits for pg_isready to succeed before starting. The recommended production answer remains application-level retry; healthchecks are a nice local-dev convenience that reduces the cognitive load of watching logs for crash-loops.

5. Volumes: data persistence and dev hot-reload

Compose supports two volume styles that serve very different purposes:

• Named volumes — pgdata:/var/lib/postgresql/data. Compose creates a managed volume; it survives container restarts and recreations. Used for stateful services (databases, caches with persistence enabled). The bytes live in Docker’s data directory on the host; you don’t see them as a filesystem path. Run docker volume ls to list them.
• Bind mounts — ./api:/app. Compose mounts a path from your laptop into the container. Used in development for hot-reload: uvicorn --reload plus a bind-mounted source directory means saving app.py and seeing it live inside the container immediately, without rebuilding the image.

The two styles can coexist on the same service. A common pattern is to give Postgres a named volume (for persistence) and the api a bind mount (for hot-reload). Do not ship bind-mounted source to production — that means the container depends on your repo to function, which defeats the purpose of immutable images.

6. Env vars and the .env convention

Compose has three layered mechanisms for setting env vars on a service, which combine in a specific precedence order:

1. environment: block with literal values or ${VAR} references. The ${VAR} references resolve from the shell where you ran docker compose up. Good for per-developer overrides and CI pipeline variables.
2. env_file: block — point at a file (usually ./.env), one KEY=VALUE per line. The file’s vars are injected into the container.
3. Compose’s implicit .env resolution — if there is a file literally named .env next to docker-compose.yml, Compose reads it automatically and substitutes ${VAR} references in the compose file itself. Note: this does not inject those vars into containers — only into the compose file template. To inject into containers, add env_file: ./.env on each service.

The standard pattern for the URL-shortener sample app:

• .env (in .gitignore) holds the actual dev values: POSTGRES_PASSWORD=devpassword, DATABASE_URL=postgresql://....
• .env.example (committed) lists the required keys with placeholder values and a comment reminding contributors to copy and fill it in.
• env_file: ./.env on each service that needs the vars.

7. Lifecycle commands you’ll actually use

A short reference for the commands you’ll type in the lab and in every project that uses Compose:

• docker compose up — build (if needed) + start in the foreground.
• docker compose up -d — detached (background).
• docker compose up --build — force rebuild before starting.
• docker compose logs -f api — follow logs for one service.
• docker compose exec api /bin/sh — shell into a running container.
• docker compose restart worker — restart one service.
• docker compose down — stop and remove containers + networks (volumes kept).
• docker compose down -v — also drop volumes (destroys DB data).
• docker compose ps — list running services and their status.

The two you will type 100 times a day during this course are docker compose up -d and docker compose logs -f <service>. Build the muscle memory early.

References

• Docker Compose documentation (docs.docker.com/compose/)
• Compose Specification (compose-spec.io) — the authoritative format reference
• Awesome Compose — example stacks for common tech combinations (github.com/docker/awesome-compose)
• cloud-devops/sample-app/README.md — per-lesson state of the sample app and what was added when