GitHub Actions Pipeline

Lesson 04 · Cloud Infrastructure & DevOps · ~13 min read

1. Why CI matters for this app

The URL-shortener has two services: an api (FastAPI) and a worker (expiry sweeper). In Lesson 03 you brought them up together with docker-compose and confirmed they interoperate locally. But every new commit creates a question: do the API and worker still play nicely after my change? Without CI, the answer lives in the maintainer’s head. Someone has to remember to run the tests, remember to check formatting, remember to rebuild both images. That works for one developer. It breaks the moment you have two.

Continuous integration (CI) moves that check into an automated runner that executes on every push and every pull request. The check is reproducible — same OS, same Python version, same clean filesystem every time — and the result is visible to everyone: a green check means “the tests passed on a runner that doesn’t have your coffee-stained laptop’s quirks.”

The operational consequence is simple: a green check = mergeable; a red check = fix me first. Teams that run CI catch regressions in minutes instead of at 2 AM in production. Lesson 04 wires up that first green check for the URL-shortener. Lesson 05 makes it a required gate via branch protection, and Lesson 06 turns the green check into a deploy trigger.

2. Anatomy of a workflow file

GitHub Actions workflows live in .github/workflows/ at the repository root. Each YAML file is one workflow. A workflow has three top-level concerns:

• name: — displayed in the GitHub Actions tab. Optional but useful.
• on: — the trigger(s) that activate the workflow. Covered in section 3.
• jobs: — the work to perform. Each job runs on its own runner (an isolated VM or container) and contains a sequence of steps.

A step is either a uses: line (invoke a reusable action from the marketplace) or a run: line (execute a shell command). Here is the minimal viable workflow for the URL-shortener’s API tests:

name: CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install -r cloud-devops/sample-app/api/requirements.txt
      - run: pytest cloud-devops/sample-app/api/tests -v

The runner starts as a bare Ubuntu VM. Step 1 (actions/checkout) clones the repo. Step 2 (actions/setup-python) installs Python 3.11 into PATH. Steps 3 and 4 are the same commands you would run in a terminal. The runner’s filesystem is discarded at the end of the job — no state leaks to the next run.

Jobs within a workflow run in parallel by default. You can form a directed acyclic graph (DAG) by declaring needs: — the build job below runs only after lint and test both succeed, which saves compute by not building a Docker image that would fail lint anyway.

3. Triggers in depth

The on: key accepts a map of event types to filters. The events relevant to this course:

• push — fires when commits are pushed to matching branches. Typically main only. Optionally filtered by paths or paths-ignore.
• pull_request — fires on every commit pushed to a PR targeting matching branches. Also fires on PR open, reopen, and synchronize events. This is the primary trigger for pre-merge checks.
• schedule — cron-based; useful for nightly integration tests or dependency scans against a fresh DB.
• workflow_dispatch — manual trigger button in the GitHub UI. Useful for one-off runs (e.g., cut a release) without pushing a commit.
• workflow_call — allows one workflow to invoke another as a reusable component — the basis of modular pipeline design.

For the URL-shortener, push on main plus pull_request against main covers 95% of cases. Monorepos benefit from path filters:

on:
  push:
    branches: [main]
    paths-ignore:
      - '**.md'
      - 'docs/**'
  pull_request:
    branches: [main]
    paths-ignore:
      - '**.md'
      - 'docs/**'

With paths-ignore, a README typo does not trigger the full pipeline. Note the distinction: paths: is an allow-list (CI only runs if a matching file changes); paths-ignore: is a deny-list (CI runs unless only matching files change). Confusing the two is a common mistake that silently skips CI for legitimate code edits.

4. Matrix builds

A matrix strategy takes one job specification and expands it into N parallel runs, one per combination of matrix values. This is the right tool when you officially support multiple language versions or operating systems.

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pytest cloud-devops/sample-app/api/tests

For the URL-shortener, we test against Python 3.11 and 3.12. The matrix produces two parallel jobs (test (3.11) and test (3.12)) from one definition. The fail-fast: false flag lets both jobs run to completion even if one fails — valuable because a failure might be Python-version-specific and you want to see all failures, not just the first.

The matrix accepts multiple dimensions. A 3-version × 3-OS grid produces 9 jobs. Use include: to add specific extra combinations that would not otherwise exist:

matrix:
  python-version: ["3.11", "3.12"]
  include:
    - python-version: "3.10"
      os: ubuntu-20.04  # older LTS for legacy support

And exclude: to subtract combinations (e.g., a specific OS + version pairing known to be broken upstream). Keep the matrix lean: adding macOS to a 3-version matrix multiplies cost by 10×. Add dimensions only when you have a concrete reason.

5. Caching strategies

Every runner starts clean. Without caching, every run reinstalls all dependencies and re-pulls all Docker base layers from the internet. Caching avoids this repetition. Three caches matter for this course:

Language dependency cache. The actions/setup-python@v5 action has a built-in cache shortcut:

- uses: actions/setup-python@v5
  with:
    python-version: "3.11"
    cache: pip
    cache-dependency-path: |
      cloud-devops/sample-app/api/requirements.txt
      cloud-devops/sample-app/worker/requirements.txt

The cache: pip key stores the pip download cache (~/.cache/pip) keyed on the SHA256 of the requirements files listed in cache-dependency-path. On a cache hit, pip install reads from the local cache rather than downloading from PyPI. This saves 30–90 seconds per job once the cache warms. An analogous shortcut exists for Node (cache: npm), Go (cache: go), and others.

Docker layer cache. The docker/build-push-action supports exporting and importing BuildKit layer caches to GitHub’s cache backend:

- uses: docker/build-push-action@v5
  with:
    context: ./cloud-devops/sample-app/api
    push: false
    tags: api:ci-${{ github.sha }}
    cache-from: type=gha,scope=api
    cache-to: type=gha,mode=max,scope=api

The type=gha backend stores layer blobs in GitHub Actions cache storage. The scope parameter namespaces the cache so the api and worker layers don’t collide. The mode=max flag exports all layers, including intermediate stages in a multi-stage build. On a warm cache, an unchanged image builds in seconds instead of minutes.

Generic cache. The actions/cache@v4 action caches arbitrary directories. Useful for tool caches not covered by language setup actions (e.g., ~/.cache/pre-commit, Terraform provider downloads, compiled test dependencies). The key/restore-keys pattern lets you do prefix-match fallback: restore the closest cache hit if the exact key is absent.

6. Integrating lint and test

Lint and test can run as steps within the same job (simpler; sequential) or as separate jobs (parallel; fail independently). Separate jobs are preferred when you want to see both failure types on a PR without one blocking the other from reporting. For the URL-shortener we use three jobs:

• lint — runs ruff check (style + bug detection) and ruff format --check (formatting gate) on both services. A format failure means the developer ran ruff format locally but didn’t commit the reformatted files.
• test — runs pytest on both services across a Python 3.11/3.12 matrix. Uses pip cache to avoid reinstalling deps on every run.
• build — runs docker/build-push-action with push: false for both services. Depends on lint and test via needs:. Only runs if cheap checks pass first.

Pinning ruff==0.6.9 rather than relying on the latest version prevents a ruff release from suddenly failing your pipeline. The same principle applies to all tools installed in CI: pin versions in requirements.txt or explicitly in the run: step.

7. Common pitfalls

References

• GitHub Actions documentation index
• actions/checkout README
• actions/setup-python README
• docker/build-push-action README
• BuildKit cache documentation
• Sample-app CI workflow (cloud-devops/sample-app/.github/workflows/ci.yml)