Lab 05 — Add security gates and image signing

Lesson 05 · Cloud Infrastructure & DevOps · ~45 min

Goal: By the end of this lab you will have: Trivy CVE scanning and SBOM generation integrated into the existing CI build job; a separate security.yml workflow that signs images keylessly on merge to main; and branch protection configured to require the security checks before merging.

Prereqs

• L04 lab done: ci.yml is committed and the CI pipeline runs green on PRs.
• The repository is pushed to GitHub and you have admin access (required for branch protection).
• Your GitHub account can write to the repository’s GHCR (GitHub Container Registry). Verify by navigating to the Packages tab on the repository page — if it exists, you can push to GHCR.

Step 1 — Add Trivy + SBOM to the existing build job

Open cloud-devops/sample-app/.github/workflows/ci.yml. The L04 version has a build job that builds the image and stops there. The L05 version adds three more steps after the build: a Trivy scan, SBOM generation with Syft, and artifact upload.

The updated build job (replace the existing build: block entirely):

  build:
    runs-on: ubuntu-latest
    needs: [lint, test]
    permissions:
      contents: read
    strategy:
      matrix:
        service: [api, worker]
    outputs:
      api-digest: ${{ steps.build-api.outputs.digest }}
      worker-digest: ${{ steps.build-worker.outputs.digest }}
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - name: Build ${{ matrix.service }} image
        id: build
        uses: docker/build-push-action@v5
        with:
          context: ./cloud-devops/sample-app/${{ matrix.service }}
          push: false
          load: true
          tags: ${{ matrix.service }}:ci-${{ github.sha }}
          cache-from: type=gha,scope=${{ matrix.service }}
          cache-to: type=gha,mode=max,scope=${{ matrix.service }}
      - name: Trivy scan
        uses: aquasecurity/trivy-action@0.24.0
        with:
          image-ref: ${{ matrix.service }}:ci-${{ github.sha }}
          severity: HIGH,CRITICAL
          exit-code: 1
          ignore-unfixed: true
      - name: Generate SBOM
        uses: anchore/sbom-action@v0
        with:
          image: ${{ matrix.service }}:ci-${{ github.sha }}
          format: spdx-json
          output-file: sbom-${{ matrix.service }}.spdx.json
      - uses: actions/upload-artifact@v4
        with:
          name: sbom-${{ matrix.service }}
          path: sbom-${{ matrix.service }}.spdx.json

What each new step does:

• load: true — loads the built image into the runner’s local Docker daemon instead of pushing to a registry. The next two steps (Trivy and Syft) need the image locally.
• Trivy scan — scans the local image for OS and language-package CVEs. exit-code: 1 fails the job if HIGH or CRITICAL findings are present and a fix is available. ignore-unfixed: true skips CVEs with no upstream patch.
• Generate SBOM — Syft scans the local image and produces an SPDX-JSON file listing every package in the image.
• upload-artifact — stores the SBOM file as a downloadable artifact on the Actions run, named sbom-api or sbom-worker.

Step 2 — Create security.yml

Cosign keyless signing requires the id-token: write permission, which allows the workflow to request a GitHub OIDC token. Granting this permission in ci.yml would widen the blast radius of the pull-request workflow (fork PRs get reduced permissions anyway, and granting this in the main CI file is bad practice). The cleaner pattern is a separate workflow that triggers only on push to main.

Create cloud-devops/sample-app/.github/workflows/security.yml:

name: Security & Sign

on:
  push:
    branches: [main]

permissions:
  contents: read
  id-token: write   # cosign keyless OIDC
  packages: write   # push to GHCR

concurrency:
  group: security-${{ github.ref }}
  cancel-in-progress: false   # never cancel a publish in flight

jobs:
  sign:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        service: [api, worker]
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Build & push ${{ matrix.service }}
        id: build
        uses: docker/build-push-action@v5
        with:
          context: ./cloud-devops/sample-app/${{ matrix.service }}
          push: true
          tags: |
            ghcr.io/${{ github.repository }}/${{ matrix.service }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}/${{ matrix.service }}:latest
          cache-from: type=gha,scope=${{ matrix.service }}
          cache-to: type=gha,mode=max,scope=${{ matrix.service }}
      - uses: sigstore/cosign-installer@v3
      - name: Sign image (keyless)
        env:
          COSIGN_EXPERIMENTAL: "true"
        run: |
          cosign sign --yes \
            ghcr.io/${{ github.repository }}/${{ matrix.service }}@${{ steps.build.outputs.digest }}
      - name: Generate SBOM and attach as attestation
        uses: anchore/sbom-action@v0
        with:
          image: ghcr.io/${{ github.repository }}/${{ matrix.service }}:${{ github.sha }}
          format: spdx-json
          output-file: sbom-${{ matrix.service }}.spdx.json
      - name: Attach SBOM attestation with cosign
        env:
          COSIGN_EXPERIMENTAL: "true"
        run: |
          cosign attest --yes --predicate sbom-${{ matrix.service }}.spdx.json \
            --type spdxjson \
            ghcr.io/${{ github.repository }}/${{ matrix.service }}@${{ steps.build.outputs.digest }}

Walking through the permissions block:

• id-token: write — allows the workflow to request a GitHub OIDC token. cosign uses this token to authenticate to Sigstore’s Fulcio CA and obtain a short-lived code-signing certificate. Without this, keyless signing fails immediately.
• packages: write — allows GITHUB_TOKEN to push images to GHCR for this repository. Without this, docker push returns 403.
• cancel-in-progress: false — unlike the CI workflow, we never cancel a security/publish run in flight. Cancelling mid-push could leave a partially signed image in the registry.

Step 3 — Commit, push, observe

git checkout -b security/initial-gates
git add cloud-devops/sample-app/.github/workflows/ci.yml \
        cloud-devops/sample-app/.github/workflows/security.yml
git commit -m "ci: add security gates and image signing"
git push -u origin security/initial-gates

Open a PR. Navigate to Actions and watch the CI workflow run. The build matrix now shows additional steps after the Docker build: Trivy scan and Generate SBOM. Both should pass on the current sample-app images.

Step 4 — Trigger a Trivy failure

Let’s verify that the gate actually works. Edit cloud-devops/sample-app/api/Dockerfile and change the base image to an old version known to have CVEs:

# Change the FROM line to:
FROM python:3.9-slim-buster

Commit and push to the same branch. Watch the Trivy scan step fail with HIGH/CRITICAL findings. Then revert the change (git revert HEAD or restore the original FROM python:3.11-slim) and push again.

Expected: after the revert, Trivy passes and the build goes green again.

Step 5 — Merge and watch the security workflow

Merge the PR to main (approve it yourself if you haven’t configured branch protection yet). Navigate to Actions → Security & Sign. Two matrix legs run in parallel: api and worker. Each should:

1. Log in to GHCR with GITHUB_TOKEN.
2. Build and push the image to ghcr.io/<org>/<repo>/<service>:<sha> and :latest.
3. Install cosign.
4. Sign the image keylessly (look for the Fulcio certificate in the log).
5. Generate the SBOM and attach it as a cosign attestation.

After the run, navigate to the Packages tab on the repository. You should see api and worker packages with the latest and SHA tags.

Step 6 — Verify the signature locally

Install cosign on your local machine:

# macOS
brew install cosign

# Linux (see https://docs.sigstore.dev/cosign/system_config/installation/ for other methods)
curl -sSfL https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64 -o cosign
chmod +x cosign && sudo mv cosign /usr/local/bin/

Verify the API image (substitute your repository path):

ORG=your-org
REPO=your-repo
cosign verify \
  --certificate-identity-regexp "https://github.com/$ORG/$REPO/.*" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  ghcr.io/$ORG/$REPO/api:latest

Expected output: cosign prints a JSON verification result showing the workflow identity that signed the image, e.g.:

[{"critical":{"identity":{"docker-reference":"ghcr.io/your-org/your-repo/api"},...},
  "optional":{"Issuer":"https://token.actions.githubusercontent.com",
              "Subject":"https://github.com/your-org/your-repo/.github/workflows/security.yml@refs/heads/main"}}]

Step 7 — Enable branch protection

This step is done in the GitHub UI and cannot be scripted without a token. Navigate to:

Settings → Branches → Add branch protection rule

1. Branch name pattern: main
2. Tick Require status checks to pass before merging. In the search box, add: lint, test (3.11), test (3.12), build (api), build (worker). These are the exact job names from ci.yml. (The checks only appear in the search box after they have run at least once on the repository.)
3. Tick Require a pull request before merging with 1 required approval.
4. Save.

What you should leave with

• CVE-scanned and SBOM’d images on every PR build (via ci.yml).
• Signed images on every push to main (via security.yml), with the SBOM attached as a verifiable attestation.
• The ability to verify image provenance locally with cosign verify.
• Branch protection requiring the CI check suite to pass before merging.

Troubleshooting