Lesson 06 Push-to-Deploy
Theme:

Push-to-Deploy

Lesson 06 · Cloud Infrastructure & DevOps · ~12 min read

1. The deploy step in context

Lesson 04 gave us green CI on every pull request: lint, build, and test run automatically and block the merge button when they fail. Lesson 05 added the security layer: every merge to main triggers a Trivy vulnerability scan, an SBOM attestation, and a cosign signature on the resulting image pushed to GHCR. At the end of L05, we have a verified, signed image sitting in the registry — provably built by our CI pipeline, untampered-with, and ready to run.

What we do not have is anything that pulls that image and runs it. The deploy step’s job is exactly that: make the image run on real infrastructure. This lesson builds the simplest realistic shape of that step — SSH to a VM running docker-compose, pull the new image by digest, run docker compose up -d. It is intentionally simple because Modules 3 and 4 replace the SSH target with a Terraform-managed VM and then a Kubernetes cluster respectively. The deploy pattern (verify → gate → execute) is what this lesson is really about; the mechanics of the target will evolve.

By the end of this lesson the sample app has a third workflow file — deploy.yml — that closes the loop from a pushed commit to a running container, with a manual approval gate via GitHub Environments and concurrency protection against deploy races.

2. Registries: GHCR, ECR, others

A container registry is where you store and distribute images. Before a deploy step can pull an image onto a target machine, the image must be pushed to a registry the target can reach. The registry choice affects authentication, cost, and how tightly the workflow integrates with your cloud provider.

GHCR (GitHub Container Registry, ghcr.io) is the natural choice when your code is on GitHub. The GITHUB_TOKEN that Actions automatically injects can authenticate a push without any extra credential setup — you just add docker/login-action@v3 with registry: ghcr.io and password: ${{ secrets.GITHUB_TOKEN }}. Images are scoped to your repository and visible alongside your code in the GitHub UI. It is free for public repositories and counts against your organization’s storage and bandwidth quotas for private ones. GHCR is what all course labs use.

ECR (Amazon Elastic Container Registry) is the AWS-native equivalent. Authentication is IAM-based. The modern way to authenticate from GitHub Actions is OIDC federation via aws-actions/configure-aws-credentials@v4: GitHub issues a short-lived OIDC token, AWS exchanges it for temporary IAM credentials, and you never store long-lived AWS access keys in GitHub Secrets. ECR is the right choice when your deployment target is an EC2 instance or ECS cluster — the instance role or task role can pull from ECR without any credential plumbing. The lab guide includes a side note on the ECR variant for learners deploying on AWS.

Docker Hub is the original public registry and still the easiest to use for personal projects. The free tier is rate-limited (currently 100 anonymous pulls per six hours per IP, 200 authenticated) which can cause CI failures if your jobs pull ubuntu:latest too frequently. For course labs, GHCR avoids those limits entirely since it uses your GITHUB_TOKEN auth implicitly.

3. GitHub Environments and scoped secrets

A GitHub Environment is a named deployment target that you configure under Settings → Environments in your repository. Environments have three capabilities that make them the right place to hold production credentials: scoped secrets, required reviewers, and deployment policies.

Scoped secrets are the key feature. A secret stored in an environment is only injected into a job that declares environment: <name>. Every other job in every other workflow runs without access to those secrets, even if it belongs to the same repository. This is a meaningful security improvement over repository-level secrets, which are accessible to all workflows. Your DEPLOY_HOST, DEPLOY_USER, and DEPLOY_KEY belong in the production environment, not in repository secrets.

Required reviewers add a human gate. When you designate a reviewer for an environment (your own GitHub username works for solo projects), any job that opts into that environment will pause and display an “Approve deployment” prompt. The job does not proceed — and the environment secrets are not injected — until an approved reviewer clicks the button in the GitHub Actions UI. This is the mechanism that prevents an accidental push from deploying untested code to production without anyone noticing.

Wait timer adds a minimum soak time between approval and the job starting. If you approve a production deployment and then immediately realise the approval was a mistake, the wait timer gives you a window to cancel.

Deployment branch policies restrict which branches can deploy to an environment. Configuring “only main can deploy to production” prevents feature branches from accidentally reaching production even if the workflow logic has a bug.

A workflow opts into an environment with two lines:

environment:
  name: production
  url: https://shortener.example.com

The url: key is optional but strongly recommended — it powers the “deployed to” link in the GitHub deployment history and in pull request timelines.

4. The deploy step itself

For this course, the simplest realistic deploy shape is SSH to a VM running docker-compose. The target machine already has Docker, docker-compose, and a /opt/shortener/docker-compose.yml that references the GHCR image by SHA. The deploy step pulls the new image and recreates the services:

- name: Deploy to VM (placeholder)
  uses: appleboy/ssh-action@v1
  with:
    host: ${{ secrets.DEPLOY_HOST }}
    username: ${{ secrets.DEPLOY_USER }}
    key: ${{ secrets.DEPLOY_KEY }}
    script: |
      set -euo pipefail
      export IMAGE_TAG=${{ github.event.workflow_run.head_sha }}
      cd /opt/shortener
      docker compose pull
      docker compose up -d
      docker compose ps

The DEPLOY_KEY secret is the contents of the SSH private key file, including the BEGIN and END lines. The appleboy/ssh-action handles multi-line key values; the common mistake is pasting only the key body without the header and footer lines.

What docker compose up -d does on the target: it compares the running container configuration with the compose file. Services whose image or configuration has changed are recreated (stopped and restarted); services that are unchanged are left running. Volumes and networks are never removed by up — only by docker compose down. This means the database data persists across deploys.

The docker compose ps at the end is a simple post-deploy sanity check: if any service fails to start (image pull fails, port conflict, etc.) the status will show as exited or restarting and the CI operator will see it in the logs before walking away.

5. Blue/green deployments

A blue/green deployment maintains two identical environments — conventionally called “blue” and “green” — where only one serves production traffic at a time. The procedure is: deploy the new version to the idle environment; run smoke tests against that environment; flip the router to point production traffic at it. Rollback is the same flip in reverse — zero redeployment time, zero compile time.

The key benefit is zero-downtime cutover. Traffic is never absent; during the flip the router atomically reroutes. The secondary benefit is instant rollback: the old environment stays running, fully warmed, until you’re confident in the new one. When something goes wrong, you don’t re-deploy the old version — you just flip the pointer back.

The cost is approximately 2× infrastructure. You need two VMs (or two ECS services, or two Kubernetes deployments), both running at all times. For production workloads that can tolerate the cost, this is usually worth it. For development and staging environments, the single-environment pattern is typically fine.

A canary deployment is the cheaper sibling: instead of flipping 100% of traffic at once, you send 5% to the new version, observe error rates, then ramp up in stages (5% → 25% → 100%). This requires a load balancer that supports weighted routing. Canary is a progressive delivery pattern; blue/green is an all-or-nothing swap. Both avoid in-place replacement, which is the riskiest deploy pattern.

The L06 lab shows the shape of a blue/green workflow in sketch form. Building the actual routing primitive — a load balancer rule or a Kubernetes ingress annotation — requires L11 and L13 respectively.

6. Deploying by digest, not by tag

Container image tags are mutable. The image at ghcr.io/org/repo/api:latest today is probably different from the image at ghcr.io/org/repo/api:latest tomorrow. If you deploy :latest, you cannot deterministically answer the question “what version is running?” after a tag-overwriting push. More critically, you cannot reproducibly roll back to “the previous :latest” because the tag no longer points there.

Image digests are immutable. A digest is a SHA-256 hash of the image manifest — it is computed from the content and cannot change without producing a different hash. An image reference like ghcr.io/org/repo/api@sha256:abc123... always refers to exactly the same image bytes, forever. When combined with L05’s cosign signature (which is also over the digest), you get a deployment that is both immutable and verifiable: the deployed image matches what CI built and signed, and you can prove it.

The practical workflow: the Security & Sign job outputs the digest as a job output; the deploy job reads that output and passes it as the IMAGE_TAG on the target. The deploy target’s docker-compose.yml pins the image by digest rather than by tag. The Homework Task 2 walks through this change.

7. Rollback strategy

Three patterns cover the majority of rollback scenarios, each with a different speed-vs-complexity tradeoff.

Re-deploy the previous digest. Find the previous successful Deploy workflow run in the Actions UI and click “Re-run all jobs”. If the deploy job reads the digest from an artifact uploaded by Security & Sign, the re-run will use the old artifact and therefore deploy the old digest. This is the simplest pattern; its only requirement is that you have a record of previous digests, which workflow artifacts provide for 90 days by default.

Blue/green flip. If you maintain blue and green environments, simply re-run the router flip in reverse. The idle environment (now blue, which was serving production before the flip) is still running the old version. No re-deploy, no CI, no waiting for builds. This is the fastest pattern. Its requirement is that the old environment must stay running after the flip — you cannot scale it down immediately.

Git revert plus auto-deploy. Create a revert commit on the offending commit and push it to main. The CI pipeline will run, Security & Sign will build and sign the reverted image, and the Deploy job will deploy it automatically. This is the slowest pattern (pipeline time is additive) but the most consistent with GitOps principles: the repository state remains the authoritative description of what is deployed. L14 (GitOps and Flux) leans heavily on this approach.

For most teams, the right answer is: keep blue/green for production (fastest rollback), use digest re-deploy as the fallback, and reserve Git revert for cases where you want the rollback recorded as a deliberate action in the commit history.

Common pitfalls

  1. Deploying without verifying the signature. The cosign verify steps in deploy.yml must run before the SSH step. If you comment them out “for speed”, you have no assurance that the image you’re deploying is the one CI built. Lesson 05’s work is only meaningful if the verify step is present and passing at deploy time.
  2. Hard-coding production hostnames in the workflow. Use environment.url and secrets.DEPLOY_HOST; the production hostname is sensitive configuration that may change (e.g., behind a load balancer). Hard-coding it in the workflow YAML leaks it into the public commit history for public repos and makes rotation painful for private ones.
  3. No concurrency control on the deploy workflow. Two merges to main in quick succession both trigger deploy.yml. Without a concurrency: block, both jobs try to SSH to the target simultaneously and can leave it in an inconsistent state (two docker compose up -d runs interleaving). Add concurrency: { group: deploy-production, cancel-in-progress: false } to queue deploys and never cancel a running one.

References