Lesson 2 of 8 · 10 min · Foundations of CI/CD

CI, CD and the stages of a pipeline

Covers: Concepts; the following lessons put them into practice. Updated 13 Nov 2026. Part of the DevOps Engineer roadmap.

What you will learn

  • Continuous integration, continuous delivery and continuous deployment, and how they differ
  • The stages of a pipeline and what belongs in each
  • Principles: fast feedback, build once, fail fast, same process everywhere
  • How tests are layered (the test pyramid)
  • The four DORA metrics

In simple words

Without automation, releasing software is a long manual checklist where people forget steps. A pipeline is that checklist run by a machine, the same way every time: build the code, test it, package it, deliver it. This lesson explains the three terms everyone mixes up and what belongs in each stage.

Words to know

  • CI (continuous integration): merging everyone's changes often, and building and testing each one automatically.
  • CD (continuous delivery): keeping every tested change ready to release at the press of a button.
  • Pipeline: the automatic series of steps from a code change to a release.
  • Stage: one step of the pipeline, such as build, test or deploy.

Three terms

  • Continuous Integration (CI): developers merge small changes into the shared branch frequently, and every change is automatically built and tested. The point is to find integration problems within minutes. A broken main branch is fixed immediately.
  • Continuous Delivery (CD): every change that passes the pipeline is releasable: automatically built, tested and deployed to staging, ready for production at the push of a button (a manual approval).
  • Continuous Deployment: like delivery, but the final step is automatic too: every change that passes all checks goes to production without a human clicking. It needs strong tests, monitoring and safe release techniques.

Most teams practise CI everywhere, continuous delivery commonly, and continuous deployment where their risk tolerance and tests allow.

Why bother

Manual releases are slow, risky and rare, so each release is large, and large changes break more. Automation turns releasing into a boring, frequent, small event: bugs are found sooner, fixes ship faster, and fewer things change at once, so problems are easy to locate. It also gives you an audit trail: who changed what, which tests passed, and what is running where.

The stages

A typical pipeline (stages run in order; a failure stops the pipeline):

commit ─▶ build ─▶ unit tests ─▶ static analysis & security scans ─▶ package ─▶
          deploy to staging ─▶ integration / smoke tests ─▶ [approval] ─▶ deploy to production ─▶ verify
Stage What happens Examples
Checkout get the exact commit git checkout, shallow clone
Build compile or assemble, resolve dependencies mvn package, npm ci && npm run build, go build, docker build
Unit tests fast tests of code in isolation pytest, JUnit, go test
Static analysis style, bugs, complexity linters, SonarQube, shellcheck, hadolint
Security scans vulnerable dependencies, secrets in code, image scan Trivy, Dependabot, gitleaks, CodeQL
Package / publish produce the deployable artifact once and store it push a Docker image, publish a jar or wheel, tag with the commit id
Deploy to staging install the artifact in a production-like environment Helm, Terraform, Ansible, kubectl
Integration / end-to-end / smoke tests test the deployed system API tests, Playwright, health checks
Approval (delivery) a human gate for production GitHub environment reviewers
Deploy to production release with a safe strategy rolling, blue-green, canary
Verify and monitor confirm health, watch metrics, roll back if bad smoke tests, error-rate and latency checks

Treat the pipeline as code: it lives in the repository, is reviewed like any change, and is versioned with the application.

Design principles

  1. Fast feedback. A developer waiting 40 minutes for CI will batch changes and avoid running it. Aim for the first signal (lint, unit tests) in under 10 minutes. Run independent jobs in parallel, cache dependencies, and split slow suites.
  2. Fail fast. Run cheap checks first (lint, unit tests) so an obvious mistake does not wait for a 20-minute integration suite.
  3. Build once, deploy many. Build the artifact once per commit, tag it with the commit id, and promote the same artifact through staging and production. Rebuilding per environment means production runs something that was never tested. Configuration differs per environment; the artifact does not.
  4. Everything in version control: the code, the pipeline, the infrastructure definitions, the environment configuration (not secrets).
  5. Same process everywhere: the same scripts locally and in CI (put logic in scripts or a Makefile, not in 200 lines of CI YAML), so "works on my machine" and "fails in CI" are comparable.
  6. Deterministic and reproducible: pinned dependency versions (lock files), pinned base images, no reliance on "latest".
  7. Keep the main branch green. If it breaks, stop and fix or revert at once.
  8. Make pipelines observable: logs, timing per stage, flaky-test tracking, notifications to the right channel.
  9. Least privilege: the pipeline gets only the permissions it needs, and production credentials are available only to production jobs.
  10. Rollback is part of the design (see the release lessons).

Tests: the pyramid

Many fast, cheap tests at the bottom; few slow, expensive ones at the top:

        /\        end-to-end / UI tests     few, slow, brittle
       /  \       integration tests         some: real database, real HTTP
      /____\      unit tests                many, milliseconds, no I/O

Unit tests are cheap and precise; end-to-end tests are realistic but slow and flaky. Flaky tests (sometimes pass, sometimes fail with no change) destroy trust in the pipeline: quarantine or fix them quickly rather than re-running until green. Add smoke tests after deployment (one or two requests proving the app is up and talking to its dependencies) and contract tests for APIs between services.

Environments and promotion

Common chain: dev → test/QA → staging → production. Staging should resemble production (same images, similar configuration and data shape), because "it worked in staging" only means something if staging is realistic. Promote by deploying the same artifact with different configuration. Use ephemeral environments (a temporary environment per pull request, destroyed after merge) to test changes safely with containers and infrastructure-as-code.

Measuring delivery: the DORA metrics

Research by DORA (DevOps Research and Assessment) identified four measures that predict software delivery performance:

Metric Meaning
Deployment frequency how often you release to production
Lead time for changes time from commit to running in production
Change failure rate share of deployments that cause a failure needing a fix or rollback
Time to restore service how quickly you recover from a failure

The first two measure speed; the last two measure stability. The finding that surprised many: elite teams are better at both. Small, frequent, automated changes are safer than big, rare ones. Track these in your own organisation.

Common mistakes

  • Slow pipelines that people work around.
  • Rebuilding for each environment, so what you tested is not what you ship.
  • Putting all logic in CI-vendor YAML that cannot be run locally.
  • Ignoring flaky tests, or re-running until green.
  • Production credentials available to every job and every branch.
  • No rollback plan, or a manual-only deployment that nobody dares to run.
  • Treating the pipeline as "done" instead of maintaining it like product code.

Try it yourself

  1. Draw the pipeline for an application you know, with each stage, its tool, and the time it takes. Mark where you could parallelise or fail faster.
  2. Take a repository and write the commands (not CI syntax) that build, test and package it in a Makefile or script. That is your pipeline's core.
  3. Identify which of your tests are unit, integration and end-to-end, and what percentage of the run time each takes.
  4. Estimate your own team's four DORA metrics roughly.

Check yourself

What is the difference between continuous delivery and continuous deployment?

In continuous delivery every change that passes is ready to release and a person triggers (approves) production deployment. In continuous deployment the release to production is automatic for every passing change.

Why "build once, deploy many"?

The same tested artifact should reach production. Rebuilding per environment can produce different artifacts, so tests in staging no longer prove anything about production. Only configuration should differ.

Why run unit tests before integration tests?

They are fast and cheap and catch most errors, so failures are reported within minutes without waiting for slower, costlier stages.

Name the four DORA metrics.

Deployment frequency, lead time for changes, change failure rate, and time to restore service.

Interview questions

Describe a CI/CD pipeline you would build for a containerised web service.

On pull request: lint, unit tests, dependency and secret scans, build the image. On merge to main: build and tag the image with the commit SHA, scan it, push to a registry, deploy to staging, run smoke and integration tests, then deploy to production with approval using a rolling or canary strategy, verify health, and roll back automatically on failure.

How do you keep a pipeline fast as the codebase grows?

Cache dependencies and build layers, run jobs in parallel, run only affected tests or build only changed modules, split test suites across runners, use bigger or more runners, and keep slow end-to-end tests out of the pull-request path.

What is a flaky test and how do you handle it?

A test that passes and fails without code changes, due to timing, shared state or external dependencies. Fix the root cause or quarantine it with a ticket; never normalise re-running until green.

Summary

  • CI integrates and tests every change; CD keeps every change releasable (delivery) or deploys it automatically (deployment).
  • Pipelines: build, test, scan, package once, deploy to staging, verify, approve, deploy to production, monitor.
  • Principles: fast feedback, fail fast, build once, everything as code, least privilege, planned rollback.
  • Measure with DORA metrics: frequency, lead time, change failure rate, time to restore.

Going deeper

See how much of this you can explain

A free mock interview asks you new questions on CI/CD pipelines and releases for DevOps Engineer and scores every answer, with what to fix.