Lesson 9 of 9 · 11 min · Securing containers

Scanning images and keeping secrets out of containers

Covers: Docker Engine 29 and BuildKit; Trivy (aquasec/trivy) with its vulnerability database; python 3.12-slim, alpine 3.20. Updated 13 Nov 2026. Part of the DevOps Engineer roadmap.

What you will learn

  • What an image vulnerability scanner does and its limits
  • Running Trivy and reading its output
  • A realistic process for fixing and tracking findings
  • Where secrets leak in container workflows
  • Safe ways to give a container a secret at build time and at run time

In simple words

An image contains software written by many other people, and some of it has known weaknesses. A scanner checks every piece against a public list of those weaknesses and tells you what to update. This lesson also covers the other common leak: passwords and keys that end up saved inside an image.

Words to know

  • Vulnerability: a known weakness in software that an attacker could use.
  • Scanner: a tool that lists the known weaknesses inside an image. Trivy is one.
  • CVE: the public ID of one known vulnerability, such as CVE-2024-12345.
  • Secret: a password, token or key. It must never be stored in an image or in Git.

Scanning images for vulnerabilities

Images contain operating-system packages and application libraries. New vulnerabilities (CVEs) are found in them every day. A scanner lists the packages in an image (an SBOM, software bill of materials), compares versions with vulnerability databases, and reports matches with severity (LOW, MEDIUM, HIGH, CRITICAL), the installed version and the fixed version, if any.

Popular tools: Trivy (open source, one binary), Grype, Docker Scout, Snyk, and registry scanners (Amazon ECR scanning, Harbor). Run a scan:

  • locally while developing: trivy image myapp:1.4.2,
  • in CI on every build, failing the pipeline for fixable high-severity findings,
  • continuously in the registry, because an image that was clean yesterday can be vulnerable today when a new CVE is published.
docker run --rm -v trivycache:/root/.cache aquasec/trivy:latest image \
  --severity HIGH,CRITICAL --scanners vuln python:3.12-slim

(The first run downloads the vulnerability database, 133 MB; the cache volume reuses it afterwards.) The report is a table: the target and OS, then one row per finding with the package (Library), Vulnerability id, Severity, Status, Installed Version, Fixed Version and a title. The summary line for our test:

python:3.12-slim (debian 13.7)
Total: 49 (HIGH: 49, CRITICAL: 0)

Comparing bases the same day (numbers change daily; the pattern does not):

Image HIGH + CRITICAL findings
python:3.12-slim (Debian) 49
python:3.12-alpine 16
alpine:3.20 2
gcr.io/distroless/static-debian12 none reported

This shows what "smaller is safer" means: fewer packages, fewer findings.

Reading results sensibly

A raw count is not a risk score. Triage:

  1. Status and fix. fixed means a patched version exists: rebuild with updated packages (new base image tag, or apt-get upgrade in the build). affected or will_not_fix / fix_deferred means the distribution has not fixed it yet; you can only mitigate, or accept the risk with a record.
  2. Is it reachable? A flaw in a package your app never uses (a Perl library in a Python image) is a lower priority than one in your web framework. Cutting unused packages (minimal images) is the real fix.
  3. Severity and exploitability. Prioritise HIGH/CRITICAL that are remotely exploitable or listed as exploited in the wild.
  4. Application dependencies too. Scan language dependencies (pip, npm, Maven) with --scanners vuln on the filesystem or image. Many incidents are in libraries, not in the OS.
  5. Track and expire exceptions. If you accept a risk, write it in an ignore file (.trivyignore) with a reason and a date.

A sustainable process: pin base images by version; rebuild on a schedule (weekly) and when a base image updates; scan in CI and in the registry; fail builds on fixable HIGH/CRITICAL findings; automate base-image updates with tools like Dependabot or Renovate; keep a short list of approved base images.

Also scan the Dockerfile and configuration: Trivy's config scanner and tools like Hadolint flag USER root, latest tags, missing health checks and similar issues. Sign images and verify signatures (cosign) so only builds from your pipeline can run.

Secrets in containers

A secret is anything that grants access: passwords, API keys, tokens, private keys, certificates. Containers make leaking them easy.

How secrets leak

We built an image with a secret as a build argument and an environment variable:

FROM alpine:3.20
ARG API_KEY
ENV DB_PASSWORD=hunter2
RUN echo "key is $API_KEY" > /dev/null

built with --build-arg API_KEY=abc123SECRET. Anyone with the image can read both:

$ docker history --no-trunc sec-leak
RUN |1 API_KEY=abc123SECRET /bin/sh -c echo "key is $API_KEY" > /dev/null
ENV DB_PASSWORD=hunter2
ARG API_KEY=abc123SECRET

$ docker inspect sec-leak --format '{{.Config.Env}}'
[... DB_PASSWORD=hunter2]

So:

  • ENV secrets are baked into the image and visible to everyone who can pull it and in docker inspect of every container.
  • ARG values appear in docker history. ARG is not a secure channel.
  • COPY .env or COPY id_rsa followed by RUN rm leaves the file in an earlier layer, recoverable by anyone who can pull the image. A .dockerignore stops accidental copies.
  • Environment variables at run time (-e TOKEN=...) are safer than baking them in, but still visible to docker inspect (we read TOKEN=s3cr3t from a running container's config), to anyone who can exec into the container (env, /proc/1/environ), in crash dumps, and often in logs or error pages if the app prints its environment.
  • Git: committing .env files or keys, then later "deleting" them, leaves them in history.
  • Logs and CI output echoing commands that contain secrets.

If a secret has leaked into an image, a registry, Git or a log: assume it is compromised, rotate it, and only then clean up.

Build-time secrets: RUN --mount=type=secret

When the build needs a credential (a private package registry token), use BuildKit secrets. The secret is mounted into one RUN step as a file and never stored in any layer:

# syntax=docker/dockerfile:1
FROM alpine:3.20
RUN --mount=type=secret,id=api_key sh -c 'echo "using key of length $(wc -c < /run/secrets/api_key)"'
docker build --secret id=api_key,src=key.txt .

The build printed using key of length 12, and a search of docker history --no-trunc for the value found 0 matches. For SSH keys use --mount=type=ssh. Also prefer multi-stage builds, so even a credential used in an early stage never reaches the final image.

Run-time secrets: files, not environment

Give the application the secret at start-up from outside the image:

  1. Mounted files: mount a secret as a read-only file and let the app read the path. Many images support *_FILE variables for this (POSTGRES_PASSWORD_FILE=/run/secrets/db_password).
docker run --rm -v "$PWD/tok:/run/secrets/token:ro" alpine:3.20 cat /run/secrets/token

Compose has a secrets: section that mounts secrets under /run/secrets/ as in-memory files, and Kubernetes has Secret objects mounted as files or exposed as variables (with the caveat that Kubernetes Secrets are base64-encoded, not encrypted, unless you enable encryption at rest or use an external store). 2. A secrets manager: AWS Secrets Manager or SSM Parameter Store, HashiCorp Vault, Azure Key Vault, Google Secret Manager. The container's identity (an IAM role, a Kubernetes service account) fetches the secret at start-up, so the secret never lives in your repository, image or Compose file, can be rotated centrally, and access is audited. This is the production answer. 3. Short-lived credentials: prefer tokens that expire (role-based access, OIDC from CI) over long-lived keys. A leaked token that expires in an hour is a small problem.

Habits

  • Never commit secrets; add .env and key files to .gitignore and .dockerignore; commit a .env.example with fake values.
  • Scan repositories for secrets (gitleaks, trufflehog, GitHub secret scanning) and block commits that contain them.
  • One secret per service and environment, least-privilege access, rotation plans.
  • Never log secrets; mask them in CI output.
  • Do not pass secrets on the command line of a process: other users can see arguments in ps.

Common mistakes

  • Putting passwords in ENV or ARG in the Dockerfile.
  • Copying a secret and deleting it in a later layer.
  • Treating scanner output as a pass/fail count without reading status and fixes.
  • Scanning once at build time and never again.
  • Ignoring application dependencies, scanning only the OS.
  • Storing the same long-lived key everywhere (laptops, CI, images).
  • Assuming Kubernetes Secrets are encrypted by default.

Try it yourself

  1. Scan an image of your own with Trivy (--severity HIGH,CRITICAL). Rebuild on a newer base image and compare the counts.
  2. Build an image with a secret in ARG and ENV, then find it with docker history --no-trunc and docker inspect.
  3. Rebuild it using RUN --mount=type=secret and prove with docker history that the value is absent.
  4. Run a container with a secret file mounted read-only, and make your app read it from the path instead of from an environment variable.

Check yourself

Why can a clean scan become a vulnerable image a week later?

New vulnerabilities are published against packages already in the image. The image did not change, but the databases did, so scanners must run regularly and images must be rebuilt.

Name three places a secret can leak in a Docker workflow.

Image layers (COPY, ENV, ARG in docker history), container environment variables (docker inspect, /proc), and Git history or CI logs.

How does RUN --mount=type=secret protect a build-time credential?

The secret is mounted as a temporary file only for that RUN instruction and is never written to a layer, so it does not appear in the image or its history.

Why is docker run -e PASSWORD=... better than ENV in the Dockerfile but still not ideal?

It keeps the secret out of the image, but it remains visible in docker inspect, to anyone who can exec into the container, and potentially in logs. Mounted files or a secrets manager are better.

Interview questions

How do you handle vulnerabilities in container images in a CI/CD pipeline?

Use a minimal pinned base, scan the image (and dependencies) in CI with a tool like Trivy, fail the build on fixable HIGH/CRITICAL findings, rebuild regularly and when base images update, rescan images in the registry, and track exceptions with expiry dates.

How would you give a containerised app its database password in production?

Fetch it from a secrets manager using the workload's identity (IAM role, Vault, Kubernetes secret store CSI), or mount it as a file; never bake it into the image or commit it, and rotate it periodically.

A developer committed an API key to Git and pushed an image containing it. What do you do?

Revoke and rotate the key immediately, check logs for misuse, remove it from the repository history and delete the affected image tags, and add controls (secret scanning, pre-commit hooks, .dockerignore) to prevent a repeat.

Summary

  • Scanners compare an image's packages with vulnerability databases; read status and fixes, rebuild on fixed versions, scan continuously, and prefer minimal images.
  • Secrets in ENV, ARG, copied files and environment variables leak through history, inspect and layers.
  • Use BuildKit secret mounts at build time; mounted files, secret stores and short-lived credentials at run time.
  • If a secret leaks, rotate first.

Going deeper

See how much of this you can explain

A free mock interview asks you new questions on Docker and containers for DevOps Engineer and scores every answer, with what to fix.