Lesson 8 of 8 · 12 min · Structure and safety
Terraform in teams and CI/CD
What you will learn
- Why
terraform applyfrom laptops does not scale - A plan-on-pull-request, apply-on-merge workflow
- A complete GitHub Actions pipeline
- Cloud credentials via OIDC with separate plan and apply roles
- Policy, security scanning, cost estimation and drift detection
- Rules of engagement for production
In simple words
One person running Terraform from a laptop works for learning. In a team it causes accidents: nobody reviews the change and nobody knows who ran what. The safer way is to let the pipeline do it: show the plan when a change is proposed, and apply it only after someone approves.
Words to know
- Plan on pull request: showing the preview of changes automatically when code is proposed.
- Apply on merge: making the change only after the code is reviewed and merged.
- OIDC: lets the pipeline get short-lived cloud access without a stored password.
- Policy check: an automatic rule, such as "no storage may be public", tested before applying.
The problem with laptops
When anyone can apply from their laptop, you get: different Terraform and provider versions, credentials with broad permissions scattered on personal machines, changes nobody reviewed, no record of who changed what, and two people applying at once. Mature teams make the pipeline the only way to change shared environments, exactly like application deployment (CI/CD course).
The workflow
developer branch ──▶ pull request ──▶ CI: fmt, validate, scan, PLAN ──▶ review the plan ──▶ merge to main
│
production approval (human) ◀────────┘
──▶ APPLY the reviewed plan ──▶ outputs / notifications
- A change is made on a branch and pushed.
- The pull request triggers CI:
fmt -check,validate, linters and security scanners, andterraform plan, whose output is posted to the pull request for reviewers. - Reviewers read both the code diff and the plan. The plan is the truth: it shows the replacements and destroys the code diff hides (a one-line change can destroy a database).
- After merge to
main, the apply job runs, for production behind a manual approval (environment protection), and applies the plan. - Failures notify the team; scheduled plans detect drift.
A tested pipeline
.github/workflows/terraform.yml (checked with actionlint, no problems):
name: Terraform
on:
pull_request:
paths: ["infra/**"]
push:
branches: [main]
paths: ["infra/**"]
permissions:
contents: read
id-token: write
pull-requests: write
concurrency:
group: terraform-${{ github.ref }}
cancel-in-progress: false
env:
TF_IN_AUTOMATION: "true"
jobs:
plan:
runs-on: ubuntu-24.04
defaults:
run:
working-directory: infra
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.17.2
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/terraform-plan
aws-region: ap-south-1
- run: terraform fmt -check -recursive
- run: terraform init -input=false
- run: terraform validate
- run: terraform plan -input=false -out=tfplan
- run: terraform show -no-color tfplan > plan.txt
- uses: actions/upload-artifact@v4
with:
name: tfplan
path: |
infra/tfplan
infra/plan.txt
retention-days: 7
apply:
needs: plan
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
runs-on: ubuntu-24.04
environment: production
defaults:
run:
working-directory: infra
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.17.2
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/terraform-apply
aws-region: ap-south-1
- uses: actions/download-artifact@v4
with:
name: tfplan
path: infra
- run: terraform init -input=false
- run: terraform apply -input=false tfplan
Points worth understanding:
TF_IN_AUTOMATIONand-input=falsemake Terraform non-interactive and quieter.- Pinned Terraform version (
1.17.2) matches what developers use (required_versionin code); the lock file pins providers. - Two IAM roles:
terraform-planis read-only (enough to refresh and plan; usable by pull requests from branches),terraform-applycan change infrastructure and is assumed only in theapplyjob that targets the protectedproductionenvironment. OIDC trust policies restrict the apply role to themainbranch and that environment (CI/CD course). - Saved plan (
-out=tfplan) passed as an artifact: the apply job applies exactly what the plan job computed in the same run. (If the plan is old and the state changed meanwhile, Terraform refuses it, which is what you want.) Plan files can contain secrets: short artifact retention, restricted access. concurrencywithcancel-in-progress: false: never cancel a running apply midway; queue instead. Together with state locking this prevents overlapping applies.paths: ["infra/**"]: only run when infrastructure files change.- Posting the plan text as a pull-request comment (
gh pr commentor a plan-commenting action) makes review convenient; for big plans link the artifact.
Managed alternatives provide this workflow ready-made: HCP Terraform, Atlantis (self-hosted, atlantis plan/apply comments on pull requests), Spacelift, env0, Terraform in GitLab CI. The principles are identical.
Checks before apply
Add automated gates to the plan job:
| Check | Tool | Catches |
|---|---|---|
| Formatting, validity | terraform fmt -check, validate |
syntax and style |
| Linting | tflint | invalid instance types, deprecated arguments |
| Security scanning | Trivy config, checkov, tfsec | public buckets, open security groups, unencrypted disks |
| Policy as code | OPA/Conftest, Sentinel (HCP), Checkov custom policies | company rules: required tags, allowed regions, no 0.0.0.0/0 on port 22 |
| Cost estimation | Infracost | "this change adds ₹40,000 per month" in the pull request |
| Tests | terraform test |
module behaviour |
Policy checks can run on the plan's JSON (terraform show -json tfplan), which also lets scripts fail the build when the plan contains destroys of protected resource types.
Treat destroy and replace actions in a plan as items needing explicit human acknowledgement. Some teams fail the pipeline automatically if the plan contains deletions unless a label is added to the pull request.
Detecting drift
Schedule the plan job (for example nightly) with -detailed-exitcode: the command exits 0 (no changes), 1 (error) or 2 (changes pending). A nightly run that exits 2 on main means something changed outside Terraform (drift) or an earlier change was not applied. Alert the team, investigate who changed what (cloud audit logs: CloudTrail), then correct either the code or the infrastructure.
Rules of engagement for production
- All changes via pull request; review the plan, not only the code.
- Apply only from the pipeline, from
main, with approval for production. - No console changes except in declared emergencies, and then follow up with a code change to reconcile.
- Small, frequent changes; avoid big-bang refactors.
- Back up and protect state; test restore.
- Pin Terraform, providers and modules; upgrade deliberately in dev first.
- Keep a runbook for failed applies, stuck locks and rollbacks. "Rollback" in Terraform means applying the previous commit's configuration (a revert), which works only if the changes are reversible; data deletion is not.
- Limit blast radius: separate states, accounts and permissions per environment and layer.
- Tag all resources with owner and cost centre.
- Review providers' release notes before upgrades.
Common mistakes
- Applying from laptops to shared environments.
- Approving pull requests from the diff alone, without reading the plan.
- One role with admin rights for plan and apply.
- Applying a fresh plan in the apply job (instead of the reviewed one) so what runs differs from what was reviewed.
- No concurrency control or cancelling applies midway.
- Leaving plan files with secrets in public artifacts.
- No drift detection, so the code slowly stops describing reality.
Try it yourself
- Create a repository with a small Terraform configuration and the workflow above (use a sandbox account or a local-only configuration), and open a pull request to see the plan job.
- Add
tflintand a security scanner as extra steps and fix their findings. - Add a nightly schedule that runs
terraform plan -detailed-exitcodeand fails on exit code 2. - Write down which IAM permissions the plan role and the apply role really need.
Check yourself
Why separate plan and apply roles?
The plan role only needs read access and can be used on every pull request; the apply role can change infrastructure and should be assumable only from the protected main-branch apply job. It limits what a compromised pull-request build can do.
Why apply a saved plan instead of running apply again?
It guarantees that what runs is exactly what was reviewed, and fails if the state has changed in the meantime.
What does terraform plan -detailed-exitcode return?
0 when there are no changes, 1 on error, and 2 when changes are pending, which makes it useful for drift detection in scheduled jobs.
What should reviewers look at in a Terraform pull request?
The plan: especially destroys, replacements (-/+, forces replacement), changes to security rules and IAM, and the resource counts, as well as the code diff.
Interview questions
Describe a production-grade Terraform delivery pipeline.
Pull requests run fmt, validate, linters, security and policy scans, cost estimation and a read-only plan posted for review; merge to main triggers an approval-gated apply of the saved plan using a separate role via OIDC; state is remote, locked and versioned; scheduled plans detect drift; changes to the pipeline itself require review.
How do you stop engineers accidentally destroying production resources?
prevent_destroy and platform deletion protection, plan reviews with automated checks for deletions, restrictive IAM on the apply role, approval gates, separate states and accounts, and no direct console or laptop access to production.
How would you roll back a bad Terraform change?
Revert the commit and run the pipeline to apply the previous configuration, if the change is reversible; for destructive or data-affecting changes restore from backups or snapshots; this is why risky changes need a plan review and protections beforehand.
Summary
- Make the pipeline the only way to change shared infrastructure: plan on pull requests, review the plan, apply from main with approval.
- Use pinned versions, saved plans, OIDC with separate read-only plan and write apply roles, and concurrency control.
- Add fmt/validate, tflint, security and policy scans, cost estimation and tests as gates.
- Detect drift with scheduled plans, protect state and pipeline definitions, and keep blast radius small.
Going deeper
- Terraform automation tutorials: running Terraform in CI/CD pipelines.
- GitHub Actions: setup-terraform: the action used above.
- Atlantis: pull-request automation for Terraform.
- Infracost: cost estimates in pull requests.
- Trivy: misconfiguration scanning: scanning Terraform for security issues.
- AWS: security best practices in IAM: least privilege and roles.
See how much of this you can explain
A free mock interview asks you new questions on Infrastructure as Code with Terraform for DevOps Engineer and scores every answer, with what to fix.