Lesson 4 of 8 · 10 min · State and teamwork
Terraform state, drift and refresh
What you will learn
- What state is and what it stores
- How Terraform uses configuration, state and real infrastructure together
- Detecting and correcting drift
terraform statecommands: list, show, mv, rm- Why state files are sensitive and must never be edited by hand
In simple words
Terraform keeps a notebook called state: a record of everything it created and each item's real ID. Before any change, it compares three things: your files, the notebook, and what truly exists in the cloud. When someone changes things by hand and they no longer match, that is called drift.
Words to know
- State: Terraform's record of the resources it manages.
- State file: the file that holds that record. It can contain secrets, so it must be protected.
- Drift: a difference between what your code says and what really exists.
- Refresh: updating the state from what really exists.
Why Terraform needs state
Your configuration says what you want. The cloud holds what exists. Terraform needs one more thing: a record of which real objects it created for which resource blocks. That is the state, a JSON file (terraform.tfstate) mapping each resource address to a real object's id and attributes:
random_pet.name -> id "immortal-caribou"
aws_vpc.main -> vpc-ffafb4dc, cidr 10.0.0.0/16, tags {...}
State serves several purposes:
- Mapping: resource
aws_vpc.mainin code corresponds tovpc-ffafb4dcin AWS. Without the mapping Terraform could not tell "mine" from "someone else's". - Change detection: the diff of configuration against recorded state decides what to update.
- Dependency and metadata: order of destruction, provider versions, outputs.
- Performance: cached attributes (
-refresh=falseskips querying everything).
Three sources of truth
Every plan compares three things:
configuration (.tf) ──┐
├──▶ plan: the actions needed
state (last known) ────┤
real infrastructure ───┘ (read by refresh during plan)
- Refresh: Terraform asks the provider for the current attributes of every object in state and updates its picture.
- Diff: it compares the refreshed state with the configuration.
- Plan: it proposes the actions that make reality match the configuration.
If an object in state no longer exists in the cloud, the plan will create it again. If something exists in the cloud but not in state, Terraform does not know about it at all (unless you import it).
Looking at state
terraform state list # all resource addresses
terraform state show aws_vpc.main # attributes of one
terraform show # whole state (or a saved plan file)
terraform output # just the outputs
terraform show -json | jq . # machine-readable
Our small project listed:
local_file.greeting
random_pet.name
and for modules and loops, addresses like module.service_conf["web"].local_file.this and aws_subnet.public["a"].
Drift
Drift is when real infrastructure no longer matches the configuration, because someone changed it outside Terraform (console edits, a script, an auto-scaler, another tool, an incident hotfix). Terraform finds it at the next plan. We reproduced it: after apply, someone overwrote the managed file with tampered. The next plan noticed that the object no longer matched what Terraform had recorded and proposed to restore it:
# local_file.greeting will be created
+ resource "local_file" "greeting" { ...
Plan: 1 to add, 0 to change, 0 to destroy.
(For this provider a modified file is treated as gone and recreated; for most cloud resources the plan shows ~ update in-place with the drifted attribute, such as ~ instance_type = "t3.large" -> "t3.micro", changing it back.) apply repaired it: the file again contained Hello from dev.
Responses to drift:
- Revert it: apply the configuration (the usual choice: Git is the source of truth).
- Accept it: change the code to match reality, so the plan is empty.
- Ignore specific attributes that something else legitimately manages:
lifecycle { ignore_changes = [desired_capacity] }(for example an autoscaler changing a count). - Prevent it: restrict console write access, make Terraform the only way, and run a scheduled
terraform planin CI (-detailed-exitcodereturns exit code 2 when changes are pending) that alerts on drift.
terraform plan -refresh-only shows only differences between state and reality without proposing config changes; terraform apply -refresh-only updates the state to match reality. Our check returned "found no differences". In older tutorials you may see terraform refresh, now superseded by this.
State surgery
Sometimes the mapping must be changed without touching the infrastructure:
terraform state mv aws_instance.web aws_instance.app # rename the address
terraform state mv 'aws_subnet.public[0]' 'aws_subnet.public["a"]'
terraform state rm aws_s3_bucket.legacy # forget it (the real bucket stays)
terraform import aws_s3_bucket.legacy my-bucket-name # adopt an existing object
terraform taint / -replace=aws_instance.web # force recreation next apply
Prefer the declarative equivalents in code where available: moved blocks for renames (reviewable in a pull request and applied by everyone), import blocks for adoption, and plan -replace=ADDR for replacement (next lessons). Manual state mv/rm bypasses review; take a backup first (terraform state pull > backup.tfstate). Never edit the JSON state file by hand, and never delete it: without it Terraform forgets everything it manages and a fresh apply tries to create duplicates (or fails on name conflicts).
State contains secrets
State holds every attribute of every resource, including ones marked sensitive (database passwords you passed in, generated keys, connection strings). We verified: after applying a resource with password=s3cr3t, the plain state file contained that string even though plan and output printed (sensitive value). Therefore:
- Never commit state to Git, never put it in a public bucket, never paste it into chat or tickets.
- Store it in an encrypted, access-controlled, versioned remote backend (next lesson).
- Treat read access to state as read access to the secrets it contains.
- Prefer designs where Terraform never sees raw secrets (generated and stored by the cloud's secret manager).
Local state is only for experiments
By default state is a local file next to the configuration. That is fine for learning, and wrong for any team: laptops get lost, two engineers apply from different copies and overwrite each other's changes, CI cannot see it, and there is no locking. The remedy is a remote backend with locking, covered next.
How big should one state be?
Everything in one state file means every plan refreshes everything (slow), and a mistake can touch everything (large blast radius). Split state by lifecycle and ownership: networking, data stores and applications in separate configurations (and often separate repositories or directories) per environment. Share values via outputs and terraform_remote_state or data sources. Rule of thumb: things that change together and are owned together live together; the VPC and the application deployment usually do not.
Common mistakes
- Losing or deleting state.
- Committing state to Git.
- Hand-editing the state file.
- Console changes ("just a quick fix") that become drift and are silently reverted at the next apply.
- One giant state for everything.
- Running
terraform state rm/mvwithout a backup or review. - Believing
sensitiveprotects secrets in state.
Try it yourself
- Apply a small configuration, run
terraform state listandterraform state showon a resource, and openterraform.tfstateto see its structure (without sharing it). - Change a managed object outside Terraform, run
planand read the drift. Fix it by applying. - Run
terraform plan -refresh-onlyandterraform plan -detailed-exitcode; echo $?and explain the exit codes (0 no changes, 1 error, 2 changes pending). - Rename a resource using
state mv, then undo it with amovedblock next time.
Check yourself
What does the state file record and why is it needed?
The mapping between resource blocks and real object ids plus their attributes. It lets Terraform know what it manages, compute differences and order operations.
What is drift and how do you detect it?
Real infrastructure changed outside Terraform so it no longer matches the configuration. terraform plan (which refreshes) reveals it; scheduled plans in CI alert on it.
Why must state be stored securely?
It contains all attribute values, including secrets, in clear text.
What happens if you delete the state file?
Terraform no longer knows what it created. A new apply tries to create everything again (duplicates or conflicts) and the old objects become unmanaged. You would need to re-import them.
Interview questions
Why should Terraform state not be stored in Git?
It contains sensitive data, changes with every apply (merge conflicts), has no locking, and Git history would preserve secrets forever. Use a remote backend instead.
Someone changed a security group in the console. What happens at the next apply, and how do you stop it happening?
Terraform detects the drift and reverts the group to the configuration (or you update the code if the change should stay). Prevent it with restricted console permissions, change processes through pull requests, and scheduled drift-detection plans.
How would you reduce the blast radius of Terraform changes?
Split into several states by layer and environment, restrict who can apply to production, require plan review, use prevent_destroy on critical resources and short-lived credentials with least privilege.
Summary
- State maps configuration to real objects and stores their attributes; plans compare configuration, refreshed state and reality.
- Drift appears in the next plan; fix by applying, updating code,
ignore_changes, or prevention and scheduled plans. - Use
state list/showto inspect; prefermovedandimportblocks to manual surgery; back up first; never hand-edit. - State contains secrets: keep it out of Git, encrypted and access-controlled, and split it to limit blast radius.
Going deeper
- State: purpose: why Terraform needs state.
- Manage resource drift: detecting and handling drift.
- Sensitive data in state: what is stored and how to protect it.
- terraform state command: list, show, mv, rm, pull and push.
See how much of this you can explain
A free mock interview asks you new questions on Infrastructure as Code with Terraform for DevOps Engineer and scores every answer, with what to fix.