Lesson 6 of 11 · 11 min · Designing on AWS
High availability across zones and auto scaling
What you will learn
- Availability numbers and what they mean
- Designing without single points of failure across availability zones
- Auto Scaling groups: launch templates, health checks, scaling policies
- Stateless design that makes scaling and replacement possible
- RTO, RPO and the four disaster-recovery strategies
In simple words
Machines fail, and sometimes a whole data centre does. High availability means your service keeps working anyway, because it runs in more than one place. Auto scaling adds machines when traffic rises and removes them when it falls, like a shop opening more billing counters when the queue grows.
Words to know
- High availability: staying in service even when one part fails.
- Single point of failure: one part whose failure takes everything down.
- Auto scaling: adding or removing machines automatically as the load changes.
- Stateless: a server that keeps nothing important on itself, so any copy can replace it.
Availability in numbers
Availability is the share of time a service works. Targets are written in "nines":
| Availability | Downtime per year | Per month |
|---|---|---|
| 99% | ~3.65 days | ~7.3 hours |
| 99.9% | ~8.8 hours | ~43 minutes |
| 99.95% | ~4.4 hours | ~22 minutes |
| 99.99% | ~53 minutes | ~4.4 minutes |
Each extra nine costs disproportionately more. Choose the target from business need (an internal report tool does not need four nines) and write it as an SLO (monitoring course). Also note that availability multiplies across components in a chain: a service needing three components of 99.9% each in series has about 99.7% overall (0.999³).
Eliminate single points of failure
A single point of failure (SPOF) is any component whose failure takes the system down. Hunt for them layer by layer:
| Layer | SPOF | Remedy |
|---|---|---|
| Data centre | everything in one AZ | spread across at least two (preferably three) AZs |
| Servers | one instance | several instances in an Auto Scaling group behind a load balancer |
| Load balancer | one self-managed LB | managed load balancer (ALB/NLB spans AZs by design) |
| Database | single instance | Multi-AZ database / Aurora replicas |
| Storage | single-AZ disk | S3 (regional, 11 nines), EFS (regional), EBS with snapshots |
| Network | one NAT gateway / one VPN | one NAT gateway per AZ; redundant VPN or Direct Connect |
| DNS | single provider | Route 53 (highly available, with health checks) |
| People and process | one person knows how | runbooks, on-call rota, automation |
| Region | everything in one region | multi-region only if the SLO requires it (cost and complexity jump) |
The standard shape of a resilient web application:
Route 53 (DNS)
│
ALB across AZ a and b (public subnets)
┌───────┴────────┐
AZ a: app servers AZ b: app servers (private subnets, Auto Scaling group)
└───────┬────────┘
Multi-AZ database: primary in AZ a, standby in AZ b (data subnets)
Design rule: lose any one AZ and the remaining capacity must still carry the load. With two AZs each must be able to handle 100% of traffic (so run at no more than 50% per AZ in steady state, or accept degradation); with three AZs each can run at ~66%, which is cheaper than two-AZ headroom. Test it by really failing instances and zones (game days, chaos engineering, AWS Fault Injection Service).
Health checks tie it together
Load balancers send requests only to targets that pass health checks (an HTTP path such as /healthz, expected status, interval, thresholds). Auto Scaling replaces instances that fail their checks. A health check must reflect real ability to serve (CI/CD and Kubernetes courses discuss shallow versus deep checks): the ALB health check should verify the app, but not cascade a shared dependency failure into removing every instance.
Cross-zone load balancing and data
ALBs distribute traffic across zones evenly (cross-zone is on by default; NLB makes it optional and charges for cross-zone data). Remember cross-AZ data transfer costs money and adds a little latency, so avoid unnecessary chatter between zones.
Auto Scaling
An Auto Scaling group (ASG) keeps a desired number of EC2 instances running, replaces unhealthy ones, spreads them across AZs, and scales the count with demand.
Ingredients:
- Launch template: the instance blueprint: AMI, type, security groups, IAM instance profile, user data, EBS settings. Version it.
- Group settings:
min,maxanddesiredcapacity; the subnets (one per AZ); health check type (use ELB health checks so a failing app, not only a failing VM, triggers replacement) and a grace period for start-up; registration with a target group. - Scaling policies:
- Target tracking (simplest, recommended): keep a metric near a target, for example average CPU at 50% or ALB requests per target at 1,000. The group adds and removes instances automatically.
- Step scaling: add N instances when an alarm crosses thresholds.
- Scheduled scaling: known patterns (scale up before 9 a.m., before a sale).
- Predictive scaling: forecasts recurring traffic.
- Instance refresh: a rolling replacement of instances when you update the launch template (new AMI), with minimum healthy percentage; the equivalent of a rolling deployment.
- Mixed instances and Spot: combine On-Demand base capacity with Spot instances across several types for savings.
- Lifecycle hooks and warm pools: run scripts at launch/terminate; keep pre-initialised instances to cut start-up time.
Scaling behaviours to understand: cooldown/warm-up periods stop thrashing; scale-in terminates instances (use connection draining, ASG termination policies, and scale-in protection for special nodes); scaling takes minutes (boot, configuration, health checks), so keep headroom or scale on leading indicators (queue length, request rate) rather than waiting for CPU to saturate. For containers: ECS service auto scaling and Kubernetes HPA plus node autoscaling follow the same ideas (Kubernetes course).
What makes scaling possible: stateless servers
Instances must be replaceable: no user sessions or uploads on local disk. Put:
- sessions in a shared store (ElastiCache/Redis, DynamoDB) or use signed tokens,
- files in S3,
- configuration in Parameter Store/Secrets Manager,
- everything else in the database.
Build instances from an image (AMI baked with Packer, or containers) so a new instance is ready in minutes, not hand-configured. If an instance takes 20 minutes to become useful, auto scaling cannot rescue a traffic spike.
Scaling the rest of the stack
Auto scaling of web servers moves the bottleneck: the database, a downstream API, or the NAT gateway may become the limit. Protect them with caching (ElastiCache, CloudFront), read replicas, connection pooling (RDS Proxy), queues (SQS) to absorb bursts and decouple work, and rate limits/back-pressure. Serverless and managed scaling services (DynamoDB on-demand, Aurora Serverless, Lambda, Fargate) shift this effort to AWS.
Disaster recovery
Multi-AZ handles a data-centre failure. Disaster recovery (DR) handles larger events: region outage, data corruption, ransomware, human error. Two numbers drive the design:
- RPO (recovery point objective): how much data you can afford to lose (time since last good copy).
- RTO (recovery time objective): how long you can take to be back.
Four standard strategies, from cheap and slow to expensive and fast:
| Strategy | What runs in the DR region | RPO / RTO | Cost |
|---|---|---|---|
| Backup and restore | nothing; backups copied there | hours / hours-days | lowest |
| Pilot light | data replicated; core infrastructure defined as code but off | minutes / tens of minutes | low |
| Warm standby | scaled-down but running copy | seconds-minutes / minutes | medium |
| Active-active (multi-site) | full capacity in both regions, traffic served by both | near zero / near zero | highest |
Whatever you choose: infrastructure as code (the Terraform course) lets you rebuild an environment quickly, backups must be tested restores, copy backups to another account and region (protect against account compromise and ransomware: immutable backups with Vault Lock/Object Lock), keep a runbook, and rehearse DR at least yearly. Route 53 health checks with failover or weighted routing steer users to the surviving side; for example, two A records for app.example.test with weights 90 and 10 and a TTL of 60 seconds (a low TTL speeds failover, per the DNS course).
Common mistakes
- One AZ (or "two instances" both in the same AZ).
- Capacity that cannot survive losing an AZ.
- Scaling on a metric that doesn't reflect load, or too slowly.
- Stateful web servers that cannot be replaced.
- Health checks too shallow or too deep.
- One NAT gateway for all AZs (its AZ failing cuts internet access for all private subnets).
- DR plans never tested; backups in the same account and region only.
- Chasing five nines without a business reason.
Try it yourself
- Draw your own application's components and mark every SPOF; for each, write the remedy and its cost.
- Create a launch template and an Auto Scaling group across two subnets in a sandbox account, with target tracking on CPU; generate load and watch the instance count.
- Terminate an instance by hand and watch the ASG replace it.
- Decide RPO and RTO for a given system and choose a DR strategy; write a one-page runbook.
Check yourself
Why spread an application across multiple availability zones?
A single AZ can fail (power, network, cooling). With instances and a standby database in different AZs, the remaining zone continues to serve, if it has enough capacity.
What does target tracking scaling do?
Automatically adds or removes capacity to keep a chosen metric (for example CPU or requests per target) near a target value.
What is the difference between RPO and RTO?
RPO is the maximum acceptable data loss measured in time; RTO is the maximum acceptable time to restore service.
Why must web servers be stateless to auto scale?
Instances are added and removed at will; any state held locally would be lost or unavailable. Sessions, files and data belong in shared services.
Interview questions
Design a highly available web application on AWS.
Route 53 → ALB across at least two AZs → Auto Scaling group of stateless instances (or ECS/EKS) in private subnets spread across AZs → Multi-AZ RDS/Aurora in data subnets, S3 and a cache for shared state, a NAT gateway per AZ, health checks, monitoring and alarms, backups tested, infrastructure in code, and capacity to survive the loss of one AZ.
When would you choose pilot light over warm standby?
When a longer RTO (tens of minutes) is acceptable and you want to minimise cost: only data replication runs continuously, and the application tier is started on demand from code and images.
How do you handle a sudden traffic spike?
Autoscaling on leading metrics with headroom, CDN and cache to absorb reads, queues for writes, pre-scaled capacity or scheduled scaling for known events, database read replicas/connection pooling, and load testing beforehand.
Summary
- Availability is multiplied across components; remove SPOFs by using multiple AZs at every tier and sizing to survive losing one.
- Auto Scaling groups use launch templates, health checks and policies (target tracking first) to keep and resize a fleet; servers must be stateless.
- Protect downstream systems with caches, replicas, pooling and queues.
- DR is driven by RPO and RTO: backup/restore, pilot light, warm standby or active-active; test it.
Going deeper
- Amazon EC2 Auto Scaling user guide: groups, templates and policies.
- Target tracking scaling policies: how they work.
- AWS Well-Architected: reliability pillar: resilience practices.
- Disaster recovery of workloads on AWS (whitepaper): the four DR strategies.
- Elastic Load Balancing: health checks: configuring checks.
See how much of this you can explain
A free mock interview asks you new questions on Cloud platforms and AWS fundamentals for DevOps Engineer and scores every answer, with what to fix.