Infrastructure, cloud & DevOps · Advanced

High availability and disaster recovery

In one line: Decide how much an hour of downtime costs you, and only then decide how much redundancy to buy.

Define a target before a solution

The gap between 99.9 and 99.99 percent availability is a gap of several times in cost and complexity. Most business systems don't need the higher one, and whoever does can justify it in money.

Layers of resilience

Multiple instances across availability zones; a database with a secondary replica and automatic failover; no dependence on local state; and health checks that pull a faulty instance out of traffic.

For the next step — a second geographic region — the cost and complexity are significant, including questions of latency and data consistency.

The disaster scenario

Not just a fallen server: accidental deletion, data corruption, and a locked cloud account. Each requires a different answer, and all three should have a written plan someone read in the past year.

Going deeper

Run controlled failure drills: bring down an instance, cut a dependency, fill a disk — in a controlled environment at a known time. A system never tested under failure isn't resilient, it's just untested.