Ask most technical teams whether they have backups, and the answer is almost always yes. Ask whether they've actually restored from those backups recently, under realistic conditions, and the answer changes. That gap — between having a backup and having a tested, working recovery process — is the single most consistent finding across recent disaster recovery research, and it's the part a diagnostics checklist tends to miss because "do you have backups" is the wrong question.
The two numbers every recovery plan actually needs
Before the testing gap makes sense, it helps to separate two metrics that get used almost interchangeably and shouldn't be: Recovery Time Objective (RTO) and Recovery Point Objective (RPO). RTO answers "how long can we be down?" — it's the maximum acceptable time between an outage starting and normal operations resuming. RPO answers a different question: "how much data can we afford to lose?" — measured as the gap between the moment of failure and your last usable backup. A system backed up every hour has a one-hour RPO no matter how fast or slow the actual restoration process is; RTO and RPO are independent dials, not two names for the same thing.
The distinction matters operationally. A four-hour RTO with a fifteen-minute RPO means your infrastructure has to restore service within four hours, but it can only afford to lose fifteen minutes of data getting there — which usually demands frequent, automated backups or replication rather than a single nightly job. Not every system needs the same targets: a customer-facing checkout flow might warrant a near-zero RPO, while an internal wiki can often tolerate a full day of data loss without real business harm. Tiering systems by criticality, and setting RTO and RPO independently for each tier, is what turns "we have backups" into an actual, costed recovery plan.
The numbers behind the gap
According to Cockroach Labs' State of Resilience 2025 survey, all organizations conduct some form of resiliency testing, but the specific tests that matter most are the ones most often skipped: 62% of organizations fail to do regular system backups and restoration exercises, and 71% do no failover testing at all to confirm their outage-prevention protocols actually work. Testing exists as a category; it just doesn't reliably include the test that proves recovery is possible.
The same survey found that organizations experienced an average of 86 outages over a 12-month period, with the average outage lasting 196 minutes — well over three hours — and only 2% of companies able to resolve an unplanned outage in 60 seconds or less.
What an outage actually costs
Cost figures vary by methodology, but the well-sourced ones converge on a consistent order of magnitude. PagerDuty's 2024 Customer Incidents Survey put the true cost of downtime at a median of $4,537 per minute, with the average organization's last major customer-facing outage costing roughly $1.5 million in lost revenue. New Relic's 2025 Observability Report found high-impact outages carrying a median cost of $2 million per hour. Both figures point the same direction: at any meaningful scale, the cost of an hour of downtime is not a rounding error.
The same PagerDuty research found something more actionable than the raw cost figures: organizations with at least five fully automated incident response processes resolved customer-impacting incidents 78 minutes faster on average than organizations relying on largely manual processes, and experienced 45% lower annual costs from customer-facing outages — $16.8 million annually compared to $30.4 million. Automation isn't just an efficiency nicety here; it's directly tied to how much an outage ends up costing.
Why the testing gap persists
It's not that teams don't know testing matters — 93% of leaders in the Cockroach Labs survey said they're concerned about the financial and organizational impact of outages, and 95% said they're aware of specific vulnerabilities in their own setup. The gap is between awareness and action: only 20% describe their organization as fully prepared for outages, and only 33% have what they'd call an organized response approach. Most teams know exactly where the weak point is and haven't gotten around to testing it.
Part of this is structural. 41% of organizations still find out about service interruptions through manual checks, customer complaints, or incident tickets — after the fact, rather than through monitoring that catches the problem first. If you don't know about a problem until a customer tells you, testing the recovery process tends to slide down the priority list indefinitely.
What "tested" actually means in practice
A backup that has never been restored is, functionally, an assumption rather than a safeguard. A meaningful test means periodically restoring from backup into an isolated environment and confirming the data is complete, uncorrupted, and usable — not just confirming that a backup job completed without an error message. The distinction matters: a backup process can run successfully and silently produce an unusable file, and the only way to find that out before an actual emergency is to try the restore on a schedule, not just hope the green checkmark means what it implies.
The same logic applies to failover testing. A redundant setup that has never actually been failed over to is a redundant setup with an unverified assumption sitting in the middle of it. The 71% of organizations skipping this step aren't necessarily wrong about having redundancy — they're wrong about knowing whether it works.
The practical takeaway
If a system's disaster recovery plan exists only as documentation, it isn't a recovery plan yet — it's a draft of one. The fix isn't more backup software; most organizations already have that. It's a recurring calendar item: restore a backup into an isolated environment on a set schedule, time how long it takes, and treat any failure to do so smoothly as the finding it is, not as an inconvenience to route around. The 20% of organizations that describe themselves as fully prepared didn't get there by buying something different — they got there by testing.
Frequently asked questions
RTO (Recovery Time Objective) measures how long a system can be down before the impact becomes unacceptable — it answers "how fast can we recover?" RPO (Recovery Point Objective) measures how much data loss is tolerable, measured backward in time from the failure to your last good backup — it answers "how much data can we afford to lose?" A system can meet its RTO by restoring quickly while still failing its RPO if the data restored is hours or days old.
Guidance varies by organization size and complexity, but a common baseline is at least once a year for smaller organizations and quarterly for larger ones with more complex infrastructure, with additional tests after any major system or infrastructure change. The more important point than frequency is realism: a useful test actually restores from backup into an isolated environment and times the result, rather than just confirming a backup job completed without errors.
No. A backup is a copy of your data; a disaster recovery plan is the broader, tested process for actually restoring systems, applications, and operations within an acceptable time frame after a disruption. A pile of successful backup jobs can still fail a business if nobody has verified that the data restores cleanly, that the recovery steps are documented, or that the restored systems come back online in the right order.
At minimum, it means periodically restoring from backup into an isolated environment and confirming the data is complete and usable, not just confirming the backup job finished. A more thorough test, sometimes called a failover test, also verifies that redundant or standby infrastructure actually takes over successfully when the primary system is deliberately taken offline, rather than assuming the redundancy works because it's never been tried.
Industry research converges on a meaningful order of magnitude rather than one precise number: PagerDuty's 2024 Customer Incidents Survey found a median cost of roughly $4,537 per minute of downtime, and New Relic's 2025 Observability Report found high-impact outages carrying a median cost of about $2 million per hour. The exact figure depends heavily on company size and how customer-facing the affected system is, but at any meaningful scale, an hour of downtime is a significant and measurable cost, not a rounding error.
