One Zone Down: What a Single Lost Zone in London Says About Cloud Redundancy

Draft. All times are Paris time (CEST, UTC+2), 20 October 2026. On Tuesday, Google Cloud lost one availability zone in London. One zone, europe-west2-b, out of three. Not a region. Not a continent. Banks, a budgeting app, a parcel carrier, the postal service and the courts' video hearings all reported problems the same day. Power events happen. The interesting part is what a single lost zone revealed about the systems running on top of it.

What Google actually lost

Google's incident report is precise. The incident began at 08:08 and ended at 16:00: 7 hours 52 minutes. At 13:12, Google named the cause: "Due to a power event we had suffered an outage within europe-west2-b zone." Two different symptoms: "Google Cloud services are unavailable in europe-west2-b zone" "some services in europe-west2 region may experience elevated errors and intermittent network issues" One zone fully unavailable. The rest of the region degraded. 38 products listed, including Compute Engine, GKE, Cloud SQL, Spanner, Persistent Disk, Cloud Load Balancing and VPC. No other region appears in the incident (Google Cloud). That evening, Google said the disruption happened "at a third-party facility in our europe-west2 cloud region due to a power event" (The Guardian).

Zone vs region

A zone is a failure domain inside a region. A region is a separate geographic area made of zones. Google's documentation says it plainly: "A failure in one zone usually does not affect the resources in the other zones in the region", and regional resources "are resilient to incidents in a specific zone" (Google Cloud). Google also publishes the availability it designs for: 99.9% for a workload in a single zone, 99.99% for multi-zone, 99.999% for multi-region (Google Cloud). On 20 October, the failure was zonal, with regional side effects. That is the scenario multi-zone designs exist for.

Who was hit, and what broke

Listed: organisations whose own statements were published or reported. None of their architectures are public, and I found no post-incident report from any of them. Revolut. Its French status page listed 31 services. At 13:10, all were still shown as down, including sign-in, card payments, ATM withdrawals and transfers (Moneyvox). Its Singapore status page showed all 27 services partially unavailable that evening, local time (CNA). At 19:00, only 6 of 27 UK services were working (Reuters via Euronext). At 19:39, Revolut said services were restored and attributed the disruption to "a recent Google Cloud outage" (X).

What Revolut customers were saying that morning: the app was out of service. Lloyds Banking Group (Lloyds, Halifax, Bank of Scotland). Lloyds said: "Our app is online and customers can log in normally, though we understand some are having issues with certain online services this morning." (The Guardian live). In the afternoon: "We understand the root issues have been resolved, so we expect all our services to be back to normal soon" (CNA/Reuters). Halifax told a user it was "aware some customers are having issues with our app and online" (The Independent). Bank of Scotland customers also reported issues (BBC). HSBC. HSBC said customers experienced "difficulties accessing our services" and that it would "monitor the situation, which is also affecting other organisations" (CNA/Reuters). Emma. The budgeting app said it was down "because our cloud provider, Google Cloud, is having a major outage", since around 08:00 Paris time (Emma). It was "up and running again" by about 15:00, warning that "some balances may take a little while to refresh" (BBC live; The Guardian live). DPD. Its Irish account said: "The connection issues affecting our website have now been resolved" (Sky News). Royal Mail. Said it was aware of the issues and that there was "no effect on letter and parcel deliveries" (RTÉ). HM Courts & Tribunals Service. The Cloud Video Platform for remote hearings in England and Wales "was not functioning throughout the morning", with users seeing an internal server error (The Guardian live). HMCTS said it was "working with our technology partners to restore service" (The Guardian). Court live streams were back in the afternoon (Sky News). Forest. The e-bike hire company said it was back up and running in the afternoon (BBC live).

The pattern matters more than any single name. Google closed its incident at 16:00. Some services were back before that. At least one, Revolut, was still largely down three hours later. Same zone, same cause, very different recovery curves.

Same region, kept running: ClickHouse

ClickHouse Cloud runs in the same region. Its own status page: 11:52: "Currently only zone B is impacted. We are migrating workloads away from this zone." 12:04: "Clusters in this region should be operational, accepting reads and writes, but provisioning of new services currently blocked until this zone is operational." 16:23: resolved. Only ClickPipes was listed as an affected component. It was not a clean pass. Creating new services was blocked for hours. But existing clusters kept serving reads and writes. ClickHouse documents that production services are spread across three availability zones "so that the cluster can recover from zone failures" (ClickHouse). That behaviour is exactly what Google's regional guidance is designed for: "If a zone outage occurs, the application stack continues to run in the other zones" (Google Cloud).

Failover that needs new capacity is not failover

Google's updates spell it out. From 11:24, the suggested workaround: "If feasible, the impacted customer may try to migrate their workload impacted to another available location (zone or region)." From 13:58, a warning: "Customers may not be able to yet create new workloads." (Google Cloud) Read those together. A recovery plan that says "spin up replacement instances in another zone" depends on provisioning at the exact moment the provider is degraded, while every other customer tries the same. ClickHouse hit that wall for new services. It just did not need new services to keep existing ones running. Two families of design: Active-active across zones. Every zone already serves traffic, with headroom (N+1) to absorb the loss of one. Data is replicated across zones. The load balancer drains the dead zone. Nothing has to be created during the incident. Failover on demand. The primary lives in one zone. The standby is cold, or does not exist yet. Recovery means provisioning, restoring snapshots, warming caches, re-pointing traffic. Cheaper every day. Fragile on the one day it matters.

Two numbers frame the choice. RTO (recovery time objective): how long a service may be down. RPO (recovery point objective): how much data you can afford to lose. Active-active aims for an RTO near zero and an RPO of zero, and pays for it in permanent capacity. On-demand failover accepts an RTO measured in rebuild time. Recovery does not end when infrastructure is back. Queues drain, payments replay, ledgers reconcile. Emma's warning about balances is that tail.

What it looks like in practice

Generic patterns on Google Cloud, not a description of any affected company. The goal: lose europe-west2-b and keep serving, without creating anything during the incident.

1. Compute across three zones

A regional GKE cluster replicates the control plane across zones and spreads nodes over the zones you list (GKE docs). In a regional node pool, node_count is per zone (Terraform google provider).

resource "google_container_cluster" "main" {
  name                     = "prod"
  location                 = "europe-west2"   # region = regional cluster
  node_locations           = ["europe-west2-a", "europe-west2-b", "europe-west2-c"]
  remove_default_node_pool = true
  initial_node_count       = 1
}

resource "google_container_node_pool" "app" {
  name       = "app"
  cluster    = google_container_cluster.main.id
  location   = "europe-west2"
  node_count = 3                              # per zone: 9 nodes, sized for N+1
  node_config {
    machine_type = "e2-standard-4"
  }
}

Sizing is the real decision. GKE's own guidance: overprovision a three-zone cluster to 150% so that losing a third of capacity still serves 100% of traffic (GKE docs). Their example gets there with autoscaler limits, which means creating nodes during the outage. On 20 October, Google warned that new workloads might not be creatable (Google Cloud). For critical paths, keep that 150% running, not just allowed. Without Kubernetes, the equivalent is a regional managed instance group with distribution_policy_zones and an autohealing health check (Terraform google provider).

2. Spread pods, probe them, protect them

topologySpreadConstraints on topology.kubernetes.io/zone keeps replicas balanced across zones (Kubernetes docs). Readiness probes take an unhealthy pod out of traffic (Kubernetes docs). A PodDisruptionBudget limits voluntary evictions, for example during node drains (Kubernetes docs).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
spec:
  replicas: 6 # 2 per zone; 4 must carry full load
  selector:
    matchLabels: { app: payments-api }
  template:
    metadata:
      labels: { app: payments-api }
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway # never leave replacements Pending
          labelSelector:
            matchLabels: { app: payments-api }
      containers:
        - name: api
          image: europe-docker.pkg.dev/my-project/app/payments-api:1.0.0
          ports: [{ containerPort: 8080 }]
          readinessProbe:
            httpGet: { path: /healthz/ready, port: 8080 }
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz/live, port: 8080 }
            periodSeconds: 10
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: payments-api
spec:
  minAvailable: 4
  selector:
    matchLabels: { app: payments-api }

One honest catch: with DoNotSchedule, replacement pods for a dead zone can stay Pending, because the constraint still counts that zone while its nodes exist. ScheduleAnyway keeps the spread as a preference, so replacements land in the surviving zones. Until they are up, the surviving four replicas carry the load. Which is the point of N+1.

3. Data that survives a zone

Cloud SQL with availability_type = "REGIONAL" keeps a standby in a second zone, with writes synchronously replicated to both zones' disks before commit (Cloud SQL docs). The provider requires backups, and point-in-time recovery for Postgres (Terraform google provider).

resource "google_sql_database_instance" "ledger" {
  name             = "ledger"
  region           = "europe-west2"
  database_version = "POSTGRES_16"
  settings {
    edition           = "ENTERPRISE"                  # Postgres 16+ defaults to Enterprise Plus
    tier              = "db-custom-4-16384"
    availability_type = "REGIONAL"          # primary + standby in another zone
    backup_configuration {
      enabled                        = true
      point_in_time_recovery_enabled = true
    }
  }
}

Trade-offs, from Google's docs: an HA instance "costs twice as much as a standalone instance", and during failover "you can expect the instance to be unavailable for about sixty seconds" (Cloud SQL docs). RPO zero, RTO about a minute. Plan connection retries. Spanner goes further. A regional configuration keeps three read-write replicas in three zones, so a single-zone failure leaves the database available. Multi-region configs such as eur5 (London and Belgium, witness in the Netherlands) survive a regional failure, at the cost of slightly higher write latency and price (Spanner docs).

4. The multi-region step

A global external Application Load Balancer can front backends in several regions and route around unhealthy ones using health checks (Cloud Load Balancing docs).

resource "google_compute_backend_service" "api" {
  name                  = "api"
  load_balancing_scheme = "EXTERNAL_MANAGED"
  protocol              = "HTTP"
  port_name             = "http"                        # named_port "http" on each instance group
  health_checks         = [google_compute_health_check.api.id]   # defined elsewhere
  backend { group = google_compute_region_instance_group_manager.london.instance_group }    # europe-west2
  backend { group = google_compute_region_instance_group_manager.belgium.instance_group }   # europe-west1
}

Add a URL map, target proxy and global forwarding rule (Terraform google provider). The hard part is not the load balancer. It is the data layer behind the second region.

5. Failover checklist

Health checks drive traffic, not humans. Capacity for a zone loss is already running. Data replication matches the RPO you claim. RTO and RPO are written down per service. Game days: drain a zone in production on purpose, regularly, and measure what users see.

Open questions, and a position

No affected organisation has published its architecture. So, for each of them, these stay questions: Were critical paths (authentication, payments, ledgers) running in all three London zones, with capacity already live in each? Did recovery depend on creating resources while Google warned new workloads might not be creatable? Is there a global dependency on europe-west2? Revolut customers in Singapore were affected too. Is there warm capacity in another region, or with another provider, for the services that let people pay? What happened between Google's recovery and each service's own recovery? Now a position. If a single-zone loss can take a large regulated bank down for most of a day, that is indefensible at that scale. Zone loss is not exotic. It is the first failure mode in every cloud reliability guide, including Google's own.

It is a solved problem

Monzo, a UK bank. Monzo runs its primary platform on AWS and built Monzo Stand-in, "an independent set of systems that run on Google Cloud Platform (GCP)" able to take over "in the event of a major incident", supporting card spending, cash withdrawals, bank transfers and balances (Monzo). AWS reports that Stand-in runs 18 services against more than 3,000 on the primary stack, at about 1% of its running cost (AWS). Netflix. Chaos Kong "kills an entire AWS Region" in regular exercises. During a real US-EAST-1 incident in 2015, Netflix "sidestepped any significant impact because Chaos Kong exercises prepare us for incidents like this" (Netflix). Later, Project Nimble cut region failover from about 50 minutes to under 10. The bottleneck it removed was provisioning: clusters were "not overprovisioned" to absorb another region's traffic, so Netflix moved to capacity that was already running (Netflix). Same lesson as Tuesday.

Concentration risk

Many firms, one provider, one location. Each design can look reasonable in isolation while the system fails together. As Keele University's Amro Al-Said Ahmad told The Register: "resilience is not achieved by simply moving to the cloud."

What is at stake for regulated banks

When a bank's platform goes dark, it is not an app that stops. It is access to deposits, salaries, bills and card payments. Commentators noted that people paid weekly or fortnightly "may have found that they are unable to access their funds on the very day where they have payments or commitments to make" (The Guardian live). For businesses and freelancers who take card payments through their bank, a down platform also means no sales: Revolut, for example, offers payment acceptance to freelancers through Revolut Pro and to companies through Revolut Business. The rules already cover this: UK operational resilience. Since 31 March 2025, firms in scope must stay within their impact tolerance for each important business service "in the event of a severe but plausible disruption" (FCA Handbook, SYSC TP 10; FCA PS21/3). Losing one cloud zone is a textbook severe but plausible scenario. Critical Third Parties. Since 13 July 2026, Google Cloud EMEA Limited is a designated Critical Third Party (legislation.gov.uk). The Treasury: "Financial firms remain responsible for managing risks arising from their third-party suppliers" (GOV.UK). The Bank of England: the regime does not replace firms' own "due diligence, risk management and contingency planning" (Bank of England). After the outage. The FCA said it was "engaging with them and affected firms to understand the impact" (RTÉ). The PRA was understood to be speaking with Google (The Guardian live). The Telegraph reported that the Bank of England is investigating (The Telegraph). DORA in the EU. The Digital Operational Resilience Act has applied since 17 January 2025 (ESMA). Google Cloud is one of 19 critical ICT third-party providers designated by EU supervisors (ESMA). Affected firms' EU entities, such as Revolut Bank UAB which serves customers in France (Revolut), fall under it. A lawyer put it in one line that day: "Outsourcing a service does not mean outsourcing responsibility for it" (The Guardian live). The rules ask firms to stay within tolerance through a severe but plausible disruption. The provider is under direct oversight. Regulators were engaging on day one. It is reasonable to expect hard questions for every affected bank about how a single-zone event turned into hours of disruption. What follows is for regulators to decide.

Takeaways for builders

Treat zone loss as routine. Test it with game days. Never build recovery on provisioning. The capacity you need in an incident must exist before it. Hunt global dependencies on one region: auth, config, ledgers. Set RTO and RPO per service. Then prove the architecture meets them. Measure recovery from the user's side. The provider closing its incident is not the end of yours. One zone went dark. Some systems moved their workloads. Others went down with it. That difference is decided long before the outage, when someone chooses between paying for redundancy and hoping not to need it. Code verified on October 21, 2026 against official Google Cloud documentation.