Service 04 / 04

Failover & high availability

Hardware fails, networks have outages, software has bugs – the question is not if, but when. A high-availability cluster is built so your application never even notices.

Approach

Planning for failure instead of hoping against it.

High availability does not come from better hardware – it comes from architecture: every component whose failure would stop your application exists more than once – nodes, load balancers, network paths, storage. If one part fails, its counterpart takes over automatically, within seconds and without manual intervention.

For the automation to decide correctly in an emergency, it needs clear rules: quorum ensures that only a majority of nodes makes decisions, and fencing cleanly isolates a faulty system before it can cause damage. This is how we prevent split-brain situations, where two halves of a cluster work against each other.

And because a failover that has never happened is not one: we test the failure scenarios regularly – controlled, documented, and without risk to operations.

Scope

What high availability includes with us.

  • Redundant nodes in an N+1 design: a node failure is business as usual, not an emergency.
  • Automatic failover without manual intervention, typically within seconds.
  • Quorum & fencing against split-brain: the cluster makes unambiguous decisions even under failure.
  • Redundant load balancers with a floating IP: the distributor itself is not a single point of failure either.
  • Replicated storage: data exists in multiple copies – a dead disk costs no data.
  • Redundant network paths and connectivity, against faults on individual routes.
  • Regular failover tests with a written record: we pull the plug before chance does.
  • Documented runbooks for the scenarios that go beyond an automatic failover.

Reference

The failover, as it actually happens.

Diagram 05: Automatic failover in time lapse. Node B fails, the load balancer takes it out of rotation, A and C take over – and B rejoins after repair.

Perspective

What availability means in numbers.

Availability is often measured in nines: 99 % sounds good, but it means more than three full days of downtime per year. 99.9 % is still almost nine hours; only 99.99 % pushes it below one hour. Which level your application needs is a business question, not a technical one: what does one hour of standstill cost you?

That is exactly why we define the availability target together before the architecture is designed – and then build as much redundancy as that target demands. Committed targets go into the operations contract, not just onto the website.

FAQ

Common questions about high availability.

Will my users notice a failover?

Normally not. The load balancer takes the failed node out of rotation within seconds; existing connections are re-established. How seamless it is in each case depends on the application – we discuss that honestly in the concept.

Does high availability replace backups?

No – and anyone claiming otherwise is planning negligently. Replication protects against hardware failure, but it also replicates mistakes: an accidental deletion lands on all copies immediately. That is why separate, tested backups are always part of the package.

How often do you test the failover?

Fully before going live, and regularly during operation after that – controlled and with a written record. We agree on the rhythm with you.

What about the failure of an entire data center?

The standard is redundancy within one location. If your availability target demands it, we design across sites – with replication to a second German data center. That is a matter of concept and budget, and exactly what the initial conversation is for.

Contact

What does one hour of downtime cost you?

Let’s talk about your availability target. Initial assessment within one business day, free of charge and without obligation.

Request a project