Server Down or a Routing Issue? Building Multi-Region Monitoring with Zabbix
Industry: Blockchain infrastructure.
Workload: Distributed blockchain nodes and backend services.
Regions: Europe and Asia.
Services: Dedicated servers, infrastructure sourcing, network troubleshooting, custom monitoring.
About the project
A blockchain infrastructure provider approached Unihost while looking for an alternative to its existing hosting environment. The customer’s infrastructure consisted of a growing number of dedicated servers used for distributed services across multiple regions. Initially, the primary requirement was straightforward: find cost-efficient dedicated hardware in Europe that could replace the customer’s previous servers without significantly increasing infrastructure costs. As the project evolved, however, the customer’s requirements became more complex. New infrastructure was added in Asia, and network reliability became one of the main operational challenges. This eventually led the Unihost support team to build a custom multi-region monitoring system designed specifically to distinguish server downtime from route-specific connectivity problems.
Initial Infrastructure
The first stage of the project focused on Germany. Unihost sourced dedicated servers based on Intel Xeon E-2388G. The configuration provided the customer with a relatively inexpensive replacement for the infrastructure it had previously operated with another European hosting provider. The initial deployment was primarily about capacity: the customer needed additional dedicated compute at an aggressive price point.
Expanding the Infrastructure to Asia
The next stage was significantly more demanding. The customer started building a new cluster in Asia and required different hardware profiles for different workloads. Unihost sourced additional infrastructure including: Malaysia – AMD Ryzen 9 7950X3D dedicated servers, Singapore – AMD EPYC 7313 dedicated servers.
The result was a geographically distributed infrastructure spanning Europe and multiple Asian locations. At this point, server specifications were no longer the main challenge. Network behavior was.
Technical Challenge
The customer periodically reported that some servers were unavailable. They were unable to reliably pinpoint the location of the problem; subsequently, a solution was jointly developed that made it possible to more accurately identify and distinguish between issues originating with backbone providers and those on the data center or server side. However, determining the actual source of an incident was difficult. A server could appear unreachable from the customer’s network while remaining fully accessible from another region.
The possible causes included:
- an actual server outage;
- an issue inside the data center network;
- a routing problem between networks;
- packet loss affecting a specific path;
- a connectivity issue outside the hosting provider’s infrastructure.
A conventional monitoring system located in a single data center could not reliably distinguish between these situations. For example, if a monitoring server in Europe lost connectivity to a server in Asia, it could generate an outage alert even though the target server itself was still online and accessible from other networks.
This created two problems:
- For the customer, it generated uncertainty about whether the infrastructure was actually down.
- For the Unihost support team, it made it difficult to establish objectively where the problem had occurred.
What the Unihost Team Did
Instead of relying on monitoring from a single location, the team built a geographically distributed monitoring system.
- Deployed Two Independent Zabbix Monitoring Points
Two Zabbix servers were deployed in separate regions:
- Monitoring node 1 in Europe
- Monitoring node 2 in Asia
Both monitoring systems independently checked the same customer servers. This provided two geographically separate perspectives on the availability of each host.
- Added Custom Trigger Processing
Standard Zabbix alerts alone were not sufficient. A custom script was developed to intercept the webhook generated when a Zabbix host trigger entered an alert state. Instead of immediately forwarding every individual alert to the customer, the script processed the event first.
- Stored Alert States in Redis
When one of the monitoring nodes detected that a host had become unavailable, the alert state was written to a Redis database. The system then checked whether Redis contained a corresponding alert for the same host from the second monitoring location. This introduced an additional correlation layer between the monitoring systems.
- Correlated Alerts from Europe and Asia
The logic was deliberately simple. If both independent monitoring locations reported the same host as unavailable, the system treated the event as a likely server or infrastructure outage. The customer was then notified. If only one monitoring location reported the failure, no customer downtime notification was generated. Instead, the event indicated that the problem could be limited to a particular network path rather than the server itself.
Monitoring Architecture
European Zabbix node – Host availability check – Trigger webhook – Redis alert state
Asian Zabbix node – Host availability check – Trigger webhook – Redis alert state
Redis correlation:
- Two matching alerts for the same host – server unavailable notification;
- Only one monitoring location reports an alert – no global downtime notification.
Why We Used Two Monitoring Regions
The important part of the solution was not Zabbix itself. It was the use of independent monitoring vantage points. Consider a server located in Asia.
If: Europe – server: unreachable, but: Asia – server: reachable; there is insufficient evidence to conclude that the server is down. The problem may exist somewhere along the European network path.
But if: Europe – server: unreachable and: Asia – server: unreachable, the probability of an actual server-side or data-center-side incident is substantially higher.
By correlating the two observations, the support team could avoid treating every isolated connectivity problem as server downtime.
Why Redis Was Added
Redis provided a lightweight shared state between the two monitoring workflows. Each Zabbix installation could operate independently, while the custom application used Redis to determine whether another monitoring location had already reported the same host as unavailable. This allowed the team to add correlation logic without making either Zabbix installation dependent on the other.
The monitoring architecture therefore remained relatively simple: Zabbix – webhook – correlation script – Redis – notification logic.
What the Solution Changed
The primary objective was not simply to send more monitoring notifications. It was to make those notifications more trustworthy. Before the system was implemented, a connectivity complaint could immediately turn into a discussion about whether the server, hosting network, transit provider, or customer’s own route was responsible. After the monitoring system was introduced, support had an independent source of data from two different regions.
This made it easier to:
- verify whether a reported outage was global or route-specific;
- reduce false downtime notifications;
- investigate recurring connectivity problems;
- separate server availability from Internet routing issues;
- provide the customer with more objective incident information;
- establish whether further investigation was required on the hosting side.
For the Unihost support team, the system also provided a clearer technical record when investigating responsibility for an incident.
Infrastructure Used
Dedicated Compute
- Germany: Intel Xeon E-2388G
- Malaysia: AMD Ryzen 9 7950X3D
- Singapore: AMD EPYC 7313
Monitoring Stack
- Zabbix – Independent infrastructure monitoring from Europe and Asia
- Redis – Shared alert-state storage and correlation
- Custom webhook integration – Processing of Zabbix triggers and notification logic
Broader Infrastructure Challenge
As the customer’s Asian infrastructure grew, networking remained a significant operational issue across multiple locations. This made the monitoring project particularly useful: instead of relying exclusively on individual connectivity reports, both teams had additional technical data for understanding whether a failure was associated with a server or with a particular network path.
The case also demonstrated an important principle for geographically distributed infrastructure: server uptime and network reachability are not always the same thing. A server can be operational while being temporarily unreachable from a particular network or region. For globally distributed services, monitoring architecture needs to account for this distinction.
Engineering Takeaway
The initial requirement for this project was dedicated server capacity. But as the infrastructure expanded across Europe and Asia, the more important problem became observability. The solution was not another server upgrade. It was a monitoring architecture that could observe the same infrastructure from multiple independent locations and correlate the results before declaring an outage. For distributed infrastructure, this provides a much more reliable answer to the question that matters during an incident: Is the server actually down, or is there a problem somewhere on the route to it?
Results at a glance: 2 independent monitoring regions, Europe + Asia availability checks, Zabbix + Redis + custom webhook correlation, reduced false-positive downtime notifications, better distinction between server outages and routing issues, more objective data for network troubleshooting.
