Grafana

This guide shows how to export the load balancer’s live traffic metrics to Grafana. We enable the built-in Prometheus metrics endpoint, which a Prometheus server scrapes at an interval. Then, Grafana displays graphs of the data stored in the Prometheus server.

Set up Prometheus and Grafana Jump to heading #

1. Install a Prometheus server and configure it to scrape metrics from your load balancer. Follow our Prometheus guide, which shows how to:

   • enable the load balancer’s built-in Prometheus metrics exporter
   • configure your Prometheus server to scrape the Prometheus endpoint for live metrics

2. Prometheus hosts a dashboard on port 9090. Open the dashboard and verify that it can scrape the endpoint successfully. Check under the menu Status > Targets where you should see the load balancer endpoint with a state of UP.

3. Follow the official Grafana documentation to install Grafana.

4. Log into the Grafana dashboard at port 3000 using the username and password admin. You will be prompted to immediately change the password.

5. In the left-hand navigation, go to the Configuration screen and add a new Prometheus data source. Set the URL to the address where your Prometheus server is listening (for example, http://localhost:9090), then save it.

6. Click the plus sign (+) in the left-hand navigation, then Create > Dashboard. Add a new panel to begin creating graphs that use metrics from the Prometheus data source.

   Tip

   Try importing a pre-baked dashboard, such as the Ricardo F HAProxy Dashboard. In this case, copy the JSON and paste it into the textbox on the Import screen. Then, click Load.

Further considerations Jump to heading #

• The easiest way to achieve high availability for the above monitoring stack is to spin up a separate node with Prometheus and Grafana installed, and ingest the same data. For most use cases this is more than enough.

• If you require a more robust high-availability solution, you can use Thanos. The project description states:

  Thanos leverages the Prometheus 2.0 storage format to cost-efficiently store historical metric data in any object storage while retaining fast query latencies. Additionally, it provides a global query view across all Prometheus installations and can merge data from Prometheus HA pairs on the fly.

• Also consider enabling the Prometheus Node Exporter to log the CPU, memory, and other system stats on every load balancer node. This will help catch maxed out CPU, high memory, and network congestion issues.