Use Consul service discovery

This section will guide you through integrating HAProxy Data Plane API and HashiCorp Consul. We will pull information from the Consul service discovery registry through the HAProxy Data Plane API and use it to update the load balancer’s configuration.

You can:

query Consul for the IP addresses of your internal services
fill these addresses into the load balancer configuration in real time

Overview Jump to heading

For this setup, you will need three servers:

a Consul server to host the service discovery registry
an HAProxy, HAProxy Enterprise, or HAProxy ALOHA server with the Data Plane API enabled
a web server to serve as an example application to load balance

Consul service discovery diagram

Create the Consul server Jump to heading

To create the Consul server:

On your Consul server machine, install Consul by following the official deployment guide.

Configure it as a Consul server by creating the file /etc/consul.d/server.hcl.

nix
sudo touch /etc/consul.d/server.hcl
sudo chown --recursive consul:consul /etc/consul.d
sudo chmod 640 /etc/consul.d/server.hcl

nix
sudo touch /etc/consul.d/server.hcl
sudo chown --recursive consul:consul /etc/consul.d
sudo chmod 640 /etc/consul.d/server.hcl

Add the following contents to the file. Note especially that server is true and bootstrap_expect is 1, since we are deploying a single-server cluster. Change bind_addr and client_addr to be your server’s IP address.

server.hcl
hcl
datacenter = "dc1"
server = true
bootstrap_expect = 1
bootstrap = true
data_dir = "/opt/consul"
# your Consul server's IP address
bind_addr = "192.168.56.21"
client_addr = "192.168.56.21"
ui_config {
  enabled = true
}
log_level = "INFO"

server.hcl
hcl
datacenter = "dc1"
server = true
bootstrap_expect = 1
bootstrap = true
data_dir = "/opt/consul"
# your Consul server's IP address
bind_addr = "192.168.56.21"
client_addr = "192.168.56.21"
ui_config {
  enabled = true
}
log_level = "INFO"

Enable and start the Consul service.

nix
sudo systemctl enable consul
sudo systemctl start consul

nix
sudo systemctl enable consul
sudo systemctl start consul

Verify that the Consul server is up and running by opening the Consul Web UI, which listens at port 8500, in a browser. It may take some time for the service to become available.

Add a service to the Consul registry Jump to heading

To add a service to Consul so that the HAProxy Data Plane API automatically adds it to the load balancer configuration:

On your web application server, install Consul by following the official deployment guide.

Configure it as a Consul agent by creating the file /etc/consul.d/consul.hcl.

nix
sudo touch /etc/consul.d/consul.hcl
sudo chown --recursive consul:consul /etc/consul.d
sudo chmod 640 /etc/consul.d/consul.hcl

nix
sudo touch /etc/consul.d/consul.hcl
sudo chown --recursive consul:consul /etc/consul.d
sudo chmod 640 /etc/consul.d/consul.hcl

Add the following contents to the file:

consul.hcl
hcl
datacenter = "dc1"
server = false
data_dir = "/opt/consul/"
bind_addr = "192.168.56.22"
retry_join = ["192.168.56.21"]
enable_syslog = true
log_level = "INFO"

consul.hcl
hcl
datacenter = "dc1"
server = false
data_dir = "/opt/consul/"
bind_addr = "192.168.56.22"
retry_join = ["192.168.56.21"]
enable_syslog = true
log_level = "INFO"

Set bind_addr to this web server’s IP address.
Set retry_join to the Consul server’s IP address. The agent uses this to join the Consul cluster.

Enable and start the Consul service.

nix
sudo systemctl enable consul
sudo systemctl start consul

nix
sudo systemctl enable consul
sudo systemctl start consul

Assuming that you have a web application running at port 80, you can register it with the local Consul agent by creating a JSON manifest. For example:
```
example.json
json
{
  "service": {
    "name": "example",
    "port": 80
  }
}
```
```
example.json
json
{
  "service": {
    "name": "example",
    "port": 80
  }
}
```
In this example, we set name to example. This is the name that will appear in the Consul registry. Also, it will appear in the name of the backend in the load balancer configuration.
Register the service with Consul:
```
nix
consul services register example.json
```
```
nix
consul services register example.json
```
The example service should appear on the Services tab in the Consul Web UI.

Configure the HAProxy Data Plane API Jump to heading

To enable the Data Plane API to pull service discovery information from the Consul server, follow these steps:

On the load balancer, make a POST request to the Data Plane API’s /services_discovery/consul endpoint. This tells the API to begin monitoring Consul’s service registry.

nix
curl -X POST \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10
  }' http://localhost:5555/v3/service_discovery/consul

nix
curl -X POST \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10
  }' http://localhost:5555/v3/service_discovery/consul

nix
curl -X POST \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10
  }' http://localhost:5555/v2/service_discovery/consul

nix
curl -X POST \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10
  }' http://localhost:5555/v2/service_discovery/consul

When you add services to Consul, the Data Plane API will automatically create a backend in the configuration. In the example below, a backend named consul-backend-192.168.56.21-8500-example was created automatically from Consul’s service registry. It has a single registered server at the IP address 192.168.56.22:

haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
haproxy
backend consul-backend-192.168.56.21-8500-example
  server SRV_A4u4y 192.168.56.22:80 check weight 128
  server SRV_1NQXA 127.0.0.1:80 disabled weight 128
  server SRV_DWcAP 127.0.0.1:80 disabled weight 128
  server SRV_IIcqg 127.0.0.1:80 disabled weight 128

haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
haproxy
backend consul-backend-192.168.56.21-8500-example
  server SRV_A4u4y 192.168.56.22:80 check weight 128
  server SRV_1NQXA 127.0.0.1:80 disabled weight 128
  server SRV_DWcAP 127.0.0.1:80 disabled weight 128
  server SRV_IIcqg 127.0.0.1:80 disabled weight 128

The Data Plane API has also added several disabled server lines. When you register more instances of the same service, the Data Plane API fills in disabled server slots. You can thus scale up or down without a reload, in most cases.

Add a frontend section that routes traffic to this backend pool of servers. For example:

haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
haproxy
frontend example
  bind :80
  default_backend consul-backend-192.168.56.21-8500-example

haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
haproxy
frontend example
  bind :80
  default_backend consul-backend-192.168.56.21-8500-example

Filter which services are load balanced Jump to heading

You can control which services from Consul’s service registry get added to the load balancer’s configuration.

Set the service_allowlist and service_denylist fields when configuring service discovery. They each take an array of service names to restrict which services to allow or deny from load balancing.

The example below adds the app1 and app2 services to the load balancer configuration, but not app3 and app4. We make a PUT request to update the Consul server we registered before:

nix
curl -X PUT \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10,
    "service_allowlist": ["app1", "app2"],
    "service_denylist": ["app3", "app4"]
  }' http://localhost:5555/v3/service_discovery/consul

nix
curl -X PUT \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10,
    "service_allowlist": ["app1", "app2"],
    "service_denylist": ["app3", "app4"]
  }' http://localhost:5555/v3/service_discovery/consul

nix
curl -X PUT \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10,
    "service_allowlist": ["app1", "app2"],
    "service_denylist": ["app3", "app4"]
  }' http://localhost:5555/v2/service_discovery/consul

nix
curl -X PUT \
  -u admin:adminpwd \
  -H 'Content-Type: application/json' \
  -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10,
    "service_allowlist": ["app1", "app2"],
    "service_denylist": ["app3", "app4"]
  }' http://localhost:5555/v2/service_discovery/consul

Product Documentation

API Documentation

Configuration Documentation

Use Consul service discovery

Overview Jump to heading

Create the Consul server Jump to heading

Add a service to the Consul registry Jump to heading

Configure the HAProxy Data Plane API Jump to heading

Filter which services are load balanced Jump to heading

Product Documentation

API Documentation

Configuration Documentation

Overview Jump to heading #

Create the Consul server Jump to heading #

Add a service to the Consul registry Jump to heading #

Configure the HAProxy Data Plane API Jump to heading #

Filter which services are load balanced Jump to heading #

Privacy Settings

Overview Jump to heading

Create the Consul server Jump to heading

Add a service to the Consul registry Jump to heading

Configure the HAProxy Data Plane API Jump to heading

Filter which services are load balanced Jump to heading