Active/Active clustering

In an active-active cluster, two or more HAProxy Enterprise nodes receive traffic in a load-balanced rotation. This allows you to scale out your load-balancing capacity. There are several ways to achieve this setup, including:

• Using DNS round-robin
• Deploy two tiers of load balancers
• Using Anycast with the HAProxy Enterprise Route Health Injection module (see Route Health Injection)

Option 1: DNS round-robin Jump to heading #

To create an active-active cluster of load balancers, you can use DNS round-robin to send traffic to each load balancer in a rotation.

To remove unhealthy HAProxy Enterprise nodes from round-robin load-balancing rotation, you need a way to detect failure. If your DNS server supports health checking upstream servers, you can use that feature to remove unhealthy nodes. However, in this section, we assume that your DNS server does not support health checking. Instead, you need to configure the HAProxy Enterprise VRRP module to assign a virtual IP address to each load balancer. In the event that one of your load balancers fails, its virtual IP address will transfer to a healthy load balancer node.

A side effect of this failover is that your healthy node, which had been receiving its own share of traffic already, will temporarily take on more traffic until the other node recovers.

In this guide, we set up two load balancers: both actively receiving traffic.

Configure VRRP Jump to heading #

1. Install the VRRP module using your system’s package manager on both HAProxy Enterprise nodes that will participate in your load balancer cluster:

   apt yum zypper pkg

   nix
   sudo apt-get install hapee-extras-vrrp

   nix
   sudo apt-get install hapee-extras-vrrp

   nix
   sudo yum install hapee-extras-vrrp

   nix
   sudo yum install hapee-extras-vrrp

   nix
   sudo zypper install hapee-extras-vrrp

   nix
   sudo zypper install hapee-extras-vrrp

   nix
   sudo pkg install hapee-extras-vrrp

   nix
   sudo pkg install hapee-extras-vrrp

2. Decide on a unique Virtual Router Identifier (VRID) for each of the two clusters we will create. A VRID can be any number between 1 and 255. It allows the instances in a cluster to share a virtual router and virtual IP address. The VRID must be the same on all instances in a cluster. When there are multiple VRRP clusters in the environment, as there are in the active-active configuration, the VRID must be unique for each cluster.

   When defining a new cluster, do not use a VRID already in use for another cluster. If the vrrp service has been started and enabled, you can list VRIDs already in use by executing the following command:

   nix
   sudo tcpdump -vvvenns0 -c 5 -i eth0 vrrp | grep -o "vrid [0-9]*"

   nix
   sudo tcpdump -vvvenns0 -c 5 -i eth0 vrrp | grep -o "vrid [0-9]*"

   output
   text
   [...]
   5 packets captured
   6 packets received by filter
   0 packets dropped by kernel
   vrid 161
   vrid 155

   output
   text
   [...]
   5 packets captured
   6 packets received by filter
   0 packets dropped by kernel
   vrid 161
   vrid 155

   In our example configuration, we will use VRID 51 for one VRRP cluster and VRID 52 for the other cluster.

3. Decide on a unique password for each cluster. A VRRP password is a clear-text string for use by all instances in a VRRP cluster. It does not provide security, but it ensures that VRRP instances from other clusters cannot join this cluster due to errors in configuration, such as a duplicate VRID. In our example configuration, we’ll use 1234 as the password for the cluster with VRID 51 and 5678 as the password for the cluster with VRID 52.

4. On the first load balancer, edit the file /etc/hapee-extras/hapee-vrrp.cfg. The file contains one vrrp_instance block. Delete the block and insert the following code in its place:

   hapee-vrrp.cfg
   text
   vrrp_instance vrrp_1 {
     interface enp0s8	              # Change network interface name
     state MASTER
     virtual_router_id 51           # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 1234               # Password unique to this cluster
     }
     priority 101
     virtual_ipaddress_excluded {
       192.168.50.10                # NEW IP address
     }
     track_interface {
       enp0s8 weight -2             # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }
   vrrp_instance vrrp_2 {
     interface enp0s8               # Change network interface name
     state BACKUP
     virtual_router_id 52           # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 5678               # Password unique to this cluster
     }
     priority 100
     virtual_ipaddress_excluded {
       192.168.50.11                # NEW IP address
     }
     track_interface {
       enp0s8 weight -2             # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }

   hapee-vrrp.cfg
   text
   vrrp_instance vrrp_1 {
     interface enp0s8	              # Change network interface name
     state MASTER
     virtual_router_id 51           # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 1234               # Password unique to this cluster
     }
     priority 101
     virtual_ipaddress_excluded {
       192.168.50.10                # NEW IP address
     }
     track_interface {
       enp0s8 weight -2             # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }
   vrrp_instance vrrp_2 {
     interface enp0s8               # Change network interface name
     state BACKUP
     virtual_router_id 52           # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 5678               # Password unique to this cluster
     }
     priority 100
     virtual_ipaddress_excluded {
       192.168.50.11                # NEW IP address
     }
     track_interface {
       enp0s8 weight -2             # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }

5. Make the following modifications to these two vrrp_instance blocks:

   • Replace the interface values with the name of the network interface on which this server receives traffic. The interface is the same for the two instances.
   • Replace the virtual_router_id fields with the VRIDs determined previously. Each instance should have a different VRID.
   • Replace the auth_pass fields with the passwords determined previously. Each instance should have a different password.
   • Replace the IP addresses listed in the virtual_ipaddress_excluded blocks with addresses you’d like to use to receive traffic. These new addresses should fall within the interface’s IP subnet, but should not be assigned to any server already. Each instance should have a different IP address.
   • In the track_interface blocks, specify the same interface name that is specified in the interface fields.

   With these modifications, the configuration creates two virtual IP addresses: 192.168.50.10 and 192.168.50.11. The first has its state set to MASTER, while the second is set to BACKUP. This means that this load balancer owns the first IP address but will only bind to the second IP address as a backup if its primary owner fails.

6. Copy the file /etc/hapee-extras/hapee-vrrp.cfg to the second load balancer.

7. On the second load balancer, edit the file and make the following changes:

   • Swap the state values so that the first vrrp_instance block is set to BACKUP, while the second is set to MASTER.
   • Swap the priority values.

   hapee-vrrp.cfg
   text
   vrrp_instance vrrp_1 {
     interface enp0s8	            # Change network interface name
     state BACKUP
     virtual_router_id 51         # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 1234             # Password unique to this cluster
     }
     priority 100
     virtual_ipaddress_excluded {
       192.168.50.10              # NEW IP address
     }
     track_interface {
       enp0s8 weight -2           # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }
   vrrp_instance vrrp_2 {
     interface enp0s8             # Change network interface name
     state MASTER
     virtual_router_id 52         # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 5678             # Password unique to this cluster
     }
     priority 101
     virtual_ipaddress_excluded {
       192.168.50.11              # NEW IP address
     }
     track_interface {
       enp0s8 weight -2           # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }

   hapee-vrrp.cfg
   text
   vrrp_instance vrrp_1 {
     interface enp0s8	            # Change network interface name
     state BACKUP
     virtual_router_id 51         # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 1234             # Password unique to this cluster
     }
     priority 100
     virtual_ipaddress_excluded {
       192.168.50.10              # NEW IP address
     }
     track_interface {
       enp0s8 weight -2           # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }
   vrrp_instance vrrp_2 {
     interface enp0s8             # Change network interface name
     state MASTER
     virtual_router_id 52         # VRID unique to this cluster
     authentication {
       auth_type PASS
       auth_pass 5678             # Password unique to this cluster
     }
     priority 101
     virtual_ipaddress_excluded {
       192.168.50.11              # NEW IP address
     }
     track_interface {
       enp0s8 weight -2           # Change network interface name
     }
     track_script {
       chk_sshd
       chk_lb
     }
   }

   With these configurations, each load balancer serves as a backup for the other. If one fails, its virtual IP will transfer to the other node, which will at that point answer requests from both addresses until the failed node recovers.

8. Start the hapee-extras-vrrp service on both servers:

   nix
   sudo systemctl start hapee-extras-vrrp
   sudo systemctl enable hapee-extras-vrrp

   nix
   sudo systemctl start hapee-extras-vrrp
   sudo systemctl enable hapee-extras-vrrp

9. Edit your HAProxy Enterprise configurations to listen on the virtual IP addresses. Note that you must append the field transparent, indicating that the address will be bound even if it does not belong to the local machine, which is necessary since the virtual IP will float to the active load balancer.

   haproxy
   frontend myfrontend
     mode http
     bind 192.168.50.10:80 transparent
     bind 192.168.50.11:80 transparent
     default_backend web_servers
   backend web_servers
     server s1 192.168.50.20:80

   haproxy
   frontend myfrontend
     mode http
     bind 192.168.50.10:80 transparent
     bind 192.168.50.11:80 transparent
     default_backend web_servers
   backend web_servers
     server s1 192.168.50.20:80

   Alternatively, if you prefer to not add transparent to every bind line, you can set the kernel parameter ip_nonlocal_bind. Edit the file /etc/sysctl.conf and add the line net.ipv4.ip_nonlocal_bind=1:

   sysctl.conf
   ini
   net.ipv4.ip_nonlocal_bind=1

   sysctl.conf
   ini
   net.ipv4.ip_nonlocal_bind=1

   Then run sudo sysctl -p to reload the configuration.

   As a third option, you can listen on all IP addresses assigned to the server by omitting the IP address:

   haproxy
   frontend myfrontend
     mode http
     bind :80
     default_backend web_servers

   haproxy
   frontend myfrontend
     mode http
     bind :80
     default_backend web_servers

In essence, we have created an active-active cluster by creating two active-standby clusters, one in each direction. Each load balancer has an active IP address, but also serves as a standby server for the other IP address.

Configure DNS round-robin Jump to heading #

DNS allows you to rotate which IP address is returned in response to a DNS query. When clients request your service by its domain name, they will receive the IP address of the next load balancer in the list. Clients tend to cache DNS results, so once a client receives a DNS answer, it will likely continue making requests to the same load balancer until the DNS answer expires.

Create an A record for each load balancer IP address. For example, consider the following DNS zone file:

text
$ORIGIN example.local.
@       3600 IN SOA dns1.example.local. admin.example.local. (
            2017042745 ; serial
            1800       ; refresh (30 minutes)
            900        ; retry (15 minutes)
            1209600    ; expire (2 weeks)
            60       ; minimum (1 minute)
         )
        
        3600 IN NS dns1.example.local.
        
@     60 IN A     192.168.50.10
@     60 IN A     192.168.50.11
www   60 IN CNAME @

text
$ORIGIN example.local.
@       3600 IN SOA dns1.example.local. admin.example.local. (
            2017042745 ; serial
            1800       ; refresh (30 minutes)
            900        ; retry (15 minutes)
            1209600    ; expire (2 weeks)
            60       ; minimum (1 minute)
         )
        
        3600 IN NS dns1.example.local.
        
@     60 IN A     192.168.50.10
@     60 IN A     192.168.50.11
www   60 IN CNAME @

Depending on your DNS server, you may need to enable load-balancing of the A records. It is also a good idea to set a shorter TTL for these records to avoid staying cached in intermediate nameservers for long.

Option 2: Two tiers of load balancers Jump to heading #

To create an active-active cluster of load balancers, you can place another tier of HAProxy Enterprise load balancers in front. These load balancers operate at Layer 4, the Network layer, and load balance the load balancers. Because HAProxy Enterprise has the ability to check the health of the lower tier of load balancers, it can remove unhealthy nodes as needed.

Deploy Layer 7 load balancers Jump to heading #

Create a tier of load balancers that will, themselves, be load-balanced:

1. Deploy at least two servers to host HAProxy Enterprise. Add one or more static IP addresses to each server.

2. Install HAProxy Enterprise on each server.

3. Configure them to receive traffic on the assigned IP addresses and relay it to backend application servers. Other than the addresses, their configurations should match.

Deploy Layer 4 load balancers Jump to heading #

Create a tier of load balancers that will load balance the layer 7 HAProxy Enterprise servers:

1. Deploy one or more servers to host HAProxy Enterprise. These will load balance the other load balancers.

2. If you deploy multiple load balancers in this tier, configure them in active-standby mode.

3. Configure them to relay traffic to the layer 7 load balancers. In the following example, a listen section defines the pool of upstream load balancers:

   haproxy
   listen load_balancer_cluster
     mode tcp
     bind :80
     option tcplog
     balance roundrobin
     server lb1 192.168.1.25:80 check
     server lb2 192.168.1.26:80 check

   haproxy
   listen load_balancer_cluster
     mode tcp
     bind :80
     option tcplog
     balance roundrobin
     server lb1 192.168.1.25:80 check
     server lb2 192.168.1.26:80 check

Active/active setup on AWS Jump to heading #

To create an active-active cluster of HAProxy Enterprise load balancers in AWS, you can place a Network Load Balancer in front. This load balancer operates at Layer 4, the Network layer, and load balances the HAProxy Enterprise load balancers. Because AWS Network Load Balancer can check the health of the lower-tier of load balancers, it can remove unhealthy load balancers as needed.

We will:

• Create a virtual network dedicated to your AWS account (VPC).
• Deploy several HAProxy Enterprise AMIs.
• Create an AWS target group.
• Create an AWS Network Load Balancer in a single Availability Zone.
• Test the setup.

Create a VPC Jump to heading #

You can launch HAProxy Enterprise nodes in a Virtual Private Cloud (VPC), which is a virtual network similar to a traditional network.

1. Open the Amazon VPC console, then click Launch VPC Wizard.

2. Create a VPC with a Single Public Subnet with Public subnet’s Availability Zone set to the availability zone of your choice, for example, eu-west-3a. In other fields, keep the default values or select other values that better suit your needs.

3. In the Virtual Private Cloud section on the left, select Your VPCs, then write down the ID of the new VPC. For example, vpc-0deecc96935b9ef73.

Deploy HAProxy Enterprise AMIs Jump to heading #

You can launch HAProxy Enterprise nodes directly from the AWS Marketplace. Create two or more HAProxy Enterprise nodes on AWS, with the following characteristics:

In other fields, keep the default values or select other values that better suit your needs.

Create an AWS target group Jump to heading #

A target group routes requests to one or more registered targets, such as HAProxy Enterprise nodes, using the TCP protocol and the port number that you specify.

1. Open the Amazon EC2 console.

2. In the Load Balancing section on the left, select Target Groups.

3. Create a target group with the following characteristics, then click Next.

   Field	Value
   Target type	Instances
   Protocol	TCP
   VPC	The virtual private cloud (VPC) you created previously. For example, vpc-0deecc96935b9ef73.

   In the other fields, keep the default values or select other values that better suit your needs.

4. On the Register targets page, select the HAProxy Enterprise nodes you created previously. Then click Include as pending below.

   

5. Click Create target group.

Create an AWS Network Load Balancer Jump to heading #

The AWS Network Load Balancer selects a target HAProxy Enterprise node using a flow hash algorithm based on the source and destination IP addresses and ports, the protocol, and the TCP sequence number.

1. In the Load Balancing section on the left, select Load Balancers.

2. Create a Network Load Balancer with the following characteristics:

   Field	Value
   Scheme	Internet-facing.
   VPC	The virtual private cloud (VPC) you created previously. For example, vpc-0deecc96935b9ef73.
   Mappings	Select your Availability Zone and the corresponding subnet.
   Listeners	Default (a listener that accepts TCP traffic on port 80).
   Default action	Select the target group you created previously.

   In the other fields, keep the default values or select other values that better suit your needs.

3. Click Create load balancer.

Test your setup Jump to heading #

Once you have created a Network Load Balancer, wait a few minutes and check that the HAProxy Enterprise nodes in your target group have passed the initial health check. You can then test that the layer 4 AWS Network Load Balancer sends traffic to your layer 7 HAProxy Enterprise load balancers.

1. In the Load Balancing section on the left, select Target Groups.

2. Select the newly created target group, and check that your HAProxy Enterprise nodes are ready.

   

3. In the Load Balancing section on the left, select Load Balancers.

4. Select the newly created load balancer.

5. Copy the DNS name of the load balancer and paste it into the address field of a web browser.

   For example, strawberry-f9f565c7eb5b3cd3.elb.eu-west-3.amazonaws.com.

   The browser displays the statistics page of your HAProxy Enterprise node.

   If you launched nodes with different versions of HAProxy Enterprise, paste the DNS name of the Network Load Balancer into the address field of another web browser or in a private browser window. The browser should display the statistics page of one of your other HAProxy Enterprise nodes, which runs another version of HAProxy Enterprise. Since each TCP connection is routed to a single target as long as the connection is active, you may have to try connecting several times.

You can now:

• configure your backend server pool.
• edit the frontend section of your HAProxy configuration file (each AWS Network Load Balancer in the Availability Zone has a static IP).
• edit the backend section of your HAProxy configuration file.
• copy the configuration file to all HAProxy Enterprise nodes in the AWS target group.

See also Jump to heading #

• GSLB
• NS1 integration
• Route Health Injection
• AWS Available Regions
• Create a Network Load Balancer
• What is Amazon VPC?
• What is a Network Load Balancer?