AWS EC2 service discovery

The HAProxy Data Plane API scales your load balancer configuration to dynamically fit the current capacity of your pool of EC2 instances.

The HAProxy Data Plane API:

• Regularly polls the Amazon EC2 API.
• Automatically creates backends.
• Populates your backends with the server addresses of your EC2 instances.
• Uses the HAProxy Runtime API to minimize reloads during Amazon EC2 Auto Scaling events.

Add resource tags to your EC2 instances Jump to heading #

Add the following resource tags to either:

• EC2 instances
• an AWS Auto Scaling Group launch template

The Data Plane API will discover tagged instances and populate a backend section in your load balancer configuration with their IP addresses. The instances may have been created either individually or from an Auto Scaling Group Launch Template.

For example, in a launch template:

Enable service discovery on the load balancer Jump to heading #

To enable service discovery:

1. Add a section such as the following to the Data Plane API configuration file:

   dataplaneapi.yml
   yaml
   service_discovery:
     aws_regions:
     - accesskeyid: KAB[...]2X6
       description: Production environment
       enabled: true
       ipv4address: private
       name: my-disc
       region: eu-west-3
       retrytimeout: 10
       secretaccesskey: qjT[...]uXV
       serverslotsbase: 10
       serverslotsgrowthincrement: 0
       serverslotsgrowthtype: exponential

   dataplaneapi.yml
   yaml
   service_discovery:
     aws_regions:
     - accesskeyid: KAB[...]2X6
       description: Production environment
       enabled: true
       ipv4address: private
       name: my-disc
       region: eu-west-3
       retrytimeout: 10
       secretaccesskey: qjT[...]uXV
       serverslotsbase: 10
       serverslotsgrowthincrement: 0
       serverslotsgrowthtype: exponential

   Alternatively, you can run a curl command with JSON keys such as the following:

   APIv3 APIv2

   nix
   curl -X POST \
     -u admin:adminpwd \
     -H 'Content-Type: application/json' \
     -d '{
           "access_key_id":"KAB[...]2X6",
           "description":"Production environment",
           "enabled":true,
           "ipv4_address":"private",
           "name":"my-disc",
           "region":"eu-west-3",
           "secret_access_key":"qjT[...]uXV",
           "retry_timeout":10
         }' \
     http://localhost:5555/v3/service_discovery/aws

   nix
   curl -X POST \
     -u admin:adminpwd \
     -H 'Content-Type: application/json' \
     -d '{
           "access_key_id":"KAB[...]2X6",
           "description":"Production environment",
           "enabled":true,
           "ipv4_address":"private",
           "name":"my-disc",
           "region":"eu-west-3",
           "secret_access_key":"qjT[...]uXV",
           "retry_timeout":10
         }' \
     http://localhost:5555/v3/service_discovery/aws

   nix
   curl -X POST \
     -u admin:adminpwd \
     -H 'Content-Type: application/json' \
     -d '{
           "access_key_id":"KAB[...]2X6",
           "description":"Production environment",
           "enabled":true,
           "ipv4_address":"private",
           "name":"my-disc",
           "region":"eu-west-3",
           "secret_access_key":"qjT[...]uXV",
           "retry_timeout":10
         }' \
     http://localhost:5555/v2/service_discovery/aws

   nix
   curl -X POST \
     -u admin:adminpwd \
     -H 'Content-Type: application/json' \
     -d '{
           "access_key_id":"KAB[...]2X6",
           "description":"Production environment",
           "enabled":true,
           "ipv4_address":"private",
           "name":"my-disc",
           "region":"eu-west-3",
           "secret_access_key":"qjT[...]uXV",
           "retry_timeout":10
         }' \
     http://localhost:5555/v2/service_discovery/aws

   This command sends configuration and authentication data to the Data Plane API’s AWS service discovery endpoint. Upon successful registration, it returns a JSON result and populates the Data Plane API configuration file.

   In this example, as we do not specify values for the server_slots_base, server_slots_growth_increment, and server_slots_growth_type directives, default values will be used.

   Configuration directive	JSON directive	Description
   AccessKeyID	access_key_id	AWS Access Key ID
   Description	description	Service discovery description
   Enabled	enabled	true or false. If set to false, the Data Plane API will not update server sections in discovered backend sections. Data can become outdated if newer EC2 instances launch or if an existing instance reboots with a new IPv4 address.
   IPV4Address	ipv4_address	private (for the private network, reachable inside the AWS VPC) or public.
   Name	name	Service discovery name
   Region	region	AWS region
   RetryTimeout	retry_timeout	Interval of time in seconds between the reconciliation and the following.
   SecretAccessKey	secret_access_key	AWS Secret Access Key
   ServerSlotsBase	server_slots_base	The minimum amount of server entries per backend section. Defaults to 10.
   ServerSlotsGrowthIncrement	server_slots_growth_increment	The number of additional slots that are allocated for server entries if there are additional entries. Defaults to 0.
   ServerSlotsGrowthType	server_slots_growth_type	Function type to implement when the number of server slots increases: exponential or linear. Defaults to exponential.

   EC2 instances with attached IAM roles

   If the EC2 instance where the Data Plane API is running has an IAM Role attached (as AmazonEC2ReadOnlyAccess), you do not need to specify credentials.

2. Restart the service:

   HAProxy HAProxy Enterprise HAProxy ALOHA

   If running the API via the HAProxy Process Manager, restart HAProxy.

   nix
   sudo systemctl restart hapee-extras-dataplaneapi

   nix
   sudo systemctl restart hapee-extras-dataplaneapi

   In the Services tab, restart the dataplaneapi service.

3. Check the load balancer configuration file. A new backend section displays:

   haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
   haproxy
   backend aws-eu-west-3-my-disc-MyApp-80
     server SRV_ONGQw 172.31.14.35:80 check weight 128
     server SRV_5hi3l 127.0.0.1:80 disabled weight 128
     server SRV_L2rw7 127.0.0.1:80 disabled weight 128
     server SRV_pG3MX 127.0.0.1:80 disabled weight 128
     server SRV_9WFsc 127.0.0.1:80 disabled weight 128
     server SRV_o2UU2 127.0.0.1:80 disabled weight 128
     server SRV_GMuw1 127.0.0.1:80 disabled weight 128
     server SRV_3Id36 127.0.0.1:80 disabled weight 128
     server SRV_6Ivld 127.0.0.1:80 disabled weight 128
     server SRV_jYF88 127.0.0.1:80 disabled weight 128

   haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
   haproxy
   backend aws-eu-west-3-my-disc-MyApp-80
     server SRV_ONGQw 172.31.14.35:80 check weight 128
     server SRV_5hi3l 127.0.0.1:80 disabled weight 128
     server SRV_L2rw7 127.0.0.1:80 disabled weight 128
     server SRV_pG3MX 127.0.0.1:80 disabled weight 128
     server SRV_9WFsc 127.0.0.1:80 disabled weight 128
     server SRV_o2UU2 127.0.0.1:80 disabled weight 128
     server SRV_GMuw1 127.0.0.1:80 disabled weight 128
     server SRV_3Id36 127.0.0.1:80 disabled weight 128
     server SRV_6Ivld 127.0.0.1:80 disabled weight 128
     server SRV_jYF88 127.0.0.1:80 disabled weight 128

   The name of the new backend comprises the following parts:

   aws-<AWS region>-<Service discovery name>-<HAProxy:Service:Name tag value>-<HAProxy:Service:Port tag value>

   This backend may only have several servers enabled out of the total number of servers specified by the ServerSlotsBase directive. When you register more instances of the same service, the API fills in disabled server slots. You can thus scale up or down without a reload, in most cases.

4. Edit the new backend as needed. For example, you can specify a load balancing algorithm via the balance directive.

5. Configure a frontend section that routes traffic to this backend pool of servers. For example:

   haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
   haproxy
   frontend example
     bind :80
     default_backend aws-eu-west-3-my-disc-MyApp-80

   haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
   haproxy
   frontend example
     bind :80
     default_backend aws-eu-west-3-my-disc-MyApp-80