HAProxy Data Plane API 2.3 expands its service discovery mechanisms and introduces native support for discovering AWS EC2 instances and auto-scaling groups. It also adds a new configuration file that supports HCL and YAML, an Inotify configuration watcher, and Syslog support.
Register for our webinar to learn more about this release.
HAProxy Data Plane API version 2.3 is now available, and you will find it in the 2.3 version of the Alpine Docker image. It does much to simplify the complexity exposed to users, as well as expand its set of features in areas like service discovery, logging, and validation.
The most obvious change is moving the command-line arguments needed to start the API into a YAML configuration file. This change, which outwardly looks cosmetic, is an indication of the maturity of the API. New features that require their own command-line arguments, such as the new Syslog support, make it clear that this API has grown bigger than its initial role, which was focused on generating an HAProxy configuration. It is fast becoming the interface between HAProxy and the services that surround it. Managing that complexity will be crucial going forward.
This version also supports service discovery of EC2 instances based on AWS instance tags. The API generates server lines in your HAProxy configuration from the instances it discovers. The importance of the cloud can’t be denied or even understated, and gaining service discovery in EC2 for HAProxy will be welcomed news for many organizations.
This latest release of the HAProxy Data Plane API also monitors the HAProxy configuration using Inotify, which means that you can make manual changes to your HAProxy configuration and use the Data Plane API simultaneously without worrying that the methods will conflict. It also adds support for sending its request logs over Syslog and allows you to customize the format using Apache Log Format directives. There are several other exciting features that were introduced, so check out the full release information below!
This release was a community effort and could not have been made possible without all of the hard work from everyone involved, both on GitHub and Slack.
HCL / YAML Configuration File
There are quite a few arguments you need to set when starting the Data Plane API.
| $ sudo dataplaneapi \ | |
| --host 127.0.0.1 \ | |
| --port 5555 \ | |
| --haproxy-bin /usr/sbin/haproxy \ | |
| --config-file /etc/haproxy/haproxy.cfg \ | |
| --reload-delay 5 \ | |
| --reload-cmd "service haproxy reload" \ | |
| --restart-cmd "service haproxy restart" \ | |
| --userlist haproxy-dataplaneapi \ | |
| --transaction-dir /tmp/haproxy |
This release introduces support for storing those command-line arguments and settings in a YAML configuration file—it also supports the HCL language as a file format. Version 2.3 of the Data Plane API requires this YAML file upon startup. Here’s an example dataplaneapi.yaml file:
| dataplaneapi: | |
| host: "127.0.0.1" | |
| port: 5555 | |
| user: | |
| - name: "admin" | |
| insecure: false | |
| password: "$5$hY1nZJpAFNDjEN$LhRDisCDdKPzQw.Z7IyIy0mwWrnM0lK6OqFy5IIVEy2" | |
| transaction: | |
| transaction_dir: "/tmp/haproxy" | |
| haproxy: | |
| config_file: "/etc/haproxy/haproxy.cfg" | |
| haproxy_bin: "/usr/sbin/haproxy" | |
| reload: | |
| reload_cmd: "service haproxy reload" | |
| restart_cmd: "service haproxy restart" | |
| reload_delay: 5 |
Or its HCL equivalent:
| dataplaneapi { | |
| host = "127.0.0.1" | |
| port = 5555 | |
| user "admin" { | |
| insecure = true | |
| password = "adminpwd" | |
| } | |
| transaction { | |
| transaction_dir = "/tmp/haproxy" | |
| } | |
| } | |
| haproxy { | |
| config_file = "/etc/haproxy/haproxy.cfg" | |
| haproxy_bin = "/usr/sbin/haproxy" | |
| reload { | |
| reload_cmd = "service haproxy reload" | |
| restart_cmd = "service haproxy restart" | |
| reload_delay = "5" | |
| } | |
| } |
Load it with the -f command-line argument. Note how the previous command with all of the arguments is now condensed:
| $ sudo dataplaneapi -f dataplaneapi.yaml |
In this file, you can specify the Basic authentication users that have access to the API by setting the user block. In previous versions, you had to configure this as a userlist section in the haproxy.cfg file. You can find complete examples for YAML here and HCL here. A converter for migrating between formats is also supplied through the following command:
| $ go run configuration/converter/converter.go <original_fmt> <new_fmt> |
This assumes that the root path is the HAProxy Data Plane API base source directory.
AWS EC2 Service Discovery
As organizations continue to move their workloads into the cloud and embrace cloud-native technologies, service discovery has become increasingly important. HAProxy Technologies recognized the importance of this need and introduced support for DNS Service Discovery in HAProxy version 1.8, released in 2017. The previous release of the HAProxy Data Plane API, version 2.2, expanded in this area by adding native support for Consul.
This release adds service discovery support within AWS EC2. Once you configure the HAProxy Data Plane API with the proper credentials, it’s able to poll the AWS API at a defined interval and populate your HAProxy backends with the addresses of your EC2 instances. If a backend does not exist, it will automatically create it. It also harnesses the HAProxy Runtime API to minimize reloads during scaling events.
To get started, first you will need to add the appropriate tags to the instances that you want to be discovered. The API can detect the following AWS tags:
Tag | Value |
HAProxy:Service:Name (required) | Name of the service |
HAProxy:Service:Port (required) | Port number that you want HAProxy to send traffic to |
HAProxy:Instance:Port (optional) | Overrides the service port for a single instance |
Your AWS tags should look like this:
.png)
Next, you will need to configure the HAProxy Data Plane API either by sending a POST request with the configuration details or by editing the HAProxy Data Plane API’s HCL/YAML configuration directly.
Below, we send a POST request using curl to the /service_discovery/aws endpoint to set up service discovery:
| $ curl -H 'Content-Type: application/json' \ | |
| -u dataplaneapi:mypassword \ | |
| -d \ | |
| '{ | |
| "access_key_id":"XXXXXXXXXXXXXXX", | |
| "secret_access_key":"XXXXXXXXXXXXXXX", | |
| "description":"AWS EC2 Production Environment", | |
| "enabled":true, | |
| "ipv4_address":"private", | |
| "name":"www", | |
| "region":"us-east-1", | |
| "retry_timeout":10}' \ | |
| http://10.2.0.3:5555/v2/service_discovery/aws |
Another way to do it is to change the API’s configuration file, adding the service_discovery block:
| service_discovery: | |
| aws_regions: | |
| - accesskeyid: <Insert AWS Access Key> | |
| description: production | |
| enabled: true | |
| ipv4address: private | |
| name: production | |
| region: us-east-1 | |
| retrytimeout: 10 | |
| secretaccesskey: <Insert AWS Secret Access Key> | |
| serverslotsbase: 10 | |
| serverslotsgrowthincrement: 10 | |
| serverslotsgrowthtype: exponential |
Your AWS credentials can also be specified through the AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID environment variables, or inherited by the IAM role with AmazonEC2ReadOnlyAccess permissions assigned to the EC2 instance.
Once running, service discovery will populate properly tagged EC2 instances within your HAProxy configuration:
| backend aws-us-east-1-production-www-80 | |
| server SRV_0tWxu 172.31.31.174:80 check weight 128 | |
| server SRV_3pQJ7 172.31.31.151:80 check weight 128 | |
| server SRV_Bd56t 127.0.0.1:80 disabled weight 128 | |
| server SRV_i2SMt 127.0.0.1:80 disabled weight 128 | |
| server SRV_YvSgD 127.0.0.1:80 disabled weight 128 | |
| server SRV_5xAV0 127.0.0.1:80 disabled weight 128 | |
| server SRV_YU99z 127.0.0.1:80 disabled weight 128 | |
| server SRV_ampta 127.0.0.1:80 disabled weight 128 | |
| server SRV_7Z7S5 127.0.0.1:80 disabled weight 128 | |
| server SRV_75KLk 127.0.0.1:80 disabled weight 128 |
The disabled servers act as empty slots that HAProxy fills in with servers on-the-fly to scale services up and down without requiring a reload.
You can specify services that you want to explicitly allow or deny from discovery by using the Allowlist and Denylist configuration arrays. For example, if you wanted to allow services that have the Environment tag Production only, you would specify this within your configuration.
| allowlist: | |
| - key: tag:Environment | |
| value: Production |
ACL Runtime API Updates
HAProxy ACLs are expressions that you can define in your HAProxy configuration that allow you to match anything found within the request or response headers and metadata. Then, you can reference them to define routing, blocking, and logging rules. This release introduces support for interacting with defined ACLs via HAProxy’s Runtime API. You can now view, add, and remove ACL entries that are stored in-memory. The following endpoints have been added:
HTTP Verb | Endpoint | Description |
GET | /services/haproxy/runtime/acls | Returns a list of ACLs. Equivalent to show acl from the Runtime API |
GET | /services/haproxy/runtime/acls/{id} | Returns the contents of a single ACL. Equivalent to show acl #id from the Runtime API. |
POST | /services/haproxy/runtime/aclfileentries | Adds a new entry to an ACL. Equivalent to add acl #id <entry> from the Runtime API. |
GET, DELETE | /services/haproxy/runtime/aclfileentries{id} | Returns or deletes a single ACL entry. Equivalent to del acl #id <entry> from the Runtime API. |
Inotify Configuration Watcher
The HAProxy Data Plane API lets you manage the HAProxy configuration through a RESTful interface. However, in some cases, you may want to make manual changes directly within the configuration. This release introduces support for inotify, which is a Linux kernel API that allows applications to monitor the filesystem for changes. This means that you can modify the HAProxy configuration directly and the HAProxy Data Plane API will automatically detect those changes, load them into its memory, and increment the current version on the HAProxy configuration file. It now also stores a new value at the top of the configuration named _md5hash, which holds the current MD5 hash of the configuration. This new functionality will only re-read the configuration and does not issue any reloads or restarts based on updates. This feature is enabled by default but can be disabled with the command-line argument --disable-inotify.
Syslog Support
Logging is a critical component to observability within any infrastructure. Whether you’re using logs for auditing and accountability purposes or for debugging, they provide deep insight and historical reference of your systems.
This release adds support for sending HAProxy Data Plane API request and debug logs to a Syslog server over a Unix socket or TCP/IP. It adds the following new command-line arguments
CLI | YAML / HCL Configuration | Description |
|
| Syslog address (with the port declaration in case of TCP type) where logs should be forwarded |
|
| Syslog server protocol. Accepted values: [tcp|tcp4|tcp6|unix|unixgram] |
|
| String to tag the syslog messages (default: dataplaneapi) |
|
| Define the required syslog messages level. Accepted values: [debug|info|notice|warning|error|critical|alert|emergency] |
|
| Define the Syslog facility number. Accepted values: [kern|user|mail|daemon|auth|syslog|lpr|news|uucp|cron|authpriv|ftp|local0|local1|local2|local3|local4|local5|local6|local7] |
This release also introduces support for using the Apache common log format to structure request log entries. The default value is:
%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-agent}i” %{us}T
This can be modified using the --apache-common-log-format option from the command-line.
Customized Configuration Validation
Do you split your HAProxy configuration into multiple .cfg files? By default, the Data Plane API validates only the main configuration file by calling haproxy -f /etc/haproxy/haproxy.cfg -c. You can now specify validate_cmd within the reload section of the API’s YAML configuration file to define an alternate command to use for validation.
| reload: | |
| reload_delay: 5 | |
| reload_cmd: service haproxy reload | |
| restart_cmd: service haproxy restart | |
| validate_cmd: /usr/local/bin/validate-haproxy.sh |
After adding validate_cmd, the API sets an environment variable named DATAPLANEAPI_TRANSACTION_FILE that contains the to-be-validated configuration that it generates. You’ll need to define a shell script that validates any HAProxy configuration files you have in addition to the file pointed at by the DATAPLANEAPI_TRANSACTION_FILE variable.
An example validation shell script, which we store at /usr/local/bin/validate-haproxy.sh, is as follows:
| #!/bin/bash | |
| haproxy -f /etc/haproxy/resolvers.cfg -f $DATAPLANEAPI_TRANSACTION_FILE -c |
Miscellaneous
New command-line and YAML/HCL configuration options for setting the uid and gid of the HAProxy Data Plane API process. The command-line flags are
--uid <uid>and--gid <gid>the HCL/YAML arguments are uid / gid.Query string parameters force_reload and skip_reload were added to the Map file and SSL certificate storage endpoints, allowing you to force a reload or skip reloads entirely. The skip_reload parameter has higher precedence.
The query string parameter acl_name allows for querying an ACL by its name.
Transactions that fail or are committed after other transactions are made could result in a mismatched version number. These transactions will now be marked as outdated and will return an HTTP 406 Not Acceptable.
This release supports all available options to the
serveranddefault-serverdirectives.Support for
http-request returnhas been added, which exposes the native response generator that was introduced in HAProxy 2.2The HAProxy Data Plane API can now be configured to write its pid file using the
--pid-filecommand-line argument orpid_filewithin thedataplaneapisection of the configuration file.
Contributors
We’d like to thank the code contributors who helped make this version possible:
Contributor | Area |
Amel Husic | FEATURE CLEANUP BUILD TEST DOCUMENTATION |
Andjelko Iharos | FEATURE BUG FIX BUILD TEST |
Daniel Corbett | BUG FIX |
Dario Tranchitella | FEATURE BUG FIX BUILD TEST DOCUMENTATION REORG OPTIM |
Goran Galinec | FEATURE CLEANUP |
Marko Juraga | FEATURE BUG FIX BUILD CLEANUP |
Robert Maticevic | FEATURE BUG FIX TEST DOCUMENTATION |
Stefan Scheglmann | FEATURE BUILD DOCUMENTATION |
Zlatko Bratkovic | FEATURE BUG FIX BUILD DOCUMENTATION TEST |
Conclusion
The HAProxy Data Plane API version 2.3 adds service discovery capabilities with a native AWS EC2 integration. It also adds a new configuration file that supports HCL and YAML, an Inotify configuration watcher, and Syslog support. The HAProxy Data Plane API continues to get better thanks to the active community surrounding it. If you’d like to get involved, head over to our GitHub repository to learn more!
Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.
