HAProxy Data Plane API 3.0 release notes

Key changes in the HAProxy Data Plane API 3.0 release include:

Breaking changes Jump to heading #

Data Plane API moves to v3, and our API resources now start with /v3 instead of /v2. In v3 of the API we introduce a number of breaking changes to improve API usability and readability. These are all based on feedback in the last four years since the release of v2.0.0. Working on that feedback we changed things around and organized it better to make the API more usable for everyone.

See also Deprecated features.

Removed _version and data wrapper fields in responses Jump to heading #

Data Plane API uses versions for optimistic locking of the configuration, and in the initial design we returned data on each configuration endpoint with _version and data fields as top keys in the response object. Sometime later we also included the Configuration-Version header and we had responses like this:

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/backends

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/backends

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 12:50:24 GMT
Content-Length: 313
Connection: close
{
  "_version": 1,
  "data": [
    {
      "balance": {
        "algorithm": "roundrobin"
      },
      "name": "be_static"
    },
    {
      "balance": {
        "algorithm": "roundrobin"
      },
      "name": "be_app"
    }
  ]
}

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 12:50:24 GMT
Content-Length: 313
Connection: close
{
  "_version": 1,
  "data": [
    {
      "balance": {
        "algorithm": "roundrobin"
      },
      "name": "be_static"
    },
    {
      "balance": {
        "algorithm": "roundrobin"
      },
      "name": "be_app"
    }
  ]
}

So we had the configuration version both in a wrapped response and in the header. In v3 we are getting rid of the redundant configuration version in the response, making the response more straightforward. This also simplifies the user’s ability to change an existing resource, as with this flat structure, it could fetch the resource, change a few fields and send back the same response, without having to unpack version and data from the response.

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/backends

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/backends

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 12:55:23 GMT
Content-Length: 291
Connection: close
[
  {
    "balance": {
      "algorithm": "roundrobin"
    },
    "name": "be_static"
  },
  {
    "balance": {
      "algorithm": "roundrobin"
    },
    "name": "be_app"
  }
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 12:55:23 GMT
Content-Length: 291
Connection: close
[
  {
    "balance": {
      "algorithm": "roundrobin"
    },
    "name": "be_static"
  },
  {
    "balance": {
      "algorithm": "roundrobin"
    },
    "name": "be_app"
  }
]

Rename of the defaults resource Jump to heading #

In the beginning of the Data Plane API we wanted to have only one defaults section, and that resided in the /v2/services/haproxy/configuration/defaults resource. That was due to the fact that the name had no meaning in the functionality of the HAProxy instance. With HAproxy 2.4, from keywords were introduced to frontend and backend sections, and that gave more sense to the functionality of having multiple named defaults. Since the defaults resource was returning an object and not a list, to keep backward compatibility we implemented another resource called named_defaults that returned a list of defaults resources that had a name and could be referenced in backend and frontend sections. The defaults resource was kept for backward compatibility as an object. With v3 of the API, we are migrating named_defaults to simply defaults, while breaking the old defaults behavior. Now the defaults resource returns a list instead of an object.

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/defaults

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/defaults

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:03:14 GMT
Content-Length: 277
Connection: close
[
  {
    "client_timeout": 300000,
    "connect_timeout": 10000,
    "dontlognull": "enabled",
    "forwardfor": {
      "enabled": "enabled",
      "except": "127.0.0.0/8"
    },
    "httplog": true,
    "maxconn": 5000,
    "mode": "http",
    "name": "unnamed_defaults_1",
    "redispatch": {
      "enabled": "enabled"
    },
    "retries": 3,
    "server_timeout": 300000
  }
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:03:14 GMT
Content-Length: 277
Connection: close
[
  {
    "client_timeout": 300000,
    "connect_timeout": 10000,
    "dontlognull": "enabled",
    "forwardfor": {
      "enabled": "enabled",
      "except": "127.0.0.0/8"
    },
    "httplog": true,
    "maxconn": 5000,
    "mode": "http",
    "name": "unnamed_defaults_1",
    "redispatch": {
      "enabled": "enabled"
    },
    "retries": 3,
    "server_timeout": 300000
  }
]

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/named_defaults

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/named_defaults

output
bash
HTTP/1.1 404 Not Found
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:05:45 GMT
Content-Length: 93
Connection: close
{
  "code": 404,
  "message": "path /v3/services/haproxy/configuration/named_defaults was not found"
}

output
bash
HTTP/1.1 404 Not Found
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:05:45 GMT
Content-Length: 93
Connection: close
{
  "code": 404,
  "message": "path /v3/services/haproxy/configuration/named_defaults was not found"
}

Moving child resources as nested resources Jump to heading #

In the v2 API we have some resources that represent lines inside a section of HAProxy configuration, making them effectively child resources of the parent section resource. One example is the acl resource that could be a child of a frontend, backend or server resource. In v2 two you could access these resources on the API by identifying them with their index (such as for acl) or name (such as for servers) and specifying their parents by using parent_type and parent_name query-string parameters.

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/acls?parent_name=fe_main&parent_type=frontend

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/acls?parent_name=fe_main&parent_type=frontend

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:20:10 GMT
Content-Length: 476
Connection: close
{
  "_version": 1,
  "data": [
    {
      "acl_name": "url_stats",
      "criterion": "path_beg",
      "index": 0,
      "value": "/hapee-stats"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_beg",
      "index": 1,
      "value": "-i /static /images /javascript /stylesheets"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_end",
      "index": 2,
      "value": "-i .jpg .gif .png .css .js"
    }
  ]
}

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:20:10 GMT
Content-Length: 476
Connection: close
{
  "_version": 1,
  "data": [
    {
      "acl_name": "url_stats",
      "criterion": "path_beg",
      "index": 0,
      "value": "/hapee-stats"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_beg",
      "index": 1,
      "value": "-i /static /images /javascript /stylesheets"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_end",
      "index": 2,
      "value": "-i .jpg .gif .png .css .js"
    }
  ]
}

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/servers?parent_type=backend&parent_name=be_static

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/servers?parent_type=backend&parent_name=be_static

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:21:55 GMT
Content-Length: 166
Connection: close
{
  "_version": 1,
  "data": [
    {
      "check": "enabled",
      "address": "127.0.0.1",
      "name": "static1",
      "port": 4331
    },
    {
      "check": "enabled",
      "address": "127.0.0.1",
      "name": "static2",
      "port": 4331
    }
  ]
}

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:21:55 GMT
Content-Length: 166
Connection: close
{
  "_version": 1,
  "data": [
    {
      "check": "enabled",
      "address": "127.0.0.1",
      "name": "static1",
      "port": 4331
    },
    {
      "check": "enabled",
      "address": "127.0.0.1",
      "name": "static2",
      "port": 4331
    }
  ]
}

Over the years, users found this confusing so now we are redesigning this into having nested resources in the URL. One child resource cannot simultaneously exist in multiple parent resources so it makes more sense to have this as part of the URL of the resource and also helps with many external tools for the future. For example, RBAC rules can be built on top of the new URL hierarchy more easily without having to rely on query-string parameters.

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends/fe_main/acls

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends/fe_main/acls

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:26:44 GMT
Content-Length: 404
Connection: close
[
  {
    "acl_name": "url_stats",
    "criterion": "path_beg",
    "value": "/hapee-stats"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_beg",
    "value": "-i /static /images /javascript /stylesheets"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_end",
    "value": "-i .jpg .gif .png .css .js"
  },
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:26:44 GMT
Content-Length: 404
Connection: close
[
  {
    "acl_name": "url_stats",
    "criterion": "path_beg",
    "value": "/hapee-stats"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_beg",
    "value": "-i /static /images /javascript /stylesheets"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_end",
    "value": "-i .jpg .gif .png .css .js"
  },
]

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/backends/be_static/servers

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/backends/be_static/servers

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:28:02 GMT
Content-Length: 144
Connection: close
[
  {
    "check": "enabled",
    "address": "127.0.0.1",
    "name": "static1",
    "port": 4331
  },
  {
    "check": "enabled",
    "address": "127.0.0.1",
    "name": "static2",
    "port": 4331
  }
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:28:02 GMT
Content-Length: 144
Connection: close
[
  {
    "check": "enabled",
    "address": "127.0.0.1",
    "name": "static1",
    "port": 4331
  },
  {
    "check": "enabled",
    "address": "127.0.0.1",
    "name": "static2",
    "port": 4331
  }
]

Removing explicit index field on index based child resources Jump to heading #

In v2 of the API all index-based resources, which represent lines in a configuration that can be repeated inside a section and are distinguished by the order in which they appear in the section, had an index field that told us the position where this specific resource appeared. For example, the acls resource. For v3 we are removing that field as we find it redundant and it is complicated to maintain when positions change. You can infer the index from the actual position in the list returned, and access it at this position.

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/acls?parent_name=fe_main&parent_type=frontend

nix
GET http://127.0.0.1:8080/v2/services/haproxy/configuration/acls?parent_name=fe_main&parent_type=frontend

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:20:10 GMT
Content-Length: 476
Connection: close
{
  "_version": 1,
  "data": [
    {
      "acl_name": "url_stats",
      "criterion": "path_beg",
      "index": 0,
      "value": "/hapee-stats"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_beg",
      "index": 1,
      "value": "-i /static /images /javascript /stylesheets"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_end",
      "index": 2,
      "value": "-i .jpg .gif .png .css .js"
    }
  ]
}

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:20:10 GMT
Content-Length: 476
Connection: close
{
  "_version": 1,
  "data": [
    {
      "acl_name": "url_stats",
      "criterion": "path_beg",
      "index": 0,
      "value": "/hapee-stats"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_beg",
      "index": 1,
      "value": "-i /static /images /javascript /stylesheets"
    },
    {
      "acl_name": "url_static",
      "criterion": "path_end",
      "index": 2,
      "value": "-i .jpg .gif .png .css .js"
    }
  ]
}

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends/fe_main/acls

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends/fe_main/acls

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:26:44 GMT
Content-Length: 404
Connection: close
[
  {
    "acl_name": "url_stats",
    "criterion": "path_beg",
    "value": "/hapee-stats"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_beg",
    "value": "-i /static /images /javascript /stylesheets"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_end",
    "value": "-i .jpg .gif .png .css .js"
  },
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Wed, 21 Aug 2024 13:26:44 GMT
Content-Length: 404
Connection: close
[
  {
    "acl_name": "url_stats",
    "criterion": "path_beg",
    "value": "/hapee-stats"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_beg",
    "value": "-i /static /images /javascript /stylesheets"
  },
  {
    "acl_name": "url_static",
    "criterion": "path_end",
    "value": "-i .jpg .gif .png .css .js"
  },
]

Remove support for HAProxy in multi-process mode Jump to heading #

HAProxy had a feature that allowed it to run in multi-process mode. When HAProxy 2.0 introducing multi-thread mode, this was discouraged. It was deprecated in 2.3 and in 2.5 it was completely removed. In v2 of the API we supported all the keywords related to multi-process mode, and also supported multiple runtime APIs, one for each process mode, so all /runtime resources worked with multiple process support. In v3 of the API we are removing this support altogether. We removed the following fields from the respective sections:

• bind_process from the backend resource
• process from the bind resource
• bind_process from the defaults resource
• bind_process from the frontend resource
• nbproc from the global resource

In addition to removing these fields, /runtime endpoints received changes.

These changes may constitute a breaking change for your configuration.

Stats endpoint change Jump to heading #

The statistics resource at /v2/services/haproxy/stats/native used to return two layers of arrays, the top layer is stats per process, where then each item contains an array of stats lines per object: server, backend, and frontend.

nix
GET http://127.0.0.1:8080/v2/services/haproxy/stats/native

nix
GET http://127.0.0.1:8080/v2/services/haproxy/stats/native

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:23:41 GMT
Connection: close
Transfer-Encoding: chunked
[
  {
    "runtimeAPI": "/var/run/hapee-2.8/master.sock@1",
    "stats": [
      {
        "name": "fe_main",
        "stats": {
    	...
        },
	  "type": "frontend"
      },
      {
        "name": "be_stats",
        "stats": {
	     ...
        },
        "type": "backend"
      },
      ...
    ]
  },
  ...
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:23:41 GMT
Connection: close
Transfer-Encoding: chunked
[
  {
    "runtimeAPI": "/var/run/hapee-2.8/master.sock@1",
    "stats": [
      {
        "name": "fe_main",
        "stats": {
    	...
        },
	  "type": "frontend"
      },
      {
        "name": "be_stats",
        "stats": {
	     ...
        },
        "type": "backend"
      },
      ...
    ]
  },
  ...
]

Now, as we don’t support multi-process mode, /v3/services/haproxy/stats/native returns an object with only one level of stats arrays per object:

nix
GET http://127.0.0.1:8080/v3/services/haproxy/stats/native

nix
GET http://127.0.0.1:8080/v3/services/haproxy/stats/native

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:23:41 GMT
Connection: close
Transfer-Encoding: chunked
{
  "runtimeAPI": "/var/run/hapee-2.8/master.sock@1",
  "stats": [
    {
   "name": "fe_main",
   "stats": {
	 ...
    },
	"type": "frontend"
    },
    {
      "name": "be_stats",
      "stats": {
	  ...
    },
    "type": "backend"
    },
    ...
  ]
}

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:23:41 GMT
Connection: close
Transfer-Encoding: chunked
{
  "runtimeAPI": "/var/run/hapee-2.8/master.sock@1",
  "stats": [
    {
   "name": "fe_main",
   "stats": {
	 ...
    },
	"type": "frontend"
    },
    {
      "name": "be_stats",
      "stats": {
	  ...
    },
    "type": "backend"
    },
    ...
  ]
}

Process Info endpoint change Jump to heading #

This change is similar to stats info change where we are not supporting multiple processes anymore. In v3 we removed one layer of arrays, which returned process infos per process.

nix
GET http://127.0.0.1:8080/v2/services/haproxy/runtime/info

nix
GET http://127.0.0.1:8080/v2/services/haproxy/runtime/info

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:45:39 GMT
Content-Length: 1122
Connection: close
[
  {
    "info": {
      "active_peers": 0,
      "busy_polling": 0,
      "bytes_out_rate": 0,
      "compress_bps_in": 0,
      "compress_bps_out": 0,
      "compress_bps_rate_lim": 0,
      "conn_rate": 0,
      "conn_rate_limit": 0,
      "connected_peers": 0,
      "cum_conns": 5040,
      "cum_req": 4,
      "cum_ssl_conns": 0,
      "curr_conns": 0,
      "curr_ssl_conns": 0,
      "dropped_logs": 0,
      "failed_resolutions": 0,
      "hard_max_conn": 10000,
      "idle_pct": 100,
      "jobs": 4,
      "listeners": 3,
      "max_conn": 10000,
      "max_conn_rate": 0,
      "max_pipes": 0,
      "max_sess_rate": 0,
      "max_sock": 20075,
      "max_ssl_conns": 0,
      "max_ssl_rate": 0,
      "mem_max_mb": 0,
      "nbthread": 8,
      "node": "test",
      "pid": 2991105,
      "pipes_free": 0,
      "pipes_used": 0,
      "pool_alloc_mb": 0,
      "pool_failed": 0,
      "pool_used_mb": 0,
      "process_num": 1,
      "processes": 1,
      "release_date": "2024-07-26",
      "run_queue": 0,
      "sess_rate": 0,
      "sess_rate_limit": 0,
      "ssl_backend_key_rate": 0,
      "ssl_backend_max_key_rate": 0,
      "ssl_cache_lookups": 0,
      "ssl_cache_misses": 0,
      "ssl_frontend_key_rate": 0,
      "ssl_frontend_max_key_rate": 0,
      "ssl_frontend_session_reuse": 0,
      "ssl_rate": 0,
      "ssl_rate_limit": 0,
      "stopping": 0,
      "tasks": 35,
      "total_bytes_out": 17552,
      "ulimit_n": 20075,
      "unstoppable": 1,
      "uptime": 1680,
      "version": "2.8.0-1.0.0-319.723"
    },
    "runtimeAPI": "/var/run/hapee-2.8/master.sock"
  },
  ...
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:45:39 GMT
Content-Length: 1122
Connection: close
[
  {
    "info": {
      "active_peers": 0,
      "busy_polling": 0,
      "bytes_out_rate": 0,
      "compress_bps_in": 0,
      "compress_bps_out": 0,
      "compress_bps_rate_lim": 0,
      "conn_rate": 0,
      "conn_rate_limit": 0,
      "connected_peers": 0,
      "cum_conns": 5040,
      "cum_req": 4,
      "cum_ssl_conns": 0,
      "curr_conns": 0,
      "curr_ssl_conns": 0,
      "dropped_logs": 0,
      "failed_resolutions": 0,
      "hard_max_conn": 10000,
      "idle_pct": 100,
      "jobs": 4,
      "listeners": 3,
      "max_conn": 10000,
      "max_conn_rate": 0,
      "max_pipes": 0,
      "max_sess_rate": 0,
      "max_sock": 20075,
      "max_ssl_conns": 0,
      "max_ssl_rate": 0,
      "mem_max_mb": 0,
      "nbthread": 8,
      "node": "test",
      "pid": 2991105,
      "pipes_free": 0,
      "pipes_used": 0,
      "pool_alloc_mb": 0,
      "pool_failed": 0,
      "pool_used_mb": 0,
      "process_num": 1,
      "processes": 1,
      "release_date": "2024-07-26",
      "run_queue": 0,
      "sess_rate": 0,
      "sess_rate_limit": 0,
      "ssl_backend_key_rate": 0,
      "ssl_backend_max_key_rate": 0,
      "ssl_cache_lookups": 0,
      "ssl_cache_misses": 0,
      "ssl_frontend_key_rate": 0,
      "ssl_frontend_max_key_rate": 0,
      "ssl_frontend_session_reuse": 0,
      "ssl_rate": 0,
      "ssl_rate_limit": 0,
      "stopping": 0,
      "tasks": 35,
      "total_bytes_out": 17552,
      "ulimit_n": 20075,
      "unstoppable": 1,
      "uptime": 1680,
      "version": "2.8.0-1.0.0-319.723"
    },
    "runtimeAPI": "/var/run/hapee-2.8/master.sock"
  },
  ...
]

nix
GET http://127.0.0.1:8080/v3/services/haproxy/runtime/info

nix
GET http://127.0.0.1:8080/v3/services/haproxy/runtime/info

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:45:39 GMT
Content-Length: 1122
Connection: close
{
  "info": {
    "active_peers": 0,
    "busy_polling": 0,
    "bytes_out_rate": 0,
    "compress_bps_in": 0,
    "compress_bps_out": 0,
    "compress_bps_rate_lim": 0,
    "conn_rate": 0,
    "conn_rate_limit": 0,
    "connected_peers": 0,
    "cum_conns": 5040,
    "cum_req": 4,
    "cum_ssl_conns": 0,
    "curr_conns": 0,
    "curr_ssl_conns": 0,
    "dropped_logs": 0,
    "failed_resolutions": 0,
    "hard_max_conn": 10000,
    "idle_pct": 100,
    "jobs": 4,
    "listeners": 3,
    "max_conn": 10000,
    "max_conn_rate": 0,
    "max_pipes": 0,
    "max_sess_rate": 0,
    "max_sock": 20075,
    "max_ssl_conns": 0,
    "max_ssl_rate": 0,
    "mem_max_mb": 0,
    "nbthread": 8,
    "node": "test",
    "pid": 2991105,
    "pipes_free": 0,
    "pipes_used": 0,
    "pool_alloc_mb": 0,
    "pool_failed": 0,
    "pool_used_mb": 0,
    "process_num": 1,
    "processes": 1,
    "release_date": "2024-07-26",
    "run_queue": 0,
    "sess_rate": 0,
    "sess_rate_limit": 0,
    "ssl_backend_key_rate": 0,
    "ssl_backend_max_key_rate": 0,
    "ssl_cache_lookups": 0,
    "ssl_cache_misses": 0,
    "ssl_frontend_key_rate": 0,
    "ssl_frontend_max_key_rate": 0,
    "ssl_frontend_session_reuse": 0,
    "ssl_rate": 0,
    "ssl_rate_limit": 0,
    "stopping": 0,
    "tasks": 35,
    "total_bytes_out": 17552,
    "ulimit_n": 20075,
    "unstoppable": 1,
    "uptime": 1680,
    "version": "2.8.0-1.0.0-319.723"
  },
  "runtimeAPI": "/var/run/hapee-2.8/master.sock"
}

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 08:45:39 GMT
Content-Length: 1122
Connection: close
{
  "info": {
    "active_peers": 0,
    "busy_polling": 0,
    "bytes_out_rate": 0,
    "compress_bps_in": 0,
    "compress_bps_out": 0,
    "compress_bps_rate_lim": 0,
    "conn_rate": 0,
    "conn_rate_limit": 0,
    "connected_peers": 0,
    "cum_conns": 5040,
    "cum_req": 4,
    "cum_ssl_conns": 0,
    "curr_conns": 0,
    "curr_ssl_conns": 0,
    "dropped_logs": 0,
    "failed_resolutions": 0,
    "hard_max_conn": 10000,
    "idle_pct": 100,
    "jobs": 4,
    "listeners": 3,
    "max_conn": 10000,
    "max_conn_rate": 0,
    "max_pipes": 0,
    "max_sess_rate": 0,
    "max_sock": 20075,
    "max_ssl_conns": 0,
    "max_ssl_rate": 0,
    "mem_max_mb": 0,
    "nbthread": 8,
    "node": "test",
    "pid": 2991105,
    "pipes_free": 0,
    "pipes_used": 0,
    "pool_alloc_mb": 0,
    "pool_failed": 0,
    "pool_used_mb": 0,
    "process_num": 1,
    "processes": 1,
    "release_date": "2024-07-26",
    "run_queue": 0,
    "sess_rate": 0,
    "sess_rate_limit": 0,
    "ssl_backend_key_rate": 0,
    "ssl_backend_max_key_rate": 0,
    "ssl_cache_lookups": 0,
    "ssl_cache_misses": 0,
    "ssl_frontend_key_rate": 0,
    "ssl_frontend_max_key_rate": 0,
    "ssl_frontend_session_reuse": 0,
    "ssl_rate": 0,
    "ssl_rate_limit": 0,
    "stopping": 0,
    "tasks": 35,
    "total_bytes_out": 17552,
    "ulimit_n": 20075,
    "unstoppable": 1,
    "uptime": 1680,
    "version": "2.8.0-1.0.0-319.723"
  },
  "runtimeAPI": "/var/run/hapee-2.8/master.sock"
}

Runtime Stick Table change Jump to heading #

This change affected /v2/services/haproxy/runtime/stick_table in a way that the process field is removed from the responses, which used to return data from the process from which we were fetching the stick table information. In addition to this, the query-string parameter process was removed for both the /v2/services/haproxy/runtime/stick_tables and /v2/services/haproxy/runtime/stick_table_entries resources.

Reorganize Global resource fields Jump to heading #

We found a need to logically reorganize the global fields. Global is our biggest resource if you measure it by number of fields, and it was mostly flat structured. To improve readability of this resource on the API, we decided to reorganize the fields into logical nested objects.

The API adheres to the field naming in HAProxy and some documentation hints in the HAProxy documentation, while trying to keep only one level of nested objects so as not to unnecessarily make the object more complex. There are still some fields left in the root level of the object, and some options are split into debug_options and performance_options. Also, we’ve added an ssl_options nested object to cover all SSL options, in addition to adding the tune_options to cover all tune.* options. There are additional nested objects like tune_ssl_options, tune_vars_options, tune_zlib_options, etc. which are all derived from tune.*.* options. Also following the dot notation in the configuration, options now exist under http_client_options, fifty_one_degrees_options, etc. For a full, detailed specification that describes the fields please see the global model specification defined in the Swagger definition.

Further API Improvements Jump to heading #

This section describes other API improvements.

Fetching a parent resource with all children embedded Jump to heading #

We’ve found that our granular approach to creating resources on the API can be helpful, but also in some cases complex. In v3 we’ve added an extra query-string parameter full_section on all the section resources (such as frontend and backend resources) which allows you to fetch, create, or replace a complete section with all of its child resources.

For example, if you have a frontend section like this in your configuration:

haproxy
frontend  fe_main
  bind 127.0.0.1:5000     # direct HTTP access
  # bind *:443 ssl ....
  
  acl url_stats      path_beg    /hapee-stats
  acl url_static     path_beg    -i /static /images /javascript /stylesheets
  acl url_static     path_end    -i .jpg .gif .png .css .js
  
  # redirect to the stats page if everything is dead
  acl be_app_ok      nbsrv(be_app) gt 0
  acl be_static_ok   nbsrv(be_static) gt 0
  http-request redirect location /hapee-stats if !be_app_ok !be_static_ok !url_stats
  
  # serve the stats page if the URL starts with /hapee-stats
  use_backend        be_stats    if url_stats
  
  # static contents go to the static backend
  use_backend        be_static   if url_static
  
  # the rest is the application
  default_backend    be_app

haproxy
frontend  fe_main
  bind 127.0.0.1:5000     # direct HTTP access
  # bind *:443 ssl ....
  
  acl url_stats      path_beg    /hapee-stats
  acl url_static     path_beg    -i /static /images /javascript /stylesheets
  acl url_static     path_end    -i .jpg .gif .png .css .js
  
  # redirect to the stats page if everything is dead
  acl be_app_ok      nbsrv(be_app) gt 0
  acl be_static_ok   nbsrv(be_static) gt 0
  http-request redirect location /hapee-stats if !be_app_ok !be_static_ok !url_stats
  
  # serve the stats page if the URL starts with /hapee-stats
  use_backend        be_stats    if url_stats
  
  # static contents go to the static backend
  use_backend        be_static   if url_static
  
  # the rest is the application
  default_backend    be_app

You can fetch it without the parameter:

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends

output
bash
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 12:48:31 GMT
Content-Length: 48
Connection: close
[
  {
    "default_backend": "be_app",
    "name": "fe_main"
  }
]

output
bash
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 12:48:31 GMT
Content-Length: 48
Connection: close
[
  {
    "default_backend": "be_app",
    "name": "fe_main"
  }
]

As you can see, it only returns the default_backend configuration keyword and a name. To fetch all the children you would need to call the resources for acls, http_request_rules, binds and backend_switching_rules. But with the new full_section=true query-string parameter, you can fetch the complete configuration of the frontend section structured as JSON like this:

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends?full_section=true

nix
GET http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends?full_section=true

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 12:51:12 GMT
Content-Length: 858
Connection: close
[
  {
    "default_backend": "be_app",
    "name": "fe_main",
    "acl_list": [
      {
        "acl_name": "url_stats",
        "criterion": "path_beg",
        "value": "/hapee-stats"
      },
      {
        "acl_name": "url_static",
        "criterion": "path_beg",
        "value": "-i /static /images /javascript /stylesheets"
      },
      {
        "acl_name": "url_static",
        "criterion": "path_end",
        "value": "-i .jpg .gif .png .css .js"
      },
      {
        "acl_name": "be_app_ok",
        "criterion": "nbsrv(be_app)",
        "value": "gt 0"
      },
      {
        "acl_name": "be_static_ok",
        "criterion": "nbsrv(be_static)",
        "value": "gt 0"
      }
    ],
    "backend_switching_rule_list": [
      {
        "cond": "if",
        "cond_test": "url_stats",
        "name": "be_stats"
      },
      {
        "cond": "if",
        "cond_test": "url_static",
        "name": "be_static"
      }
    ],
    "http_request_rule_list": [
      {
        "cond": "if",
        "cond_test": "!be_app_ok !be_static_ok !url_stats",
        "redir_type": "location",
        "redir_value": "/hapee-stats",
        "type": "redirect"
      }
    ],
    "binds": {
      "127.0.0.1:5000": {
        "name": "127.0.0.1:5000",
        "address": "127.0.0.1",
        "port": 5000
      }
    }
  }
]

output
bash
HTTP/1.1 200 OK
Configuration-Version: 1
Content-Type: application/json
Vary: Origin
Date: Thu, 22 Aug 2024 12:51:12 GMT
Content-Length: 858
Connection: close
[
  {
    "default_backend": "be_app",
    "name": "fe_main",
    "acl_list": [
      {
        "acl_name": "url_stats",
        "criterion": "path_beg",
        "value": "/hapee-stats"
      },
      {
        "acl_name": "url_static",
        "criterion": "path_beg",
        "value": "-i /static /images /javascript /stylesheets"
      },
      {
        "acl_name": "url_static",
        "criterion": "path_end",
        "value": "-i .jpg .gif .png .css .js"
      },
      {
        "acl_name": "be_app_ok",
        "criterion": "nbsrv(be_app)",
        "value": "gt 0"
      },
      {
        "acl_name": "be_static_ok",
        "criterion": "nbsrv(be_static)",
        "value": "gt 0"
      }
    ],
    "backend_switching_rule_list": [
      {
        "cond": "if",
        "cond_test": "url_stats",
        "name": "be_stats"
      },
      {
        "cond": "if",
        "cond_test": "url_static",
        "name": "be_static"
      }
    ],
    "http_request_rule_list": [
      {
        "cond": "if",
        "cond_test": "!be_app_ok !be_static_ok !url_stats",
        "redir_type": "location",
        "redir_value": "/hapee-stats",
        "type": "redirect"
      }
    ],
    "binds": {
      "127.0.0.1:5000": {
        "name": "127.0.0.1:5000",
        "address": "127.0.0.1",
        "port": 5000
      }
    }
  }
]

This query-string parameter works on both PUT and POST endpoints, meaning that you can now edit or create the whole parent section in just one API call. An important thing to note here is that we have two different types of child resources. One is the named child resource (in this example the bind resource). These are all the resources that can be uniquely identified by their name inside a parent section, and then that name is used as a key and those resources are represented as a map. The other child resource type is index-based as its unique identifier is the position in the configuration where the directive appears. It is represented as an index of a zero-based array. Those have a suffix _list.

This change helps you read and change the HAProxy configuration in less API calls.

Adding PUT operation on index based resources Jump to heading #

While using the v2 API, if you wanted to make automated changes on a list of index based resources, it proved to be complex for a multitude of reasons. First, you had to reconcile all the index fields in each resource, and for each update it was hard to find the resource if something was deleted or added as you had to reconcile the indexes of each resource. Also, if the list was bigger with multiple changes, you were stuck with making multiple API calls.

With v3, we added PUT operation on the list endpoints, which allows you to atomically replace the complete list of resources. Say you have a list of ACLs in your frontend as follow:

haproxy
acl url_stats      path_beg    /hapee-stats
acl url_static     path_beg    -i /static /images /javascript /stylesheets
acl url_static     path_end    -i .jpg .gif .png .css .js
acl be_app_ok      nbsrv(be_app) gt 0
acl be_static_ok   nbsrv(be_static) gt 0

haproxy
acl url_stats      path_beg    /hapee-stats
acl url_static     path_beg    -i /static /images /javascript /stylesheets
acl url_static     path_end    -i .jpg .gif .png .css .js
acl be_app_ok      nbsrv(be_app) gt 0
acl be_static_ok   nbsrv(be_static) gt 0

And through the API you want to delete the first one, edit the second one to not have /images, delete the be_app_ok ACL and add a new one in its place called be_app_down. In v2 you would have to call one DELETE, then a PUT on the 0 index, as it’s index would have now changed, DELETE the 2 index, and finally POST on the 2 index. With v3 you can simply make the following call:

nix
PUT http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends/fe_main/acls?version=1

nix
PUT http://127.0.0.1:8080/v3/services/haproxy/configuration/frontends/fe_main/acls?version=1

output
bash
[
 {
   "acl_name": "url_static",
   "criterion": "path_beg",
   "value": "-i /static /javascript /stylesheets"
 },
 {
   "acl_name": "url_static",
   "criterion": "path_end",
   "value": "-i .jpg .gif .png .css .js"
 },
 {
   "acl_name": "be_app_down",
   "criterion": "nbsrv(be_app)",
   "value": "lt 0"
 },
 {
   "acl_name": "be_static_ok",
   "criterion": "nbsrv(be_static)",
   "value": "gt 0"
 }
]

output
bash
[
 {
   "acl_name": "url_static",
   "criterion": "path_beg",
   "value": "-i /static /javascript /stylesheets"
 },
 {
   "acl_name": "url_static",
   "criterion": "path_end",
   "value": "-i .jpg .gif .png .css .js"
 },
 {
   "acl_name": "be_app_down",
   "criterion": "nbsrv(be_app)",
   "value": "lt 0"
 },
 {
   "acl_name": "be_static_ok",
   "criterion": "nbsrv(be_static)",
   "value": "gt 0"
 }
]

And with this replace the complete ACLs list in frontend fe_main.

Handle timeout options with preferred units Jump to heading #

In the API we handle timeout options as integers, as we want to represent numeric values as numbers wherever possible. However, HAProxy supports strings by accepting time unit suffixes and in v2 we would recalculate the user’s input values to the default unit (in most cases ms) and write it as an integer in the configuration file. This proved challenging for anyone using and reading the raw configuration file, as it changed their preferred inputs in the file, making it less readable.

In v3 we are introducing a new option in the Data Plane API configuration file called preferred_time_suffix where you can specify one of the options:

• nearest
• none
• ms
• s
• m
• h
• d

Data Plane API will then honor your preferred time suffix. By default it’s nearest, which means that it will be calculated to the smallest possible value with the according suffix. When using none, your timeout keywords in the configuration file will be written without the suffix using the default value for the respective keyword.

For further information, we’ve added some documentation to our Open API specification by annotating such fields with "x-duration": true and x-default-unit which indicates the default time unit for that specific field.

Keep Data Plane API config file unchanged Jump to heading #

Since the Data Plane API doesn’t have its own storage, it wrote its state to the configuration file. Notably, it wrote service discovery and users state there. This is problematic when the Data Plane API is used in combinations with automation software like Ansible, Puppet, etc.

In v3, Data Plane API will no longer write to its own configuration file. We are introducing something called dataplane_storage_dir, where Data Plane API’s state will be stored, for now in specific JSON files for each feature. So now, you’ll find users.json, service_discovery/consul.json and service_discovery/aws.json. It will all be migrated on the first start, and used from there in the future. We opted for using plain JSON files so it’s easier for users to read and debug those files.

Deprecated features Jump to heading #

Some deprecated fields and resources have been removed.

During the development of Data Plane API, we deprecated some fields and resources in order to change or improve the behavior and support configuration keywords. Now we are removing those deprecated keywords. Here is an extensive list of removed deprecations in v3:

• Global section: Removed deprecated tune_ssl_default_dh_param that can now be found in tune_ssl_options as default_dh_param.

• Backend section: Removed deprecated http-check. You can now specify multiple http_check resources on /v3/services/haproxy/configuration/backends/{parent_name}/http_checks. Removed http-keep-alive, http-server-close, and httpclose fields as they are mutually exclusive and can be set using http_connection_mode field.

• HTTP Request/Response Rules and TCP Request Rule resource: Removed deprecated track_sc<stick_counter> fields. In v2 we supported only 0, 1 and 2 stick counter indexes by having those hard-coded in the field names. Now with this change we added the track_sc_stick_counter field, which specifies the stick counter index and can be more than 2 if configured using the tune.stick_counters keyword in global (it defaults to 3), which was introduced in HAProxy 2.8.

• Service Discovery resources: Removed the deprecated service-whitelist and service-blacklist, which were replaced by service_allowlist and service_denylist respectively.