Observability

Send HAProxy ALOHA logs to an external syslog server

HAProxy ALOHA does not permanently store logs. It stores them only in memory and not on the filesystem, making them available for viewing for a limited amount of time via the Logs tab. For long-term storage of logs, deploy a remote syslog server and configure HAProxy ALOHA to ship logs to it.

HAProxy ALOHA generates several types of logs, each of which can be collected and sent to a separate or a shared syslog server. The types are defined in the table below, where each has a unique name that you will use to configure it.

Name Description
system Major operating system events
events Load balancer events
console Administration Web UI events
traffic Traffic traversing all HAProxy ALOHA frontends

Info

  • Select the Logs tab to view a limited history of in-memory logs.
  • You can also log the traffic that traverses a specific HAProxy ALOHA frontend.

Syslog

Configure the syslog server Jump to heading

Configure a remote syslog server to receive log entries. This server should be an additional Linux server external to your HAProxy ALOHA instance whose firewall and other network settings allow incoming log traffic from your HAProxy ALOHA instance. On this Linux server:

  1. Install a syslog server such as rsyslog.

    Example:

    nix
    sudo apt install rsyslog
    nix
    sudo apt install rsyslog
  2. Create a file named /etc/rsyslog.d/10-aloha.conf with the directives below.

    Here, we configure rsyslog to listen on all IP addresses at port 514. Store incoming log messages in the file /var/log/aloha.log when they come from the HAProxy ALOHA IP address.

    /etc/rsyslog.d/10-aloha.conf
    text
    $ModLoad imudp
    $UDPServerRun 514
    $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
    if $fromhost-ip=='172.16.24.237' then /var/log/aloha.log
    /etc/rsyslog.d/10-aloha.conf
    text
    $ModLoad imudp
    $UDPServerRun 514
    $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
    if $fromhost-ip=='172.16.24.237' then /var/log/aloha.log

    The directives are as follows:

    Directive Description
    $ModLoad imudp Receive logs over UDP.
    $UDPServerRun 514 Start on the specified port.
    $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat Use the traditional syslog format.
    if $fromhost-ip==‘172.16.24.237’ then /var/log/aloha.log Store incoming log messages in the file /var/log/aloha.log when they come from the HAProxy ALOHA IP address. Replace 172.16.24.237 with your own IP address. You can specify several of these directives, or use startswith to match a range of IPs.
  3. Restart the rsyslog server.

    nix
    sudo systemctl restart rsyslog
    nix
    sudo systemctl restart rsyslog

Configure load balancers Jump to heading

The load balancers do not need to be configured to send syslog messages.

  • The layer 4 (LVS) load balancer sends all messages to syslog by default. This behavior cannot be modified.
  • The layer 7 (HAProxy) load balancer sends all messages to syslog by default. This behavior can be modified by editing the configuration file in the LB Layer7 tab.

Configure layer 7 (HAProxy) load balancer Jump to heading

The layer 7 (HAProxy) load balancer provides these global log settings, which cannot be modified:

haproxy
global
log 127.0.0.2 local0 # log traffic events with facility local0
log 127.0.0.4 local1 notice # log important events with facility local1
haproxy
global
log 127.0.0.2 local0 # log traffic events with facility local0
log 127.0.0.4 local1 notice # log important events with facility local1

The default frontend and backend sections include the log global directive, so all messages are sent to syslog. You can modify any frontend, backend, and defaults sections as desired to customize logging.

  1. Click the LB Layer7 tab.

  2. In the sections where you want to enable or change logging, add a log directive.

    Example: In frontend fe_main, log all events as specified by global log directives:

    haproxy
    frontend fe_main
    log global
    haproxy
    frontend fe_main
    log global
  3. From the Services tab, reload the haproxy service.

  4. To make the configuration changes persist after a reboot, go to the Setup tab and click Save within the Configuration section.

Use cases Jump to heading

You can customize the syslog configuration for a variety of use cases.

Log operating system events Jump to heading

Configure the system log type to send major HAProxy ALOHA operating system events, such as kernel errors, to an external syslog server.

  1. In the Services tab, click syslog setup.

    Syslog Setup

  2. In the service syslog system section, specify the IP address and port of the destination Syslog server. There can be only one server in the section.

    Example:

    Send operating system events to a syslog server listening at 172.16.24.236 on UDP port 514.

    ruby
    service syslog system
    server 172.16.24.236:514
    ruby
    service syslog system
    server 172.16.24.236:514
  3. Restart the syslog service.

    Syslog Restart

    The Message 7010: Last action returns success message displays.

  4. To make the configuration changes persist after a reboot, go to the Setup tab and click Save within the Configuration section.

Log load balancer events Jump to heading

Configure the events log type to send errors related to starting/stopping the load balancers, and related errors or warnings, to an external syslog server.

  1. In the service syslog events section, specify the IP address and port of the destination syslog server. There can be only one server in the section.

    Example:

    Send load balancer errors to a syslog server listening at 172.16.24.236 on UDP port 514.

    ruby
    service syslog events
    server 172.16.24.236:514
    ruby
    service syslog events
    server 172.16.24.236:514
  2. Restart the syslog service.

  3. To make the configuration changes persist after a reboot, go to the Setup tab and click Save within the Configuration section.

Log administration events Jump to heading

Configure the console log type to send events such as logins to the HAProxy ALOHA command-line interface to an external syslog server.

  1. In the service syslog console section, specify the IP address and port of the destination syslog server. There can be only one server in the section.

    Below, we send login events to a syslog server listening at 172.16.24.236 on UDP port 514.

    ruby
    service syslog console
    server 172.16.24.236:514
    ruby
    service syslog console
    server 172.16.24.236:514
  2. Restart the syslog service.

  3. To make the configuration changes persist after a reboot, go to the Setup tab and click Save within the Configuration section.

Log traffic from all frontends Jump to heading

Configure the traffic log type to send traffic logs for all frontends to an external syslog server. This configuration logs all load balancer traffic.

  1. In the service syslog traffic section, specify the IP address and port of the destination syslog server. There can be only one server in the section.

    Below, we send traffic logs to a syslog server listening at 172.16.24.236 on UDP port 514.

    ruby
    service syslog traffic
    server 172.16.24.236:514
    ruby
    service syslog traffic
    server 172.16.24.236:514
  2. Restart the syslog service.

  3. To make the configuration changes persist after a reboot, go to the Setup tab and click Save within the Configuration section.

Log traffic from a specific layer 7 section Jump to heading

You can log traffic that traverses a specific HAProxy ALOHA frontend or backend. This configuration logs only layer 7 load balancer traffic for sections configured as described here.

  1. On the remote rsyslog server, edit the /etc/rsyslog.d/10-aloha.conf file.

    Below, we capture messages from syslog facilities local0 and local1, and write them to the frontend1-traffic.log and frontend2-traffic.log files.

    text
    local0.* /var/log/frontend1-traffic.log
    local1.* /var/log/frontend2-traffic.log
    text
    local0.* /var/log/frontend1-traffic.log
    local1.* /var/log/frontend2-traffic.log
  2. On HAProxy ALOHA:

    • Click the LB Layer7 tab.

    • Add the following directive to a frontend, backend, or defaults section:

      text
      log <syslog server IP address>:<port> <facility>
      text
      log <syslog server IP address>:<port> <facility>

      Example: For events occurring on frontend webservice, send log messages to facility local0 to an rsyslog server listening at 172.16.24.236 on UDP port 514.

      haproxy
      frontend webservice
      bind :80
      mode http
      log 172.16.24.236:514 local0
      option httplog
      default_backend webfarm
      haproxy
      frontend webservice
      bind :80
      mode http
      log 172.16.24.236:514 local0
      option httplog
      default_backend webfarm
    • To make the configuration changes persist after a reboot, go to the Setup tab and click Save within the Configuration section.

Test the setup Jump to heading

  1. Make a web request either to:

    • the HAProxy ALOHA Web UI
    • a HAProxy ALOHA frontend
    nix
    curl http://172.16.24.237:8080
    nix
    curl http://172.16.24.237:8080
  2. Inspect the logs on your rsyslog server.

    nix
    sudo less /var/log/aloha.log
    nix
    sudo less /var/log/aloha.log
    output
    text
    Jan 13 11:12:58 ALOHA1 alohactl2[15685] ALOHA1# /opt/bin/alohactl2 -S root l7_dump
    Jan 13 11:12:58 ALOHA1 alohactl2[15722] ALOHA1# /opt/bin/alohactl2 -S root l4_dump
    Jan 13 11:13:04 ALOHA1 alohactl2[15859] ALOHA1# /opt/bin/alohactl2 -S root l7_dump
    Jan 13 11:52:27 172.16.24.237 haproxy[9522]: 172.29.1.90:46714 [13/Jan/2022:11:52:27.745] webservice webfarm/websrv1 0/0/0/1/1 200 818 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    output
    text
    Jan 13 11:12:58 ALOHA1 alohactl2[15685] ALOHA1# /opt/bin/alohactl2 -S root l7_dump
    Jan 13 11:12:58 ALOHA1 alohactl2[15722] ALOHA1# /opt/bin/alohactl2 -S root l4_dump
    Jan 13 11:13:04 ALOHA1 alohactl2[15859] ALOHA1# /opt/bin/alohactl2 -S root l7_dump
    Jan 13 11:52:27 172.16.24.237 haproxy[9522]: 172.29.1.90:46714 [13/Jan/2022:11:52:27.745] webservice webfarm/websrv1 0/0/0/1/1 200 818 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    nix
    sudo less /var/log/frontend1-traffic.log
    nix
    sudo less /var/log/frontend1-traffic.log
    output
    text
    Jan 13 14:09:38 172.16.24.237 haproxy[18201]: 172.29.1.90:40710 [13/Jan/2022:14:09:38.751] webservice webfarm/websrv1 0/0/0/1/1 200 818 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    Jan 13 14:23:09 172.16.24.237 haproxy[18201]: 172.29.1.90:45748 [13/Jan/2022:14:23:09.407] webservice webfarm/websrv1 0/0/0/1/1 404 304 - - --NI 1/1/0/0/0 0/0 "GET /8080 HTTP/1.1"
    Jan 13 14:23:50 172.16.24.237 haproxy[18201]: Proxy webservice stopped (cumulated conns: FE: 2, BE: 0).
    Jan 13 14:25:21 172.16.24.237 haproxy[19247]: 172.29.1.90:37120 [13/Jan/2022:14:25:21.318] webservice webfarm/websrv1 0/0/0/0/0 200 602 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    Jan 13 14:25:21 172.16.24.237 haproxy[19247]: 172.29.1.90:37120 [13/Jan/2022:14:25:21.548] webservice webfarm/websrv1 0/0/0/0/0 404 351 - - --VN 1/1/0/0/0 0/0 "GET /favicon.ico HTTP/1.1"
    Jan 13 14:25:37 172.16.24.237 haproxy[19247]: 172.29.1.90:37224 [13/Jan/2022:14:25:37.052] webservice webfarm/websrv1 0/0/0/0/0 200 818 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    output
    text
    Jan 13 14:09:38 172.16.24.237 haproxy[18201]: 172.29.1.90:40710 [13/Jan/2022:14:09:38.751] webservice webfarm/websrv1 0/0/0/1/1 200 818 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    Jan 13 14:23:09 172.16.24.237 haproxy[18201]: 172.29.1.90:45748 [13/Jan/2022:14:23:09.407] webservice webfarm/websrv1 0/0/0/1/1 404 304 - - --NI 1/1/0/0/0 0/0 "GET /8080 HTTP/1.1"
    Jan 13 14:23:50 172.16.24.237 haproxy[18201]: Proxy webservice stopped (cumulated conns: FE: 2, BE: 0).
    Jan 13 14:25:21 172.16.24.237 haproxy[19247]: 172.29.1.90:37120 [13/Jan/2022:14:25:21.318] webservice webfarm/websrv1 0/0/0/0/0 200 602 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"
    Jan 13 14:25:21 172.16.24.237 haproxy[19247]: 172.29.1.90:37120 [13/Jan/2022:14:25:21.548] webservice webfarm/websrv1 0/0/0/0/0 404 351 - - --VN 1/1/0/0/0 0/0 "GET /favicon.ico HTTP/1.1"
    Jan 13 14:25:37 172.16.24.237 haproxy[19247]: 172.29.1.90:37224 [13/Jan/2022:14:25:37.052] webservice webfarm/websrv1 0/0/0/0/0 200 818 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"

Syslog service reference Jump to heading

The syslog service in the Services tab supports the following configuration directives:

Directive Description
console_level <level> Sets the maximum syslog severity level to send to the console.
forward_timestamp (version 13.5 and 14.0) Adds an RFC 3164 header to log messages if it is missing (which includes a timestamp and hostname). The log’s raw message changes from:

<134>haproxy[26940]: Connect from 192.168.68.117:60749 to 192.168.68.124:80 (web/TCP)

to:

<134>Jan 24 16:49:40 ALOHA1 haproxy[26940]: Connect from 192.168.68.117:60749 to 192.168.68.124:80 (web/TCP)
keyid <key> An identifier to use for a second syslog server.
listen <local IP[:port]> Collect UDP log messages from the given local IP address and optional port.
listen_kernel Collect kernel messages.
[no] listen_unix Collect (or do not collect if prefixed with no) messages from /dev/log.
output "buffer"|<filename> Records log messages to either a ring buffer or to a file.
rotate <number> The number of log files to keep before rotating them.
server <remote IP:port> The IP address and port of a remote syslog server that will receive log messages.
size <kb> The maximum size in kilobytes of the buffer or file when output "buffer" or output <filename> is set.

Do you have any suggestions on how we can improve the content of this page?