Troubleshooting
Common questions
This guide isn’t intended to solve problems directly, but rather to lead you from a general problem to a specific one to be solved. This isn’t to replace the ability to ask our support team for help directly, just to give a direction to the questions, especially if you aren’t fully familiar with HAProxy Enterprise.
You receive a connection timeout response Jump to heading
Possible causes include:
-
HAProxy Enterprise has reached its global
maxconn
value of connections.Check the HAProxy Enterprise Stats page or Real Time Dashboard and see if the frontend section’s
cur
column, which indicates the number of current connections, is equal to themax
column (or the same as the global settings in the top left of the page). This would indicate that the load balancer or its frontend are at the maximum connection limit (maxconn
in the configuration) and that that number needs to be raised (or the reason it is at the limit otherwise investigated). -
HAProxy Enterprise can’t be reached on the network
If the logs don’t show anything, try using
tcpdump
on the load balancer server to see if it is receiving any SYN packets at all. Make this filter as specific as needed to keep it quiet enough to read. For example,host 192.168.122.14 and port 443
to restrict to a specific inbound IP address.nixtcpdump -vv -i any "port 80"nixtcpdump -vv -i any "port 80" -
Other causes can be more complicated to troubleshoot, but may be discovered by checking the access logs in
/var/log/hapee-3.0/
.
You receive a connection refused response Jump to heading
Possible causes include:
- HAProxy Enterprise is not running. Check the service with
sudo systemctl status hapee-3.0-lb
. - The connection reached the wrong server. Check whether traffic was received on another server.
You receive an empty response Jump to heading
- Check the Stats page or Real Time Dashboard to see if the backend application may be down.
- Check the HAProxy Enterprise logs to see the termination state code, which shows the reason that the connection was aborted.
You receive a 503 Service Unavailable response Jump to heading
-
Check the Stats page or Real Time Dashboard to see if the backend application may be marked as down.
On the Stats page, check for any backends with all servers colored red. If you see any, you can mouse over the
LastChk
column value with its dotted underline to get a more specific reason for the failure. Alternatively, you can grep foris DOWN
in the access or admin logs to find a message with the same information. You can also grep for503
. -
If the logs show the backend name being the same as the frontend name, check if the frontend has a
default_backend
line. If it doesn’t, it’s possible that none of theuse_backend
lines matched the request. Requests that fall out of a frontend without matching ause_backend
ordefault_backend
rule return a 503 response.Once you’ve found the line in the logs, look at the termination state code to see why the request failed.
You receive a 504 Gateway Timeout response Jump to heading
- Check the access logs to see why HAProxy Enterprise timed out while waiting for the server to respond. If
curl
works fine but problems still exist, the next step is to find out what the difference is. For example, SSL cipher/protocol mismatches.
HAProxy Enterprise fails to restart or reload Jump to heading
-
The load balancer configuration may have a syntax error. Check the status of the service:
nixsystemctl status hapee-3.0-lbnixsystemctl status hapee-3.0-lbOr display the recent log entries:
nixtail -n50 /var/log/messagesnixtail -n50 /var/log/messagesLook for lines that begin with
[ALERT]
, such as:nixMar 31 13:04:09 rhel8vm hapee-lb[3055]: [ALERT] 090/130409 (3055) : parsing [/etc/hapee-3.0/hapee-lb.cfg:123] : unknown keyword 'hxtp-request' in 'frontend' sectionnixMar 31 13:04:09 rhel8vm hapee-lb[3055]: [ALERT] 090/130409 (3055) : parsing [/etc/hapee-3.0/hapee-lb.cfg:123] : unknown keyword 'hxtp-request' in 'frontend' sectionThere may be multiple alert lines, but the first one is the most urgent.
-
If the error mentions not being able to bind to a socket and this is a passive node in a VRRP cluster and HAProxy Enterprise is configured to bind to a specific IP address, check that the sysctl option
net.ipv4.ip_nonlocal_bind
is set to 1. Usually, you can do this by uncommenting the line in the sysctl file HAProxy Enterprise ships with in/etc/sysctl.d/30-hapee-3.0.conf
.
Do you have any suggestions on how we can improve the content of this page?