Failed to run haproxy on CentOS7 - corrupted haproxy.cfg and haproxy.service

[krzotr]

Hi,

I tried to run HAProxy on CentOS7 with the default configuration (build, install, run). I think the problem is global for all OS. What I see. haproxy.cfg file is corrupted (fatal error),haproxy.service is corrupted, and something is wrong with -W and -S. In general cannot start HAProxy after build and install.

Details:

• corrupted haproxy.cfg file:
  • ca-base, crt-base, ssl-default-bind-ciphers - needs to be in global section
  • stats timeout

  [WARNING]  (15411) : config : missing timeouts for proxy 'stats'.
     | While not properly invalid, you will certainly encounter various problems
     | with such a configuration. To fix this, please ensure that all following
     | timeouts are set to a non-zero value: 'client', 'connect', 'server'.
  

  • path to socket - directory haproxy does not exist stats socket /run/haproxy/haproxy.sock mode 600 level admin
• haproxy.service
  • Cannot start service

  [ALERT]    (15643) : config : a master CLI socket was defined, but master-worker mode (-W) is not enabled.
  

• systemctl start haproxy shows

  Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.
  

Steps to reproduce:

# yum -y groupinstall 'Development Tools'
# cd /opt
# git clone https://github.com/philyuchkoff/HAProxy-2-RPM-builder.git
# cd ./HAProxy-2-RPM-builder
# make
# rpm -i ./rpmbuild/RPMS/x86_64/haproxy-2.9.7-1.el7.x86_64.rpm
# haproxy -v
HAProxy version 2.9.7-5742051 2024/04/05 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.9.7.html
Running on: Linux 3.10.0-1160.99.1.el7.x86_64 #1 SMP Wed Sep 13 14:19:20 UTC 2023 x86_64

After I ran systemctl start haproxy got the error:

Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.

● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since pią 2024-04-05 22:50:46 CEST; 1min 53s ago
     Docs: man:haproxy(1)
  Process: 15115 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=1/FAILURE)

kwi 05 22:50:45 XXXXX systemd[1]: Failed to start HAProxy Load Balancer.
kwi 05 22:50:45 XXXXX systemd[1]: Unit haproxy.service entered failed state.
kwi 05 22:50:45 XXXXX systemd[1]: haproxy.service failed.

As you can see below, the default configuration file haproxy.cfg was corrupted.

# haproxy -c -f /etc/haproxy/haproxy.cfg
[NOTICE]   (15167) : haproxy version is 2.9.7-5742051
[NOTICE]   (15167) : path to executable is /sbin/haproxy
[ALERT]    (15167) : config : parsing [/etc/haproxy/haproxy.cfg:25] : unknown keyword 'ca-base' in 'listen' section; did you mean 'balance' maybe ?
[ALERT]    (15167) : config : parsing [/etc/haproxy/haproxy.cfg:26] : unknown keyword 'crt-base' in 'listen' section
[ALERT]    (15167) : config : parsing [/etc/haproxy/haproxy.cfg:30] : unknown keyword 'ssl-default-bind-ciphers' in 'listen' section; did you mean 'default_backend' maybe ?
[ALERT]    (15167) : config : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
[ALERT]    (15167) : config : Fatal errors found in configuration.

I fixed fatal errors and warnings. I tried to start HAProxy again and got an error

# systemctl start haproxy.service
Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.

The full command (ExecStartPre) to run HAProxy is /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -S /run/haproxy-master.sock (to debug removed -q)

And return got a fatal error

/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c  -S /run/haproxy-master.sock
[NOTICE]   (15643) : haproxy version is 2.9.7-5742051
[NOTICE]   (15643) : path to executable is /usr/sbin/haproxy
[ALERT]    (15643) : config : a master CLI socket was defined, but master-worker mode (-W) is not enabled.

I can play with -W parameter to solve the problem, but this is not a point. I cannot run haproxy service because configuration file and haproxy.service is corrupted.

[thinkpoddr]

Well, there are several problems with the RPM package:

• /var/lib/haproxy with permissions root:haproxy permissions is not created (service crashes due to that)
• haproxy.cfg with the ssl certificate stuff should be removed and is a blocker
• pidfile configuration is not needed (produces a warning that it is already defined (in the system service))
• stat socket path cannot be created due missing /run/haproxy directoy (maybe put into /var/lib/haproxy?) [2nd reason it crashes)

I include a proper working basic haproxy.cfg as demonstration configuration, based on the used one here:

#
# THIS IS SAMPLE CONFIG, FOR TEST, NOT FOR PRODUCTION!!!
#
global
    #log /dev/log    local0
    log 127.0.0.1 local0
    #log /dev/log    local1 notice
    log 127.0.0.1 local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/haproxy.sock mode 600 level admin
    pidfile /var/run/haproxy.pid
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    
listen stats
  bind :9000
  mode http
  stats enable
  stats realm Haproxy\ Statistics  # Title text for popup window
  stats auth haproxy:haproxy
  stats admin if TRUE
  stats uri /haproxy_stats

  timeout connect 5s
  timeout client 50s
  timeout server 450s

  # Default SSL material locations
  #ca-base /etc/ssl/certs
  #crt-base /etc/ssl/private

  # Default ciphers to use on SSL-enabled listening sockets.
  # For more information, see ciphers(1SSL).
  #ssl-default-bind-ciphers kEECDH+aRSA+AES:kRSA+AES:+AES256:RC4-SHA:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

[MattBianco]

It's not the TLS things themselves that should be commented out, it's their placement that is wrong.

Move the listen section down after the defaults in the provided haproxy.cfg
Comment out the pidfile from the provided haproxy.cfg
And also adjust the systemd service unit file to use -W flag in all three Execs of /usr/sbin/haproxy (for consistency)
This because the -c (check) will fail on warnings and prevent startup

On RHEL9 you typically also need to
sudo systemctl unmask rpcbind
sudo systemctl unmask rpcbind.socket
sudo systemctl enable rpcbind.socket
sudo systemctl enable rpcbind
and possibly also start them.