CentOS 7 - mod_fcgid error

Cookbook version

0.4.0

Chef-client version

12.12.13

Platform Details

CentOS 7 (fully up-to-date)

Scenario:

Running the application Request Tracker behind httpd.
This requires mod_fcgid, or at least that's the one I'm trying to configure at the moment.

Documentation on this can be found here: https://docs.bestpractical.com/rt/4.4.1/web_deployment.html

Steps to Reproduce:

I have the following setup using a wrapper cookbook in front of the httpd cookbook.

# Create httpd instance
httpd_service 'webserver' do
  listen_ports ['80', '443']
  action [:create, :start]
end

# Install a few modules
%w{ssl unixd fcgid}.each do |mod|
  httpd_module mod do
    instance 'webserver'
    action :create
  end
end

# Create a vhost
httpd_config 'vhost1' do
  config_name 'vhost1'
  source 'vhost1.cnf.erb'
  instance 'webserver'
  notifies :restart, 'httpd_service[webserver]'
  action :create
end

When running I get the following error, this happens during the module installation (or so I think at least).

      Error executing action `start` on resource 'service[httpd-webserver]'

Thus, httpd can't start.

Expected Result:

I would expect everything to start without errors.

Actual Result:

The httpd service can't start. below the root cause of this problem:

systemctl status httpd-webserver -l
● httpd-webserver.service - The Apache HTTP Server
   Loaded: loaded (/etc/systemd/system/httpd-webserver.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since di 2016-07-19 11:28:16 CEST; 4min 54s ago
  Process: 17800 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
  Process: 17178 ExecReload=/usr/sbin/httpd-webserver -f /etc/httpd-webserver/conf/httpd.conf -k graceful (code=exited, status=0/SUCCESS)
  Process: 17798 ExecStart=/usr/sbin/httpd-webserver -f /etc/httpd-webserver/conf/httpd.conf -DFOREGROUND (code=exited, status=1/FAILURE)
 Main PID: 17798 (code=exited, status=1/FAILURE)

jul 19 11:28:16 webserver.local systemd[1]: Starting The Apache HTTP Server...
jul 19 11:28:16 webserver.local httpd-webserver[17798]: httpd-webserver: Syntax error on line 19 of /etc/httpd-webserver/conf/httpd.conf: Syntax error on line 1 of /etc/httpd-webserver/conf.modules.d/fcgid.load: Cannot load /usr/lib64/httpd/modules/mod_fcgid.so into server: /usr/lib64/httpd/modules/mod_fcgid.so: undefined symbol: ap_unixd_setup_child
jul 19 11:28:16 webserver.local systemd[1]: httpd-webserver.service: main process exited, code=exited, status=1/FAILURE
jul 19 11:28:16 webserver.local kill[17800]: kill: cannot find process ""
jul 19 11:28:16 webserver.local systemd[1]: httpd-webserver.service: control process exited, code=exited status=1
jul 19 11:28:16 webserver.local systemd[1]: Failed to start The Apache HTTP Server.
jul 19 11:28:16 webserver.local systemd[1]: Unit httpd-webserver.service entered failed state.
jul 19 11:28:16 webserver.local systemd[1]: httpd-webserver.service failed.

This gives an error on loading mod_fcgid which is pointing to mod_unixd: "undefined symbol: ap_unixd_setup_child"

After some googeling this has to do with the order in which the modules are loaded. By default everything is loaded in alphabetic order, using this cookbook that means that all modules are loaded in this order:

ls -l /etc/httpd-webserver/conf.modules.d
totaal 104
-rw-r--r--. 1 root root 78 19 jul 11:24 access_compat.load
-rw-r--r--. 1 root root 62 19 jul 11:24 alias.load
-rw-r--r--. 1 root root 72 19 jul 11:24 auth_basic.load
-rw-r--r--. 1 root root 72 19 jul 11:24 authn_core.load
-rw-r--r--. 1 root root 72 19 jul 11:24 authn_file.load
-rw-r--r--. 1 root root 72 19 jul 11:24 authz_core.load
-rw-r--r--. 1 root root 72 19 jul 11:24 authz_host.load
-rw-r--r--. 1 root root 72 19 jul 11:24 authz_user.load
-rw-r--r--. 1 root root 70 19 jul 11:24 autoindex.load
-rw-r--r--. 1 root root 66 19 jul 11:24 deflate.load
-rw-r--r--. 1 root root 58 19 jul 11:24 dir.load
-rw-r--r--. 1 root root 58 19 jul 11:24 env.load
-rw-r--r--. 1 root root 62 19 jul 11:24 fcgid.load
-rw-r--r--. 1 root root 64 19 jul 11:24 filter.load
-rw-r--r--. 1 root root 72 19 jul 11:24 log_config.load
-rw-r--r--. 1 root root 62 19 jul 11:24 logio.load
-rw-r--r--. 1 root root 60 19 jul 11:24 mime.load
-rw-r--r--. 1 root root 70 19 jul 11:24 mpm_event.load
-rw-r--r--. 1 root root 74 19 jul 11:24 negotiation.load
-rw-r--r--. 1 root root 68 19 jul 11:24 setenvif.load
-rw-r--r--. 1 root root 58 19 jul 11:24 ssl.load
-rw-r--r--. 1 root root 64 19 jul 11:24 status.load
-rw-r--r--. 1 root root 66 19 jul 11:24 systemd.load
-rw-r--r--. 1 root root 62 19 jul 11:24 unixd.load
-rw-r--r--. 1 root root 66 19 jul 11:24 version.load
-rw-r--r--. 1 root root 68 19 jul 11:24 watchdog.load

Meaning mod_fcgid is loaded before mod_unixd.

This can be solved by changing the filenames in the directory "/etc/httpd-webserver/conf.modules.d". Being able to set names like e.g. 10-unixd and 20-fcgid would solve this issue.

I've tried doing this with the cookbook in it's current state, but that didn't give me the expected result (it changed the loadmodule line).

This is the snippet:

counter = 0
%w{ssl unixd fcgid}.each do |mod|
  counter = counter + 10
  httpd_module "#{counter}-#{mod}" do
    filename "#{counter}-#{mod}"
    module_name mod
    instance 'webserver'
    action :create
  end
end

This didn't do what I wanted, but if we were allowed to change the filename in conf.modules.d then this would be a way of setting the load order for these modules.

As a reference, this is how the default httpd module configuration is done on CentOS 7:

ls -la httpd/conf.modules.d/
totaal 44
drwxr-xr-x. 2 root root 4096 19 jul 10:44 .
drwxr-xr-x. 5 root root   86 19 jul 10:43 ..
-rw-r--r--. 1 root root 3739 18 jul 17:22 00-base.conf
-rw-r--r--. 1 root root  139 18 jul 17:22 00-dav.conf
-rw-r--r--. 1 root root   41 18 jul 17:22 00-lua.conf
-rw-r--r--. 1 root root  742 18 jul 17:22 00-mpm.conf
-rw-r--r--. 1 root root  957 18 jul 17:22 00-proxy.conf
-rw-r--r--. 1 root root   41 18 jul 17:22 00-ssl.conf
-rw-r--r--. 1 root root   88 18 jul 17:22 00-systemd.conf
-rw-r--r--. 1 root root  451 18 jul 17:22 01-cgi.conf
-rw-r--r--. 1 root root  448 17 sep  2014 02-perl.conf
-rw-r--r--. 1 root root   45 10 jun  2014 10-fcgid.conf

That's basically what I'm trying to achieve in order to resolve the issue I'm having.

Ok, I've hacked my way into fixing this issue for myself.
The following snippet ensures the correct order for unixd and fcgid:

httpd_module 'fcgid' do
  instance 'webserver'
  action :create
  notifies :run, 'execute[fix_unixd-fcgid]', :immediately
end

execute 'fix_unixd-fcgid' do
 command '/usr/bin/mv /etc/httpd-webserver/conf.modules.d/unixd.load /etc/httpd-webserver/conf.modules.d/10-unixd.load && /usr/bin/mv /etc/httpd-webserver/conf.modules.d/fcgid.load /etc/httpd-webserver/conf.modules.d/20-fcgid.load'
 action :nothing
end

Note: unixd is loaded by default, thus any reference using httpd_module for unixd is ignored.

It's rather ugly, but it works. So my issue (for now) is resolved. The ugly part is that it will create and move the unixd and fcgid config files during every chef run (in our case every 15 minutes). This doesn't seem to trigger a reload of the httpd service, so it's limited to file creation/move.

Thus that only leaves a feature request to do this in a nice way without doing a hack in this manner.

[iennae]

@bgroenendal Thank you so much for the detailed report and sharing your testing. We'll take a look at this in the next 2 weeks and try to come up with a better way to do this.

[iennae]

Still don't have a fix for this, but please be aware that this is something we're still looking into.