PHP-FPM Signal 13 (SIGPIPE)

[frederikbosch]

Description

Today I started seeing crashes on one of my PHP-FPM 8.2.23 servers with a SIGPIPE message.

WARNING: [pool www] child 1206 exited on signal 13 (SIGPIPE) after 27.818575 seconds from start

This server was using a fresh (Docker) build that was created 2 days ago. Another PHP 8.2.23 server that was created directly after its release was not seeing the issues.

My guess is that a version of another linked library or extension was upgraded in the meanwhile and that this causes the newer build to crash whereas the older one does not. When I downgraded the crashing server to a 8.2.22 build of one month ago, the errors disappeared.

So I wondered what library or extension might be causing the trouble. The only result I found online on the SIGPIPE error was related to an update in libxml. Maybe I am running into this problem or a similar one too. I will continue investigating and report what I find.

PHP Version

8.2.23 / 8.3.11

Operating System

Alpine 3.18 (8.2.23) / 3.19 (8.3.11)

[frederikbosch]

When looking at my development server's logs, which is on PHP 8.3.11, is also running into the error.

[devnexen]

Hi @frederikbosch

extension was upgraded in the meanwhile

speaking of this, what would be those extensions ? ie php -m

[frederikbosch]

[PHP Modules]
bcmath
Core
ctype
curl
date
dom
exif
fileinfo
filter
ftp
gd
gmp
hash
iconv
imagick (using fixed versions, didn't change)
intl (74.1)
json
libxml (2.11.8)
mbstring
mysqlnd
openssl
pcre (10.42)
PDO
pdo_mysql
pdo_sqlite
Phar
posix
random
readline
redis (6.0.2)
Reflection
session
SimpleXML
sockets
sodium
SPL
sqlite3
standard
tokenizer
xdebug
xml (2.11.8)
xmlreader
xmlwriter
xsl (1.1.39)
Zend OPcache
zip
zlib

[Zend Modules]
Xdebug (is disabled in production where the error also occurs)
Zend OPcache

*striking through what has remained the same since a build that does not contain these crashes

[devnexen]

Thanks. It might be because of xdebug. Would it be possible to disable it, for a while at least, and sees what happens ?

[frederikbosch]

In production there is no Xdebug, and that's where I found the error originally. The above is from my dev server that is now running 8.3.11.

[frederikbosch]

While I am programming in PHP for decades (since version 3), I still don't know how to debug these kind of crashes.

[devnexen]

Thanks for the clarifications. Would debugging/launching fpm with GDB be an option ? I won t have time to look into it only later today, but having a backtrace of the calls would help I think.

[frederikbosch]

How do I do that? Can I do that with my current setup, or do I need to build with certain flags enabled?

[devnexen]

Once FPM is launched, you might be able to attach a gdb instance per child worker. ie gdb -p <FPM child process id>, launching your requests, once SIGPIPE is triggered to a child process, in the related gdb instance typing bt full. Ideally, fpm with debug symbols is better but at least if we get the call traces it might help determine which extension is problematic.

[devnexen]

I may come up with other ideas in the meantime unless someone else beat me to it :-)

[frederikbosch]

Thanks so far, I am building a new container with gdb installed, and with --enabled-debug and without the stripping in my usual builds.

[frederikbosch]

• docker container is started with cap-add SYS_PTRACE
• FPM is started with gdb php-fpm
• gdb -p <id> is attached to every fpm process
• cat /proc/sys/kernel/yama/ptrace_scope gives 0

And I still get:

ERROR: failed to ptrace(ATTACH) child 41: Operation not permitted (1)

[devnexen]

I see. Did you try adding --privileged when running your container ?