0

I have a php-fpm pool serving files from network attached file systems, so sometimes I see messages like this in my dmesg logs:

 [Sat Jul 10 01:51:56 2021] INFO: task php-fpm:2260874 blocked for more than 1208 seconds.
[Sat Jul 10 01:51:56 2021]       Not tainted 5.11.10-051110-generic #202103251032
[Sat Jul 10 01:51:56 2021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Sat Jul 10 01:51:56 2021] task:php-fpm         state:D stack:    0 pid:2260874 ppid: 10448 flags:0x00004004
[Sat Jul 10 01:51:56 2021] Call Trace:
[Sat Jul 10 01:51:56 2021]  __schedule+0x23d/0x670
[Sat Jul 10 01:51:56 2021]  schedule+0x4f/0xc0
[Sat Jul 10 01:51:56 2021]  io_schedule+0x46/0x70
[Sat Jul 10 01:51:56 2021]  wait_on_page_bit_common+0x11a/0x3a0
[Sat Jul 10 01:51:56 2021]  ? trace_event_raw_event_filemap_set_wb_err+0xd0/0xd0
[Sat Jul 10 01:51:56 2021]  __lock_page+0x44/0x50
[Sat Jul 10 01:51:56 2021]  invalidate_inode_pages2_range+0x290/0x440
[Sat Jul 10 01:51:56 2021]  ? fuse_simple_request+0xdc/0x280
[Sat Jul 10 01:51:56 2021]  ? fuse_send_open+0xb9/0x110
[Sat Jul 10 01:51:56 2021]  invalidate_inode_pages2+0x17/0x20
[Sat Jul 10 01:51:56 2021]  fuse_finish_open+0x89/0x150
[Sat Jul 10 01:51:56 2021]  fuse_open_common+0x1b2/0x1c0
[Sat Jul 10 01:51:56 2021]  ? fuse_open_common+0x1c0/0x1c0
[Sat Jul 10 01:51:56 2021]  fuse_open+0x10/0x20
[Sat Jul 10 01:51:56 2021]  do_dentry_open+0x156/0x370
[Sat Jul 10 01:51:56 2021]  vfs_open+0x2d/0x30
[Sat Jul 10 01:51:56 2021]  do_open+0x1c3/0x340
[Sat Jul 10 01:51:56 2021]  path_openat+0x10a/0x1d0
[Sat Jul 10 01:51:56 2021]  do_filp_open+0x8c/0x130
[Sat Jul 10 01:51:56 2021]  ? __check_object_size+0x1c/0x20
[Sat Jul 10 01:51:56 2021]  do_sys_openat2+0x9b/0x150
[Sat Jul 10 01:51:56 2021]  __x64_sys_openat+0x56/0x90
[Sat Jul 10 01:51:56 2021]  do_syscall_64+0x38/0x90
[Sat Jul 10 01:51:56 2021]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Is it possible to have php-fpm automatically reload with either systemd or .ini settings when that happens ?

Freedo
  • 1,255
  • You could write a script to monitor the kernel log and restart the daemon when it sees the pattern task php-phm:.*blocked for more than. But you'd be better off finding out what's causing the problem and fixing it - it looks like it's related to a FUSE filesystem. Which fuse fs are you using? – cas Jul 20 '21 at 06:51
  • I use rclone mount (for local disk caching) + glusterfs as backend – Freedo Jul 20 '21 at 06:53

0 Answers0