Segfault while copying parent interfaces table

[danog]

Description

We've been encountering regular php-fpm segfaults in production.

Inspecting coredumps revealed gibberish class entries in the interfaces field of the ce of a class named Z\Module\Comment\Orm\CommentObject, which caused segfaults when copying interfaces over to the interfaces field of the ce of child class Z\Module\Zext\Orm\ZextCommentObject.

Native backtrace:

#0  0x000055a86a9e586f in do_implement_interface (iface=0x41868fc8, ce=0x7f53332a4da0) at ./Zend/zend_inheritance.c:1288
#1  zend_do_inherit_interfaces (ce=0x7f53332a4da0, iface=<optimized out>, iface=<optimized out>) at ./Zend/zend_inheritance.c:1326
#2  0x000055a86a9ea16e in zend_try_early_bind (ce=0x7f53332a4da0, ce@entry=0x43e5fca0, parent_ce=0x43e67d60, lcname=0x417daa70, delayed_early_binding=delayed_early_binding@entry=0x55a86b5f9f90)
    at ./Zend/zend_inheritance.c:3062
#3  0x000055a86a92ec6b in zend_do_delayed_early_binding (op_array=op_array@entry=0x7f531a905000, first_early_binding_opline=<optimized out>) at ./Zend/zend_compile.c:1381
#4  0x00007f533ec49a9b in zend_accel_load_script (persistent_script=persistent_script@entry=0x43e5fa40, from_shared_memory=from_shared_memory@entry=1)
    at ./ext/opcache/zend_accelerator_util_funcs.c:255
#5  0x00007f533ec38260 in persistent_compile_file (type=<optimized out>, file_handle=<optimized out>) at ./ext/opcache/ZendAccelerator.c:2247
#6  persistent_compile_file (file_handle=<optimized out>, type=<optimized out>) at ./ext/opcache/ZendAccelerator.c:1957
#7  0x000055a86a9164ad in compile_filename (type=type@entry=2, filename=filename@entry=0x7f533329e0c0) at ./Zend/zend_language_scanner.c:707
#8  0x000055a86a984a86 in zend_include_or_eval (inc_filename_zv=<optimized out>, type=2) at ./Zend/zend_execute.c:4753
#9  0x000055a86a992f8e in ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER () at ./Zend/zend_vm_execute.h:38732
#10 0x000055a86a9b9c95 in execute_ex (ex=0x41868fc8) at ./Zend/zend_vm_execute.h:59146
#11 0x000055a86a9440fc in zend_call_function (fci=<optimized out>, fci_cache=<optimized out>) at ./Zend/zend_execute_API.c:908
#12 0x000055a86a9444cd in zend_call_known_function (fn=0x4344a338, object=<optimized out>, called_scope=<optimized out>, retval_ptr=retval_ptr@entry=0x0, param_count=param_count@entry=1,
    params=params@entry=0x7ffc1f850030, named_params=0x0) at ./Zend/zend_execute_API.c:997
#13 0x000055a86a844377 in spl_perform_autoload (class_name=0x417daa30, lc_name=0x7f531a8d6a80) at ./ext/spl/php_spl.c:436
#14 0x000055a86a94338c in zend_lookup_class_ex (name=<optimized out>, key=key@entry=0x0, flags=flags@entry=0) at ./Zend/zend_execute_API.c:1141
#15 0x00007f533ec30df3 in zend_accel_inheritance_cache_get (ce=0x43e67d60, parent=0x435a0830, traits_and_interfaces=0x7ffc1f850120) at ./ext/opcache/ZendAccelerator.c:2317
#16 0x000055a86a9e8612 in zend_do_link_class (ce=ce@entry=0x43e67d60, lc_parent_name=lc_parent_name@entry=0x41716d98, key=0x4181cfa8) at ./Zend/zend_inheritance.c:2783
#17 0x000055a86a92e3e6 in zend_bind_class_in_slot (class_table_slot=<optimized out>, lcname=0x43e8f620, lc_parent_name=0x41716d98) at ./Zend/zend_compile.c:1131
#18 0x000055a86a92e4ac in do_bind_class (lcname=0x43e8f620, lc_parent_name=0x41716d98) at ./Zend/zend_compile.c:1164
#19 0x000055a86a985ca9 in ZEND_DECLARE_CLASS_SPEC_CONST_HANDLER () at ./Zend/zend_vm_execute.h:5335
#20 0x000055a86a9b956f in execute_ex (ex=0x41868fc8) at ./Zend/zend_vm_execute.h:56216
#21 0x000055a86a9440fc in zend_call_function (fci=<optimized out>, fci_cache=<optimized out>) at ./Zend/zend_execute_API.c:908
#22 0x000055a86a9444cd in zend_call_known_function (fn=0x4344a338, object=<optimized out>, called_scope=<optimized out>, retval_ptr=retval_ptr@entry=0x0, param_count=param_count@entry=1,
    params=params@entry=0x7ffc1f8506b0, named_params=0x0) at ./Zend/zend_execute_API.c:997
#23 0x000055a86a844377 in spl_perform_autoload (class_name=0x7f531a8d69c0, lc_name=0x7f531a8d6ac0) at ./ext/spl/php_spl.c:436
#24 0x000055a86a94338c in zend_lookup_class_ex (name=<optimized out>, key=key@entry=0x0, flags=flags@entry=0) at ./Zend/zend_execute_API.c:1141
#25 0x000055a86a94358d in zend_lookup_class (name=<optimized out>) at ./Zend/zend_execute_API.c:1162
#26 0x000055a86a96c645 in class_exists_impl (return_value=0x7f533f014950, skip_flags=3, flags=8, execute_data=0x7f533f014960) at ./Zend/zend_builtin_functions.c:990
#27 zif_class_exists (execute_data=0x7f533f014960, return_value=0x7f533f014950) at ./Zend/zend_builtin_functions.c:1004
#28 0x000055a86a9bbdf4 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER () at ./Zend/zend_vm_execute.h:1297
#29 execute_ex (ex=0x41868fc8) at ./Zend/zend_vm_execute.h:55775
#30 0x000055a86a9c286d in zend_execute (op_array=0x7f533f070000, return_value=0x0) at ./Zend/zend_vm_execute.h:60147
#31 0x000055a86a952995 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at ./Zend/zend.c:1799
#32 0x000055a86a8ef78a in php_execute_script (primary_file=primary_file@entry=0x7ffc1f852e30) at ./main/main.c:2541
#33 0x000055a86a793aed in main (argc=<optimized out>, argv=<optimized out>) at ./sapi/fpm/fpm/fpm_main.c:1914

PHP backtrace:

[0x7f533f014b70] Composer\Autoload\includeFile("/zoon/zoon/vendor/composer/../..//Module/Zext/Orm/ZextCommentObject.php") /zoon/zoon/vendor/composer/ClassLoader.php:571
[0x7f533f014af0] Composer\Autoload\ClassLoader->loadClass("Z\Module\Zext\Orm\ZextCommentObject") /zoon/zoon/vendor/composer/ClassLoader.php:428
[0x7f533f014aa0] (main) /zoon/zoon/Module/Comment/Orm/CommentObject.php:154
[0x7f533f014a40] Composer\Autoload\includeFile("/zoon/zoon/vendor/composer/../..//Module/Comment/Orm/CommentObject.php") /zoon/zoon/vendor/composer/ClassLoader.php:571
[0x7f533f0149c0] Composer\Autoload\ClassLoader->loadClass("Z\Module\Comment\Orm\CommentObject") /zoon/zoon/vendor/composer/ClassLoader.php:428
[0x7f533f014960] class_exists("Z\Module\Comment\Orm\CommentObject") [internal function]
[0x7f533f0148f0] Zoon\ORM\Mongo\MongoMapper->prepareObjectClassName() /zoon/zoon/vendor/zoon/orm/src/Mongo/MongoMapper.php:174

... other stuff

Here's what I get when inspecting the class entries of the 3 interfaces which CommentObject actually extends (0x43e67d60 is the CE of CommentObject aka the parent_ce of zend_try_early_bind):

(gdb) printf "%s\n", ((zend_class_entry*)0x43e67d60)->name->val
Z\Module\Comment\Orm\CommentObject

(gdb) p (((zend_class_entry*)0x43e67d60)->interfaces[0])
$72 = (zend_class_entry *) 0x41868f48
(gdb) p (((zend_class_entry*)0x43e67d60)->interfaces[1])
$73 = (zend_class_entry *) 0x41868f88
(gdb) p (((zend_class_entry*)0x43e67d60)->interfaces[2])
$74 = (zend_class_entry *) 0x41868fc8

(gdb) p (((zend_class_entry*)0x43e67d60)->interfaces[0])->type
$68 = 65 'A'
(gdb) p (((zend_class_entry*)0x43e67d60)->interfaces[1])->type
$69 = 2 '\002'
(gdb) p (((zend_class_entry*)0x43e67d60)->interfaces[2])->type
$70 = 97 'a'

(gdb) printf "%s\n", (((zend_class_entry*)0x43e67d60)->interfaces[0])->name->val
Cannot access memory at address 0xddde8fbc0245db50
(gdb) printf "%s\n", (((zend_class_entry*)0x43e67d60)->interfaces[1])->name->val
Cannot access memory at address 0xf16aa6cb353b4ff0
(gdb) printf "%s\n", (((zend_class_entry*)0x43e67d60)->interfaces[2])->name->val
Cannot access memory at address 0x8b207ca4bb626fc1

As you can see only the second interface has a valid type of ZEND_USER_CLASS, and in all cases the names are unreadable, possibly indicating that pointers contained in interfaces point to just gibberish.

Note that the autoloading of the ZextCommentObject seems to be triggered while parsing CommentObject by some internal opcache logic (maybe zend_accel_inheritance_cache_get?), not by any userland code requiring ZextCommentObject in the CommentObject class (like 154 of CommentObject from the PHP backtrace is just the line with the class declaration, which doesn't mention ZextCommentObject in any way).

PHP Version

PHP 8.1.13, opcache enabled, JIT disabled (due to other segfaults I'll try to report too)

Operating System

Ubuntu 20.04, ondrej repos

[cmb69]

Apparently, an inheritance cache issue.

[danog]

Might be worth tagging @dstogov then :)

[dstogov]

(gdb) printf "%s\n", (((zend_class_entry*)0x43e67d60)->interfaces[2])->name->val
Cannot access memory at address 0x8b207ca4bb626fc1

As you can see only the second interface has a valid type of ZEND_USER_CLASS, and in all cases the names are unreadable, possibly indicating that pointers contained in interfaces point to just gibberish.

zend_class_entry.interfaces becomes valid only after the linking. At the point of failure you may access zend_class_entry.interface_names.

[dstogov]

Unfortunately, I can't guess the reason of failure. Could you please analyse core dump at the line where crash occurs and print all the used data.

1288: 	if (!(ce->ce_flags & ZEND_ACC_INTERFACE) && iface->interface_gets_implemented && iface->interface_gets_implemented(iface, ce) == FAILURE) {

(gdb) p (char*)ce.name.val
(gdb) p (char*)iface.name.val
(gdb) p iface->interface_gets_implemented

I suspect interface_gets_implemented may be invalid. But I can't guess the reason yet.
Do you use file cache?

[danog]

(gdb) p (char*)ce.name.val
$1 = 0x417daa48 "Z\\Module\\Zext\\Orm\\ZextCommentObject"
(gdb) p (char*)iface.name.val
$2 = 0x8b207ca4bb626fc1 <error: Cannot access memory at address 0x8b207ca4bb626fc1>
(gdb) p iface->interface_gets_implemented
$3 = (int (*)(zend_class_entry *, zend_class_entry *)) 0x2e2e2f2e2e2f7265

And we don't use file cache.

[dstogov]

I have no ideas. To fix this, I need a way to reproduce the crash (locally or in some container/VM/remote host).
You may write details to my personal email.

[danog]

I have no way of reproducing this yet, but I wonder if #10719 fixed this...

[dstogov]

Very probably that patch might fix your case as well. Unfortunately, nothing can be proved without a reproduction scenario.

[iluuu1994]

@danog Are you not hitting this issue at all anymore? In that case I would suggest closing this and reopening if it occurs again. Is that ok?

[danog]

Yeah, I stopped encountering the issue after making the config changes described in #10477.
Closing for now.