Skip to content

Segfault while assigning property #11656

New issue
New issue
Closed as not planned
Closed as not planned
Segfault while assigning property#11656
Labels
BugStatus: Needs Triage

Description

@danog
danog
opened on Jul 10, 2023

Description

We're using guzzlehttp/guzzle 7.5.0.

PHP config: https://paste.daniil.it/config

gdb backtrace:

(gdb) backtrace
#0  0x000056330533d16a in ZEND_ASSIGN_OBJ_SPEC_UNUSED_CONST_OP_DATA_CV_HANDLER () at ./Zend/zend_vm_execute.h:32960
#1  0x000056330534b898 in execute_ex (ex=0x50) at ./Zend/zend_vm_execute.h:58929
#2  0x000056330536bc5c in zend_generator_resume (orig_generator=0x7faf2dceabc0) at ./Zend/zend_generators.c:770
#3  0x000056330536cf59 in zend_generator_ensure_initialized (generator=0x7faf2dceabc0) at ./Zend/zend_generators.c:826
#4  zend_generator_ensure_initialized (generator=0x7faf2dceabc0) at ./Zend/zend_generators.c:823
#5  zim_Generator_current (execute_data=<optimized out>, return_value=0x7faf2dc14eb0) at ./Zend/zend_generators.c:881
#6  0x0000563305353d1b in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at ./Zend/zend_vm_execute.h:1951
#7  execute_ex (ex=0x50) at ./Zend/zend_vm_execute.h:56068
#8  0x0000563305355701 in zend_execute (op_array=0x7faf2dc7e000, return_value=0x0) at ./Zend/zend_vm_execute.h:60396
#9  0x00005633052e2e55 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at ./Zend/zend.c:1827
#10 0x000056330527e3fa in php_execute_script (primary_file=<optimized out>) at ./main/main.c:2542
#11 0x00005633053cc50a in do_cli (argc=9, argv=0x5633073e9170) at ./sapi/cli/php_cli.c:964
#12 0x000056330510ec5b in main (argc=9, argv=0x5633073e9170) at ./sapi/cli/php_cli.c:1333

gdb zbacktrace:

(gdb) zbacktrace
[0x7faf2dc151b0] GuzzleHttp\HandlerStack->setHandler(object[0x7faf2dc15200]) /zoon/zoon/vendor/guzzlehttp/guzzle/src/HandlerStack.php:116
[0x7faf2dc15100] Zoon\Queue\KafkaProxy\Consumer->getClient() /zoon/zoon/vendor/zoon/queue/src/KafkaProxy/Consumer.php:290
[0x7faf2dc15050] Zoon\Queue\KafkaProxy\Consumer->__construct(array(10)[0x7faf2dc150a0], object[0x7faf2dc150b0], object[0x7faf2dc150c0], object[0x7faf2dc150d0], array(6)[0x7faf2dc150e0])
/zoon/zoon/vendor/zoon/queue/src/KafkaProxy/Consumer.php:62
[0x7faf2dc14fc0] Zoon\Queue\KafkaProxy\Context->createConsumer(object[0x7faf2dc15010]) /zoon/zoon/vendor/zoon/queue/src/KafkaProxy/Context.php:68
[0x7faf2dc14f30] Zoon\Queue\Driver\KafkaPixy->getConsumer("Z\Module\Parser\Controller\Queue\ZextMatcherMatchFixer") /zoon/zoon/vendor/zoon/queue/src/Driver/KafkaPixy.php:140
[0x7faf20eb06e0] Zoon\Queue\Driver\KafkaPixy->consume("Z\Module\Parser\Controller\Queue\ZextMatcherMatchFixer") /zoon/zoon/vendor/zoon/queue/src/Driver/KafkaPixy.php:76
[0x7faf2dceac78] ???
[0x7faf2dc14ee0] Generator->current() [internal function]
[0x7faf2dc14e00] Zoon\Queue\ConsumerJsonRPC->run(290000) /zoon/zoon/vendor/zoon/queue/src/ConsumerJsonRPC.php:100
// more stuff

I can DM the coredump if needed.

ref: compressed_coredump-php.168655.1688664312

PHP Version

PHP 8.2.7

Operating System

Ubuntu 20.04, ondrej repos

Activity

  1. danog
    added
    Bug
    Status: Needs Triage
    on Jul 10, 2023

  2. danog commented on Jul 10, 2023

    @danog
    danog
    on Jul 10, 2023
    ContributorAuthor

    Note that we're using kafka-pixy with guzzle, not ext-rdkafka.

  3. iluuu1994 commented on Jul 10, 2023

    @iluuu1994
    iluuu1994
    on Jul 10, 2023
    Member

    It's not obvious what the issue is from the backtrace. I also see that you have quite a few extensions enabled (including ext-rdkafka), any of which could have some connection with the issue. Would it be possible to run PHP with valgrind? You should also set the ZEND_DONT_UNLOAD_MODULES=1 USE_ZEND_ALLOC=0 environment variables. If there is memory corruption it should reveal the origin.

  4. iluuu1994
    added
    Status: Needs Feedback
    and removed
    Status: Needs Triage
    on Jul 10, 2023

  5. danog commented on Jul 10, 2023

    @danog
    danog
    on Jul 10, 2023
    ContributorAuthor

    Hmm, I can ask, but I'm afraid running valgrind on a production queue consumer would slow it down too much, would DM-ing the coredump help a bit more?

  6. github-actions
    added
    Status: Needs Triage
    and removed
    Status: Needs Feedback
    on Jul 10, 2023

  7. danog commented on Jul 10, 2023

    @danog
    danog
    on Jul 10, 2023
    ContributorAuthor

    I actually have two coredumps with the same C trace now (compressed_coredump-php.97418.1688701817 and compressed_coredump-php.168655.1688664312), I can DM both if needed

  8. iluuu1994 commented on Jul 10, 2023

    @iluuu1994
    iluuu1994
    on Jul 10, 2023
    Member

    @danog Oh, valgrind is definitely not viable in production. I was hoping you could reproduce this in your dev environment.

  9. iluuu1994 commented on Jul 10, 2023

    @iluuu1994
    iluuu1994
    on Jul 10, 2023
    Member

    @danog The problem is that very often memory corruption will only cause a segfault down the line. It's very hard to backtrace and understand where there corruption occurred, even with a coredump. Valgrind or AddressSanitizer can be very helpful because they report violations when they happen.

  10. danog commented on Jul 10, 2023

    @danog
    danog
    on Jul 10, 2023
    ContributorAuthor

    Yeah understandable, I previously did start configuring a production docker image with AddressSanitizer enabled, guess I'll have to finish it and deploy it, because I can't reproduce this on the dev environment :)

  11. iluuu1994 commented on Jul 10, 2023

    @iluuu1994
    iluuu1994
    on Jul 10, 2023
    Member

    As long as it's in a separate environment (i.e. not redirecting all traffic onto it) using AddressSanitizer should be ok. There's a 2-3x slowdown. Note that failures will be printed to stderr, you can redirect it to a file with ASAN_OPTIONS="log_path=asan.log if that's easier.

  12. danog commented on Jul 10, 2023

    @danog
    danog
    on Jul 10, 2023
    ContributorAuthor

    Thanks!
    I'll get to it then, in the meantime I found another similar coredump which I assume is also just a memory corruption, correct? :)

    backtrace:

    (gdb) #0  ZEND_INIT_METHOD_CALL_SPEC_UNUSED_CONST_HANDLER () at ./Zend/zend_vm_execute.h:33302
#1  execute_ex (ex=0x7f83d5615600) at ./Zend/zend_vm_execute.h:58949
#2  0x000055865073fc5c in zend_generator_resume (orig_generator=0x7f83d56e9cc0) at ./Zend/zend_generators.c:770
#3  0x0000558650740f59 in zend_generator_ensure_initialized (generator=0x7f83d56e9cc0) at ./Zend/zend_generators.c:826
#4  zend_generator_ensure_initialized (generator=0x7f83d56e9cc0) at ./Zend/zend_generators.c:823
#5  zim_Generator_current (execute_data=<optimized out>, return_value=0x7f83d5614eb0) at ./Zend/zend_generators.c:881
#6  0x0000558650727d1b in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at ./Zend/zend_vm_execute.h:1951
#7  execute_ex (ex=0x7f83d5615600) at ./Zend/zend_vm_execute.h:56068
#8  0x0000558650729701 in zend_execute (op_array=0x7f83d567e000, return_value=0x0) at ./Zend/zend_vm_execute.h:60396
#9  0x00005586506b6e55 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at ./Zend/zend.c:1827
#10 0x00005586506523fa in php_execute_script (primary_file=<optimized out>) at ./main/main.c:2542
#11 0x00005586507a050a in do_cli (argc=9, argv=0x558652553170) at ./sapi/cli/php_cli.c:964
#12 0x00005586504e2c5b in main (argc=9, argv=0x558652553170) at ./sapi/cli/php_cli.c:1333

    zbacktrace:

    (gdb) [0x7f83d56155b0] GuzzleHttp\Psr7\Request->__construct("GET", object[0x7f83d5615610], array(1)[0x7f83d5615620], object[0x7f83d5615630], "1.1")
/zoon/zoon/vendor/guzzlehttp/psr7/src/Request.php:42
[0x7f83d5615490] GuzzleHttp\Psr7\Utils->modifyRequest(object[0x7f83d56154e0], array(2)[0x7f83d56154f0]) /zoon/zoon/vendor/guzzlehttp/psr7/src/Utils.php:218
[0x7f83d5615390] GuzzleHttp\Client->applyOptions(object[0x7f83d56153e0], reference) /zoon/zoon/vendor/guzzlehttp/guzzle/src/Client.php:440
[0x7f83d56152f0] GuzzleHttp\Client->transfer(object[0x7f83d5615340], reference) /zoon/zoon/vendor/guzzlehttp/guzzle/src/Client.php:326
[0x7f83d5615210] GuzzleHttp\Client->requestAsync("GET", object[0x7f83d5615270], array(13)[0x7f83d5615280]) /zoon/zoon/vendor/guzzlehttp/guzzle/src/Client.php:168
[0x7f83d5615180] GuzzleHttp\Client->request("GET", "/messages", array(3)[0x7f83d56151f0]) /zoon/zoon/vendor/guzzlehttp/guzzle/src/Client.php:187
[0x7f83d5615100] GuzzleHttp\Client->get("/messages", array(2)[0x7f83d5615160]) /zoon/zoon/vendor/guzzlehttp/guzzle/src/ClientTrait.php:44
[0x7f83d5614fd0] Zoon\Queue\KafkaProxy\Consumer->doReceive(32) /zoon/zoon/vendor/zoon/queue/src/KafkaProxy/Consumer.php:141

    ref: compressed_coredump-php.2103386.1688761504.txt

  13. iluuu1994 commented on Jul 10, 2023

    @iluuu1994
    iluuu1994
    on Jul 10, 2023
    Member

    I'll get to it then, in the meantime I found another similar coredump which I assume is also just a memory corruption, correct? :)

    Most likely, yes. Similarly to the previous one, it fails at this line:

    EXPECTED(CACHED_PTR(opline->result.num) == called_scope)

    It seems that this should only segfault if EX(run_time_cache) or opline->result.num got corrupted.

  14. danog commented on Jul 11, 2023

    @danog
    danog
    on Jul 11, 2023
    ContributorAuthor

    Btw should I also set the ZEND_DONT_UNLOAD_MODULES=1 USE_ZEND_ALLOC=0 env variables when running PHP with ASAN (not valgrind)?

  15. iluuu1994 commented on Jul 11, 2023

    @iluuu1994
    iluuu1994
    on Jul 11, 2023
    Member

    @danog USE_ZEND_ALLOC=0 should suffice.

  16. kamil-tekiela commented on Nov 7, 2026

    @kamil-tekiela
    kamil-tekiela
    on Nov 7, 2026
    Member

    Same as #11655 I am closing it for lack of reproducible steps or ASAN log. We don't even know if the issue is in PHP core or in one of the loaded extensions.

  17. kamil-tekiela
    closed this as not plannedon Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugStatus: Needs Triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions