Repository navigation
JIT cause 100% cpu usage #11736
Description
Description
The following code:
[opcache]
opcache.enable=1
opcache.enable_cli=1
opcache.memory_consumption=512
opcache.jit_buffer_size=256M
opcache.interned_strings_buffer=16
opcache.max_accelerated_files=200000
opcache.max_wasted_percentage=5
opcache.use_cwd=1
opcache.validate_timestamps=0
opcache.revalidate_freq=600
opcache.revalidate_path=0
opcache.save_comments=1
opcache.enable_file_override=0
opcache.optimization_level=0x7FFFBFFF
opcache.dups_fix=0
opcache.max_file_size=0
opcache.consistency_checks=0
opcache.force_restart_timeout=180
opcache.file_cache_only=0
opcache.file_cache_consistency_checks=1
opcache.huge_code_pages=1
opcache.preload=/path/to/config/preload.php
opcache.preload_user=userResulted in this output:
[18-Jul-2023 03:40:09] NOTICE: [pool www] child 179 started
Fatal error: Maximum execution time of 99+2 seconds exceeded (terminated) in /path/to/file.php on line 33
[18-Jul-2023 03:40:13] WARNING: [pool www] child 47 exited with code 124 after 381.082060 seconds from start
[18-Jul-2023 03:40:13] NOTICE: [pool www] child 182 started
Fatal error: Maximum execution time of 99+2 seconds exceeded (terminated) in /path/to/file.php on line 33
[18-Jul-2023 03:40:25] WARNING: [pool www] child 42 exited with code 124 after 393.292804 seconds from start
But I expected this output instead:
no errors in log
Hello,
after enabling JIT, our application start perform better (~10% faster). But, some php-fpm pods (~1 in 20) after start use all available cpu and timeout the requests. After pod restart, everything back to normal. Without JIT everything is ok and cpu utilization is at normal value.
Regards
opcache.blacklist_filename => no value => no value
opcache.consistency_checks => 0 => 0
opcache.dups_fix => Off => Off
opcache.enable => On => On
opcache.enable_cli => On => On
opcache.enable_file_override => Off => Off
opcache.error_log => no value => no value
opcache.file_cache => no value => no value
opcache.file_cache_consistency_checks => On => On
opcache.file_cache_only => Off => Off
opcache.file_update_protection => 2 => 2
opcache.force_restart_timeout => 180 => 180
opcache.huge_code_pages => On => On
opcache.interned_strings_buffer => 16 => 16
opcache.jit => tracing => tracing
opcache.jit_bisect_limit => 0 => 0
opcache.jit_blacklist_root_trace => 16 => 16
opcache.jit_blacklist_side_trace => 8 => 8
opcache.jit_buffer_size => 256M => 256M
opcache.jit_debug => 0 => 0
opcache.jit_hot_func => 127 => 127
opcache.jit_hot_loop => 64 => 64
opcache.jit_hot_return => 8 => 8
opcache.jit_hot_side_exit => 8 => 8
opcache.jit_max_exit_counters => 8192 => 8192
opcache.jit_max_loop_unrolls => 8 => 8
opcache.jit_max_polymorphic_calls => 2 => 2
opcache.jit_max_recursive_calls => 2 => 2
opcache.jit_max_recursive_returns => 2 => 2
opcache.jit_max_root_traces => 1024 => 1024
opcache.jit_max_side_traces => 128 => 128
opcache.jit_prof_threshold => 0.005 => 0.005
opcache.lockfile_path => /tmp => /tmp
opcache.log_verbosity_level => 1 => 1
opcache.max_accelerated_files => 200000 => 200000
opcache.max_file_size => 0 => 0
opcache.max_wasted_percentage => 5 => 5
opcache.memory_consumption => 512 => 512
opcache.opt_debug_level => 0 => 0
opcache.optimization_level => 0x7FFFBFFF => 0x7FFFBFFF
opcache.preferred_memory_model => no value => no value
opcache.preload => /path/to/config/preload.php => //path/to/config/preload.php
opcache.preload_user => user => user
opcache.protect_memory => Off => Off
opcache.record_warnings => Off => Off
opcache.restrict_api => no value => no value
opcache.revalidate_freq => 600 => 600
opcache.revalidate_path => Off => Off
opcache.save_comments => On => On
opcache.use_cwd => On => On
opcache.validate_permission => Off => Off
opcache.validate_root => Off => Off
opcache.validate_timestamps => Off => Off
PHP Version
PHP 8.1.20
Operating System
Debian GNU/Linux 10 (buster) docker container on Amazon Linux 2
Activity
ndossche commentedon Jul 18, 2023
I think this will be pretty much impossible to debug and fix without having a reproducer or at least more info about where it goes wrong.
maaarghk commentedon Jul 18, 2023
looks like #11609 - @iluuu1994 do you know offhand if it'd be relevant to php 8.1.20 as indicated here?
iluuu1994 commentedon Jul 18, 2023
@maaarghk I don't think this is related, as this reverting the tracing counter handlers was only added in 8.2.
andrii29 commentedon Jul 18, 2023
what tool should i use to get relevant trace?
some trace logs from stuck workers
ndossche commentedon Jul 18, 2023
Do you have a script that reliably reproduces the problem, or is it unrelated to a script?
The syscall trace you show indicates it seems stuck in an infinite EAGAIN loop, but that doesn't include the source information...
andrii29 commentedon Jul 20, 2023
we use symfony 5.4 + overblog/GraphQLBundle
I enabled slow log for 15 seconds and got next results
logs from workers without slow log
in normal state requests are finished in less than second
andrii29 commentedon Oct 5, 2023
same behaviour on php 8.2
smepti commentedon Jun 18, 2024
same behaviour on php 8.1.19, symfony 6.3.4
Same, word for word
"after enabling JIT, our application start perform better (~10% faster). But, some php-fpm pods (~1 in 20) after start use all available cpu and timeout the requests. After pod restart, everything back to normal. Without JIT everything is ok and cpu utilization is at normal value."
we using k8s and after deploy ~1 in 20 pods use 100% cpu
@andrii29 may be you resolve issue?
smepti commentedon Jun 18, 2024
here is our opcache config
andrii29 commentedon Jun 19, 2024
@smepti nope(
we upgraded php to 8.3, but the issue is still present
kamil-tekiela commentedon Nov 7, 2026
It looks like it got fixed in 8.2.21 (GH-14475) but I am not 100% sure. If you can retests and confirm it would be great. If it's not fixed, please share some more details and we can reopen the issue or post a brand new GH issue.