Add multi-threaded compilation and evaluation tests and resolve compilation deadlock in CEL Python. - #103
Open
copybara-service[bot] wants to merge 1 commit into
Open
Add multi-threaded compilation and evaluation tests and resolve compilation deadlock in CEL Python.#103copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
…lation deadlock in CEL Python. This change adds cel_parallel_test.py to benchmark and validate compiling and evaluating a diverse set of CEL expressions concurrently across multiple worker threads using concurrent.futures.ThreadPoolExecutor as well as sequentially. In addition, this resolves an AB-BA deadlock between the Python GIL and Protobuf DescriptorPool C++ mutex during concurrent compilation by releasing the GIL in PyCelEnv::Compile and acquiring the GIL via py::gil_scoped_acquire on-demand in PyDescriptorDatabase callbacks, while protecting PyCelEnvInternal with a mutex. Test duration metrics: - Multi-threaded compilation (1,000 iterations): ~248 ms - Sequential compilation (1,000 iterations): ~394 ms - Multi-threaded evaluation (10,000 iterations): ~782 ms - Sequential evaluation (10,000 iterations): ~434 ms PiperOrigin-RevId: 968187521
copybara-service
Bot
force-pushed
the
test_968187521
branch
from
August 21, 2026 21:11
71a11bd to
0beaf42
Compare
copybara-service
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add multi-threaded compilation and evaluation tests and resolve compilation deadlock in CEL Python.
This change adds cel_parallel_test.py to benchmark and validate compiling and
evaluating a diverse set of CEL expressions concurrently across multiple worker
threads using concurrent.futures.ThreadPoolExecutor as well as sequentially.
In addition, this resolves an AB-BA deadlock between the Python GIL and Protobuf
DescriptorPool C++ mutex during concurrent compilation by releasing the GIL
in PyCelEnv::Compile and acquiring the GIL via py::gil_scoped_acquire on-demand
in PyDescriptorDatabase callbacks, while protecting PyCelEnvInternal with a mutex.
Test duration metrics: