Tracing Through ProcessPoolExecutor

[Mr0grog]

I have an app that uses ProcessPoolExecutor to run CPU-heavy work in parallel and to isolate it from the rest of the service it’s inside of. While adding tracing to it, I discovered that spans from code that the executor runs don’t ever show up in Sentry.

For example:

import sentry_sdk
from concurrent.futures import ProcessPoolExecutor

sentry_sdk.init(traces_sample_rate=1.0)
pool = concurrent.futures.ProcessPoolExecutor()

def worker(message):
    # This span never shows up.
    with sentry_sdk.start_span(op="my.task"):
        return f"Hello {message}")

with sentry_sdk.start_transaction(name="My Job", op="my.job"):
    future = pool.submit(worker, "world")
    print(future.result())

That mostly makes sense (it is running in a different process, after all). As a workaround, I used Sentry’s distributed tracing tooling (get_traceparent(), get_baggage(), and continue_trace()) to solve the problem.

But I would love it if there was an integration to automatically solve this. For example, I’m thinking something like:

_process_pool_submit = concurrent.futures.ProcessPoolExecutor.submit

def sentry_subprocess_wrapper(fn, trace):
    transaction = sentry_sdk.continue_trace(trace, op='executor.run', name='executor.run')
    with sentry_sdk.start_transaction(transaction):
        return fn()

def sentry_submit(self, fn, /, *args, **kwargs):
    trace = {
        'sentry-trace': sentry_sdk.get_traceparent(),
        'baggage': sentry_sdk.get_baggage(),
    }
    bound_fn = functools.partial(fn, *args, **kwargs)
    return _process_pool_submit(self, sentry_subprocess_wrapper, bound_fn, trace)

concurrent.futures.ProcessPoolExecutor.submit = sentry_submit