test_dtrace should try to terminate SystemTap gracefully before SIGKILL

[stratakis]

Bug report

Bug description:

While checking out gh-156701 where the systemtap compilation was taking more than 10 seconds (hence an increase to timeout is required) I noticed also that the buildbot that I was testing at had too many D-state processes. That's possibly due to TraceBackend.trace() sending a SIGKILL immediately to the tracer process group when a timeout happens.

This prevents SystemTap and its children from performing normal cleanup. As mentioned in gh-156701 this left stap* temporary files behind, on the Fedora ppc64le buildbots, but also due repeated SystemTap timeouts, the buildbots had many orphaned staprun processes in Dstate, and the machines waited for them during shutdown.

From the system tap documentation: https://sourceware.org/systemtap/man/stap.1.html#abnormal-termination

One should avoid killing the stap process forcibly, for example with SIGKILL, because the stapio process (a child process of the stap process) and the loaded module may be left running on the system. If this happens, send SIGTERM or SIGINT to any remaining stapio processes, then use rmmod to unload the systemtap module.

Increasing the usability timeout should fix the premature timeout #156720, but genuine timeout still needs to be handled properly with SIGTERM and then if that fails a SIGKILL.

CPython versions tested on:

CPython main branch, 3.16, 3.15

Operating systems tested on:

Linux

Linked PRs

• gh-156722: Terminate tracing processes gracefully on timeout for test_dtrace #156753