Repository navigation
[humble] Fix starvation in mutually exclusive callback groups of the MultiThreadedExecutor - #3417
Open
HarunTeper wants to merge 1 commit into
Open
HarunTeper wants to merge 1 commit into
HarunTeper wants to merge 1 commit into
Conversation
…adedExecutor A handle that is ready while its mutually exclusive callback group is in use is skipped and dropped before the next wait. Its callback group is left out of the wait set while it is in use, so once the group is free again all of its ready handles are collected at the same time and the first one in the list is picked every time. With callbacks that take longer than their period the other entities of the group are never executed. Let the default memory strategy keep the skipped handles and execute them first, in the order in which they were skipped, when their callback group can be taken from again. This is done in a class that is local to the library, so the public classes keep their layout. Executors that are given another memory strategy behave as before. Signed-off-by: HarunTeper <harun.teper@tu-dortmund.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Humble version of #2702. Fixes #2360 and #2645 on humble.
Problem
If two entities share a mutually exclusive callback group and both are ready whenever the
group becomes free, the MultiThreadedExecutor always picks the one that comes first in the
wait set. The other one is never executed. Humble has no EventsCBGExecutor, so there is no
executor to switch to.
Change
Same idea as in #2702: the ready entities that were skipped because their callback group
was in use are remembered in the order in which they were skipped, and they are executed
first when their group is free again.
Humble has no
ExecutorImplementation, so this is done in the memory strategy:DefaultMemoryStrategyinmemory_strategies.cpp, derived fromAllocatorMemoryStrategy<>.create_default_strategy()returns it. It is local to thelibrary.
allocator_memory_strategy.hpp: the handle vectors becomeprotectedinstead of
private.Executor,MemoryStrategyor any other class layout, no new virtualfunction, no additional mutex. Executors that are given another memory strategy behave
as before.
Tests
mutually_exclusive_timers_starvationintest_multi_threaded_executor.cpp,the same as in Fix starvation in mutually exclusive callback groups of the MultiThreadedExecutor #2702: two 5 ms timers with 20 ms callbacks in one group, each must get
at least 5 of 20 executions. Without the change one timer gets all 20.
are clean.
Measurements
Humble container with the released binaries and Fast DDS,
rclcppbuilt from sourcewithout and with the change. The method is described in #2702.
A fast timer (5 ms period, 20 ms callback) shares a group with a victim, which is either a
subscription that gets a message every 100 ms or a 100 ms timer. 5 runs of 30 s, "wait" is
the time until the victim's callback starts:
benchmark_executor, MultiThreadedExecutor, 3 rounds, change against before:spin_someadd_noderemove_nodespin_node_until_future_completeThe number of allocations is the same in all four scenarios.
Autoware reference system, 5 runs of 10 minutes:
ABI
abidiffbetween the two builds oflibrclcpp.soexits with "incompatible", although norclcpp type, size, virtual table or function differs. It lists 5 removed and 5 added
functions. The removed ones are the
std::make_sharedcontrol block forAllocatorMemoryStrategy<>, which the library no longer instantiates because it nowcreates the derived class. The added ones are
std::vector<...>::_M_erasefor the fivehandle types. Both are weak template instantiations that every binary using them emits
for itself, so a program that creates an
AllocatorMemoryStrategy<>has its own copy.What it does not do
As in #2702 the entities of a group do not strictly take turns. An entity that is blocked
goes before the others of its group, but an entity that just ran can run once more before
a sibling that became ready in the meantime (longest wait 40 ms instead of 20 ms in the
table above). #2702 describes a turn rule that closes this gap. A humble version of it
exists, I would propose it after the discussion there.