Skip to content

batching: Fix missed batch result watches - #152

Merged
mnasiadka merged 1 commit into
stackhpc/2025.1from
2025.1-fix-batching
Oct 16, 2026
Merged

mnasiadka merged 1 commit into
stackhpc/2025.1from
2025.1-fix-batching

Conversation

@bbezak

@bbezak bbezak commented Oct 16, 2026

Copy link
Copy Markdown
Member

A batch result can be written to etcd while the waiting caller still does not receive the watch event. This has been seen with Cumulus 5 trunk configuration using many VLAN subports, where the result key was written but the waiting caller still timed out waiting for the watch event.

When this happens, the caller waits until the full batch timeout and fails even though the result key exists. Use shorter watch waits and check etcd between them so missed watch events do not cause false batch timeouts.

Conflicts:
networking_generic_switch/batching.py
2025.1 has no "import threading"; added only "import time".

Assisted-By: openai codex gpt 5.5
Closes-Bug: #2152047

Change-Id: Ifa6b4a3ec447712c0ccf9fe0b79f8236be0ce33e

(cherry picked from commit 2306676)

A batch result can be written to etcd while the waiting caller
still does not receive the watch event. This has been seen with
Cumulus 5 trunk configuration using many VLAN subports, where the
result key was written but the waiting caller still timed out
waiting for the watch event.

When this happens, the caller waits until the full batch timeout and
fails even though the result key exists. Use shorter watch waits and
check etcd between them so missed watch events do not cause false
batch timeouts.

Conflicts:
  networking_generic_switch/batching.py
    2025.1 has no "import threading"; added only "import time".

Assisted-By: openai codex gpt 5.5
Closes-Bug: #2152047

Change-Id: Ifa6b4a3ec447712c0ccf9fe0b79f8236be0ce33e
Signed-off-by: Bartosz Bezak <bartosz@stackhpc.com>
(cherry picked from commit 2306676)
@bbezak
bbezak requested a review from a team as a code owner October 16, 2026 11:13
@mnasiadka
mnasiadka merged commit 571d1d3 into stackhpc/2025.1 Oct 16, 2026
4 checks passed
@mnasiadka
mnasiadka deleted the 2025.1-fix-batching branch October 16, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants