You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
T11d: RequestId, LL-Request-Id, and deduplication - #9
Implements T11d: RequestId as a public value type, sent as LL-Request-Id.
The type. A uint64 wrapper, not a string — a string id would allocate on every call, and this is a game SDK's hot path. ToString() is hand-rolled decimal rather than std::to_string, which goes through locale-aware formatting for a value that is always plain digits.
The counter is a process-wide static atomic on RequestId, minted only through RequestId::Next(). Process-wide rather than per-client because two clients in one process are legal (D20), and a per-client counter would let both mint id 5 — making a support ticket quoting "request 5" ambiguous. It is a CAS loop that skips 0, so a wrap cannot produce an invalid id.
The header is stamped at send time, so a retry re-sends the same logical request with the same id — which is what makes a server log show one id across attempts rather than one per attempt.
Error::request_id is populated on every delivered failure, not just in the log. A support ticket is usually raised from a failure, so leaving it empty there defeated the purpose of the field.
No deduplication or grouping, at all. An earlier commit on this branch merged identical idempotent requests. That is removed, because the API is not guaranteed to be idempotent: "add 5 gold" performed five times must yield 25, not 5. Merging would silently grant a fifth of what the caller asked for, and the SDK cannot tell that apart from a harmless repeated read. The owner built this in the Unity SDK and turned it off for the same reason.
No idempotent flag either. It existed to close R25 (never retry a non-idempotent request whose response was lost). The owner's call is that the other SDKs do not do this and we should not either — the flag moves a correctness decision onto the caller, and a caller who forgets it gets a request that silently never retries. A lost response is now retried whatever the method, so a POST that actually succeeded may be sent twice. That is recorded as an accepted risk in the spec's risk table, not a closed one.
Verification: 9 new tests. All 11 suites pass in both curl and no-curl configurations, zero warnings, boundary checks clean. Sabotage-tested: reintroducing a merge fails four assertions, removing request_id fails two, and a non-atomic counter fails the concurrency test.
One thing worth your eye: Submit returns an invalid RequestId on rejection rather than 0. That is a public API change, and it is the shape T14–T18 will be written against.
Every async call now returns a RequestId instead of void (D25). It is a value type
wrapping a uint64, not a string, because a string id would allocate on every call and
this is a game SDK's hot path.
The id goes on the wire as LL-Request-Id so a support ticket can be matched to a
server log line. It is stamped at SEND time, not on the stored request, for two
reasons: the stored request is what deduplication compares, so stamping it would make
every request unique and dedupe would never fire; and a retry re-sends the same
logical request, so it carries the same id, which is what makes a server log show one
id across attempts rather than one per attempt.
Deduplication, with the rule I proposed and the owner approved:
- Only an idempotent request is ever merged. Merging two writes would silently drop
one, which is the same class of bug as retrying a lost POST (R25).
- "Identical" means same method, same URL, same headers in the same order, and same
body byte for byte. Headers are compared as an ordered list rather than a set,
because order and case are part of what goes on the wire.
- The existing id is returned, which is how the caller can tell it happened, and
every handler on the entry is called with the same completion.
Two bugs found by the tests while writing this:
- Stamping LL-Request-Id on the stored request made dedupe impossible, because no
incoming request could ever match a stamped one. Moved to send time.
- Three existing tests submitted identical GETs and asserted three separate
requests. That is now correctly one request, so those tests use distinct URLs -
they were measuring the concurrency cap, not dedupe.
Verified by sabotage: disabling the dedupe lookup fails five assertions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Two active moderate issues remain in src/core/executor.cpp: completion errors do not receive the request ID needed for correlation, and duplicate detection performs costly linear scans over queued and in-flight requests.
This PR introduces a public RequestId value type and integrates it into the executor for request headers, logging, retries, and idempotent-request deduplication.
Changes:
Added RequestId generation, validation, ordering, and decimal formatting.
Added LL-Request-Id stamping at send time and preserved IDs across retries.
Added deduplication for identical idempotent requests with shared completion handling.
Updated executor tests and build configuration for the new request identity behavior.
File
Description
tests/test_executor.cpp
Updates executor tests and adds coverage for request IDs, retries, headers, and deduplication.
src/core/request_id.cpp
Implements allocation-free decimal conversion for RequestId::ToString().
src/core/executor.h
Updates executor interfaces and state to use RequestId and support multiple completion handlers.
src/core/executor.cpp
Implements request ID assignment, header stamping, logging, retries, request comparison, and deduplication.
include/lootlocker/request_id.h
Defines the public RequestId value type and its validity, comparison, and formatting APIs.
The owner built this in the Unity SDK and then turned it off, and the reason is
decisive: the API is not guaranteed to be idempotent. "Add 5 gold" performed five times
must yield 25, not 5. Merging those five into one would silently grant a fifth of what
the caller asked for, and the SDK cannot tell that case apart from a harmless repeated
read.
My earlier rule - merge only when the caller says the request is idempotent - was
worse than no rule, because it moved the decision onto a flag the caller has to get
right, and a wrong flag loses a grant silently. There is no version of this the SDK can
decide correctly, so it does not decide at all.
Every submission is now its own request, always. RequestId stays: it is still the
correlation id for LL-Request-Id and the handle Cancel will take. Only the merging goes.
Two tests replace the four dedupe tests, and they assert the opposite of what the old
ones did: two identical GETs are both sent, and two identical writes are both sent.
Verified by sabotage - reintroducing a merge fails four assertions across two tests.
The distinct-URL workarounds I added to two existing tests are reverted, since
identical URLs are no longer a problem.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Four review comments, all acted on.
Owner, on PendingRequest::idempotent: "The other SDKs don't have this. We should be ok
skipping it." Dropped, along with the R25 guard it fed. A lost response is now retried
whatever the method, which means a POST that actually succeeded may be sent twice. That
is a deliberate reversal of R25, so it is recorded as an ACCEPTED RISK in the spec
rather than quietly deleted - the flag moved a correctness decision onto the caller, and
a caller who forgot it got a request that silently never retried.
Owner, on the id counter: it should live on RequestId as a static atomic with wrap
handling, not on the executor. Done. It is process-wide rather than per-client, because
two clients in one process are legal (D20) and a per-client counter would let both mint
id 5, making a support ticket quoting "request 5" ambiguous. 0 is skipped so a wrap
cannot produce an invalid id.
Copilot, on Completion::error.request_id: it was never populated, so a caller holding a
failed result had no support id even though Error documents the field as the id sent in
LL-Request-Id. Set in FinishOrRetry and in FailAll. A support ticket is usually raised
from a failure, not a success, so leaving it empty on failure defeated the purpose.
Verified by sabotage: removing it fails both new tests.
Copilot, on the dedupe scan being quadratic: moot, dedupe is gone.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Approval is blocked by unresolved correctness and contract issues, including a non-unique concurrent request-ID wraparound, inconsistent retry semantics, and several stale or warning-producing code paths.
id is never read in this test after the transport lookup was changed to Order().back(). With this repository's warning flags (-Wall -Wextra, and MSVC /W4), the new declaration produces an unused-variable warning; remove the local or assert on it.
This issue also appears in the following locations of the same file:
line 528
line 560
line 582
line 613
Remove unused RequestId local
tests/test_executor.cpp:682
With -Wall -Wextra this RequestId local is now unused: the test uses the transport order for completion. Remove the local so the test suite keeps the repository's zero-warning build.
This issue also appears on line 716 of the same file.
- The README still documented the removed idempotent flag and claimed a lost response
is never retried. It now states the accepted risk and the no-dedupe rule. The PR
description was stale in the same way and is updated separately.
- The LL-Request-Id comment still justified send-time stamping by deduplication, which
no longer exists. It now gives the only reason that is still true: a retry re-sends
the same logical request with the same id.
- Entry::on_done is a single handler again, not a vector. Requests are never grouped,
so an entry has exactly one caller; the vector added an allocation per submission and
implied an invariant that no longer exists.
- RequestId::Next is a CAS loop that skips 0, rather than fetch_add plus a fix-up.
On that last one: Copilot's claim that the fetch_add version was racy is wrong. fetch_add
hands out each value exactly once, so only one thread could ever observe 0 and take the
next value - two threads cannot both return 1. I changed it anyway, because the CAS loop
makes "never 0" an invariant of the loop rather than a special case that needs a
paragraph to justify.
The concurrency test needed two attempts to be worth anything. At 4x2000 with push_back
it passed against a deliberately non-atomic counter, because the allocator dominated and
the race window was too small. It is now 8x20000, threads start on a barrier, and the
vectors are pre-sized so the increment is the hot path - which does catch it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements T11d:
RequestIdas a public value type, sent asLL-Request-Id.The type. A
uint64wrapper, not a string — a string id would allocate on every call, and this is a game SDK's hot path.ToString()is hand-rolled decimal rather thanstd::to_string, which goes through locale-aware formatting for a value that is always plain digits.The counter is a process-wide static atomic on
RequestId, minted only throughRequestId::Next(). Process-wide rather than per-client because two clients in one process are legal (D20), and a per-client counter would let both mint id 5 — making a support ticket quoting "request 5" ambiguous. It is a CAS loop that skips 0, so a wrap cannot produce an invalid id.The header is stamped at send time, so a retry re-sends the same logical request with the same id — which is what makes a server log show one id across attempts rather than one per attempt.
Error::request_idis populated on every delivered failure, not just in the log. A support ticket is usually raised from a failure, so leaving it empty there defeated the purpose of the field.No deduplication or grouping, at all. An earlier commit on this branch merged identical idempotent requests. That is removed, because the API is not guaranteed to be idempotent: "add 5 gold" performed five times must yield 25, not 5. Merging would silently grant a fifth of what the caller asked for, and the SDK cannot tell that apart from a harmless repeated read. The owner built this in the Unity SDK and turned it off for the same reason.
No
idempotentflag either. It existed to close R25 (never retry a non-idempotent request whose response was lost). The owner's call is that the other SDKs do not do this and we should not either — the flag moves a correctness decision onto the caller, and a caller who forgets it gets a request that silently never retries. A lost response is now retried whatever the method, so a POST that actually succeeded may be sent twice. That is recorded as an accepted risk in the spec's risk table, not a closed one.Verification: 9 new tests. All 11 suites pass in both curl and no-curl configurations, zero warnings, boundary checks clean. Sabotage-tested: reintroducing a merge fails four assertions, removing
request_idfails two, and a non-atomic counter fails the concurrency test.One thing worth your eye:
Submitreturns an invalidRequestIdon rejection rather than0. That is a public API change, and it is the shape T14–T18 will be written against.