You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
T11: request executor, and the end-to-end transport tests - #8
Implements T11: the queue state machine, retry with back-off, timeouts, and the caller-pumped Update() contract.
The executor (src/core/executor.{h,cpp}) does a frame's worth of work per Update() — poll, retry, promote, deliver — and returns false only when nothing is in flight and nothing is queued.
The retry policy lives in one place:
5xx, 429 and a transport failure are retried; any other 4xx is not.
Retry-After is obeyed exactly when the server sends one.
A non-idempotent request whose response was lost is never retried (R25). A 5xx on a POST is retried, because the server told us it failed — only a lost response is ambiguous.
Time is read through an internal core::Clock so tests advance it by hand and nothing sleeps. It is deliberately not a Config seam.
Two bugs the tests found, both fixed here:
A dead transport only failed in-flight requests, so a queued request with no transport sat in the queue for ever and Update() returned true for ever. A hang is worse than an error; Update() now checks for a transport first.
That failure reported Cancelled rather than Transport — the request did not fail because anyone cancelled it.
The two tests T10 could not write:
A real round trip through the real CurlHttpTransport against a loopback server, asserting the request line, headers and body that actually arrived on the wire. Verified against curl 8.22.0 locally, not merely compiled. Also covers a refused connection and a 302.
A transport-injection test proving Config::transport is consulted rather than the default being hard-wired.
Both were verified by sabotage: removing CURLOPT_HTTPHEADER fails three assertions in the round-trip test, and ignoring Config::transport fails three in the injection test. Neither passes vacuously.
All 11 test suites pass, zero warnings on MSVC, both boundary checks clean, and -DLOOTLOCKER_MODULE_ONLY configures and builds for core and auth;broadcasts.
The queue state machine from D25: retry with exponential back-off, timeouts, the
concurrency cap, and the caller-pumped Update() contract. Update() does a frame's
worth of work - poll, retry, promote, deliver - and returns false only when nothing
is in flight and nothing is queued, which is what lets a loop-less program exit.
The retry policy is the part that bites, so it lives in one place:
- 5xx, 429 and a transport failure are retried; any other 4xx is not.
- Retry-After is obeyed exactly when the server sends one.
- A non-idempotent request whose response was LOST is never retried (R25). A POST
that timed out may have succeeded, and retrying it can double-grant. A 5xx on a
POST is still retried, because the server told us it failed - only a lost
response is ambiguous.
Time is read through an internal core::Clock so tests advance it by hand and nothing
sleeps. It is deliberately not a Config seam: nobody has asked to supply time, and a
seam with no consumer is a seam nobody maintains.
Two bugs found by the tests, both fixed here:
- A dead transport only failed IN-FLIGHT requests, so a queued request with no
transport sat in the queue for ever and Update() returned true for ever. That is
a hang, which is worse than an error. Update() now checks for a transport first
and fails everything with Transport.
- That failure reported Cancelled, which is the wrong code: the request did not
fail because anyone cancelled it, it failed because there is no way to send it.
The two tests T10 could not write, both here:
- A real round trip through the real CurlHttpTransport against a loopback server,
asserting the request line, headers and body that actually arrived on the wire,
and the response the SDK decoded. Verified against curl 8.22.0, not just
compiled. Also covers a refused connection (status 0) and a 302 (reported as-is,
not followed).
- A transport-injection test proving Config::transport is consulted rather than the
default being hard-wired. It was a declared field that nothing read.
Both were verified by sabotage: removing CURLOPT_HTTPHEADER fails three assertions in
the round-trip test, and ignoring Config::transport fails three in the injection test.
Neither passes vacuously.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Several unresolved moderate and critical issues remain in client request handling, retry safety, transport lifecycle, response polling, and test-server portability and synchronization.
Implements T11 by adding a caller-pumped request executor with queueing, retries, backoff, timeouts, transport handling, and callback delivery, plus deterministic executor and end-to-end transport tests.
Changes:
Added the executor state machine, clock abstraction, retry policy, timeout handling, and transport lifecycle management.
Wired configurable and default transports into the client and documented the update and queue behavior.
Added executor unit tests, loopback curl tests, transport injection coverage, and build integration.
File
Description
tests/test_transport_e2e.cpp
Adds loopback end-to-end tests for real curl requests, transport failures, redirects, and injected transports.
tests/test_executor.cpp
Adds deterministic coverage for executor scheduling, retries, timeouts, queueing, logging, and transport behavior.
tests/loopback_server.h
Provides the socket-based loopback HTTP server used by the transport tests.
tests/CMakeLists.txt
Registers the executor and end-to-end transport test targets.
src/core/executor.h
Defines the executor API, queue state, pending requests, and completion types.
src/core/executor.cpp
Implements queue processing, polling, retries, backoff, timeouts, callback delivery, and transport failure handling.
src/core/clock.h
Adds the internal monotonic clock abstraction used by the executor.
src/core/client.cpp
Constructs the client executor and resolves configured or default transports and loggers.
README.md
Documents caller-pumped updates, queue behavior, retries, and idempotency policy.
include/lootlocker/client.h
Exposes the public client update and transport-management APIs.
CMakeLists.txt
Adds the executor and client implementations to the core library.
Two portability bugs, both caught by CI rather than by me:
- htonl and ntohs are MACROS on macOS (__DARWIN_OSSwapInt32), so ::htonl(...)
expands to ::__DARWIN_OSSwapInt32(...) and cannot be qualified. Dropped the
::. This is the kind of thing that only shows up on the platform you are not
building on.
- m_wsa_started is Windows-only, so clang warned about an unused private field.
Guarded on _WIN32 rather than left alone: a warning in the test suite is a
warning someone eventually silences by deleting the field.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Linux CI job sat in the test step for twenty minutes. The cause is in the test
server, not the SDK: on Linux, closing a listening socket does NOT unblock a thread
already blocked in accept(), so Stop() hung on join() for ever. Windows and macOS do
unblock it, which is exactly why only one of the three targets hung.
shutdown() before close() makes the pending accept return, which is the portable way
to wake it.
Also added a 120 s CTest timeout to the two new suites. A hang should fail the job in
two minutes, not block CI until someone notices - which is what happened here.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All eight were real. The two that mattered most:
- Poll was called TWICE per Update for a finished transfer: once to detect
completion, then again to collect the response. The interface documents "called
once per Update()", and a transport is entitled to consume its completion on the
first poll - so the second poll would lose the response and report an empty
result. The response is now kept from the first poll. Sabotaging this fix breaks
six assertions across three tests.
- SetTransport replaced the transport under in-flight entries, which hold only a
transfer id. That id means nothing to a different transport, so the next Update
would poll and cancel it on the new one and the old transfer would never be
cancelled. Now refused while anything is in flight, returning false and logging
why; the caller drains first, because only the caller knows whether waiting is
acceptable.
The rest:
- Back-off overflowed signed int64 on the multiply before the cap was checked,
which is undefined behaviour rather than a late clamp. The cap is now checked
before multiplying.
- Retry-After was accumulated in a signed int from server-controlled input, so a
large header overflowed. Parsed into a checked 64-bit accumulator and clamped.
- Retry-After: 0 is a valid instruction to retry immediately, but the > 0 check
treated it as absent and applied the exponential back-off instead. Error now
carries retry_after_present so "the server said zero" is distinguishable from
"there was no header".
- An invalid config made Update() return early, so a handler submitted through
core::GetExecutor was stranded for ever. The executor now carries the config
error and rejects through the normal path, so the handler runs on the next
Update.
- The loopback server's Stop() closed the listening socket while Run() was blocked
in accept() on it: a data race, and not a portable wake-up. Replaced with a
self-connection, which is portable and removes the race.
- test_transport_e2e now links Threads::Threads, because the loopback server runs
on its own thread and pthreads is not folded into libc everywhere.
Five new tests cover the new behaviour, and the Retry-After: 0 fix was verified by
sabotage as well.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Several active correctness and portability issues remain, including incorrect whole-request timeout handling, stranded callbacks, incomplete Retry-After and HTTP-status handling, Windows socket narrowing, and a flaky refused-connection test.
The legal HTTP-date form of Retry-After is treated as absent here, so a response such as Retry-After: Wed, 21 Oct 2015 07:28:00 GMT falls back to exponential back-off instead of obeying the server's instruction. That contradicts the PR/README promise that Retry-After is obeyed exactly; parse both allowed forms or narrow that documented contract.
Update can report no work while callback-submitted requests remain
src/core/executor.cpp:360
FailAll() delivers handlers before this return, and those handlers are allowed to submit new requests. If a callback submits while the transport is unavailable, m_entries is non-empty here but Update() still returns false, so a while (Update()) driver stops with that request stranded. Return the same pending-work condition used at the normal end of Update().
Retries reset the whole-request timeout budget
src/core/executor.cpp:479
started_at_ms is reset when every retry is promoted, while the timeout check compares against that per-attempt timestamp. request_timeout_seconds is documented as a whole-request budget in both QueueConfig and IHttpTransport::Request, so a request can run for roughly (retries + 1) * timeout before it is finally delivered. Keep a deadline from the initial attempt and use the remaining budget for retries (or stop retrying once it expires).
Refused-connection test assumes port 1 is unused
tests/test_transport_e2e.cpp:137
Using 127.0.0.1:1 as the refused-connection fixture assumes that port is unused on every test host. A service, sandbox proxy, or local security tool can legally listen there, turning this into a real HTTP response or a different failure and making the test flaky. Reserve an ephemeral loopback port (or otherwise create a guaranteed-closed endpoint) instead of relying on a conventional low port.
Two more review findings, both real.
- A 3xx left completion.error as None, so a 302 was delivered to callers as a
SUCCESS. That matters specifically because redirects are disabled in the default
transport (D3): the redirect arrives as a plain response, and the module layer
would try to decode the redirect body as a normal API response and report a
confusing decode error instead of "the server redirected us". Every non-2xx is
now an error; 429 keeps its own code, and 204 is still a success because it is a
2xx. Verified by sabotage: restoring >= 400 fails the new test.
- The loopback server narrowed every socket handle to int. On 64-bit Windows
SOCKET is a pointer-sized UINT_PTR, so a handle above 32 bits would be truncated
and the call would operate on the wrong socket. Introduced a SocketHandle alias
(SOCKET on Windows, int on POSIX) and a CloseSocket helper, and removed every
narrowing cast. The one remaining int cast is a byte count, not a handle.
Two new tests: a 302 is an error and is not retried, and a 204 is still a success -
the second exists because widening the error range to "everything not 2xx" is exactly
the change that would break EndSession.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements T11: the queue state machine, retry with back-off, timeouts, and the caller-pumped
Update()contract.The executor (
src/core/executor.{h,cpp}) does a frame's worth of work perUpdate()— poll, retry, promote, deliver — and returnsfalseonly when nothing is in flight and nothing is queued.The retry policy lives in one place:
Retry-Afteris obeyed exactly when the server sends one.Time is read through an internal
core::Clockso tests advance it by hand and nothing sleeps. It is deliberately not aConfigseam.Two bugs the tests found, both fixed here:
Update()returnedtruefor ever. A hang is worse than an error;Update()now checks for a transport first.Cancelledrather thanTransport— the request did not fail because anyone cancelled it.The two tests T10 could not write:
CurlHttpTransportagainst a loopback server, asserting the request line, headers and body that actually arrived on the wire. Verified against curl 8.22.0 locally, not merely compiled. Also covers a refused connection and a 302.Config::transportis consulted rather than the default being hard-wired.Both were verified by sabotage: removing
CURLOPT_HTTPHEADERfails three assertions in the round-trip test, and ignoringConfig::transportfails three in the injection test. Neither passes vacuously.All 11 test suites pass, zero warnings on MSVC, both boundary checks clean, and
-DLOOTLOCKER_MODULE_ONLYconfigures and builds forcoreandauth;broadcasts.