Skip to content

Add the transport seam and the libcurl default (T10) - #7

Merged
kirre-bylund merged 7 commits into
devfrom
feat/transport
Nov 16, 2026
Merged

kirre-bylund merged 7 commits into
devfrom
feat/transport

Conversation

@kirre-bylund

@kirre-bylund kirre-bylund commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Implements T10: the transport seam and the libcurl default.

The contract

IHttpTransport is pollable, not blocking: Start returns a handle without blocking, Poll reports completion without waiting, Cancel releases. That shape is the whole point of D3 — a blocking Perform called from Update() would stall the host's frame for the length of the transfer.

Three pieces

HeaderMap is public because an adopter implementing the interface has to build one. Insertion-ordered and case-insensitive on lookup, deliberately not a std::map: header order and case matter to some servers and proxies, and a map would silently reorder them and lose the original casing. Set replaces in place, so overwriting Content-Type doesn't move it to the end.

FakeHttpTransport is public rather than test-only, because an adopter writing their own tests needs the same thing. Transfers complete only when the test says so, which is what makes the pollable contract assertable without sleeping.

CurlHttpTransport is curl_multi, which is already pollable — Start adds an easy handle, Poll drives the multi handle once, Cancel removes it. The one thing needing care is that curl's completion notification is separate from the per-handle result, so a transfer is only done once both have happened.

libcurl is optional, not required

A console adopter injects its own transport and must not be forced to build curl; a studio that vendors dependencies itself may already have one. LOOTLOCKER_WITH_CURL defaults ON, but a missing curl is a status message rather than an error — the header guards on the same macro, so an adopter without curl gets a clean build and no dangling declaration.

CI now installs libcurl on all three targets, including the module-boundary job. Without that the curl path would be dead code that only a downstream adopter discovers is broken.

Verification, and one honest gap

  • 12 test functions; all 9 suites pass; zero warnings on MSVC.
  • Breaking the pollable contract fails 4 assertions.
  • Both boundary checks clean; -DLOOTLOCKER_MODULE_ONLY still configures for core and auth;broadcasts.

The gap: libcurl is not installed on this machine, so the curl TU was compiled against a stub header to prove it is syntactically sound. That is weaker than a real build — it proves the code parses and type-checks against curl's API shape, not that it links or works. CI is what closes that gap, and it is the first thing to check on this PR.

Spec

lootlocker/index#1905 records the transport decisions.

IHttpTransport with the pollable contract: Start returns a handle without
blocking, Poll reports completion without waiting, Cancel releases. That shape
is the whole point of D3 - a blocking Perform called from Update() would stall
the host's frame for the length of the transfer.

HeaderMap is public because an adopter implementing the interface has to build
one. Insertion-ordered and case-insensitive on lookup, not a std::map: header
order and case matter to some servers and proxies, and a map would silently
reorder them and lose the original casing. Set replaces in place so overwriting
Content-Type does not move it to the end.

FakeHttpTransport is public rather than test-only, because an adopter writing
their own tests needs the same thing. Transfers complete only when the test says
so, which is what makes the pollable contract assertable without sleeping.

CurlHttpTransport is curl_multi, which is already pollable - Start adds an easy
handle, Poll drives the multi handle once, Cancel removes it. The one thing that
needs care is that curl's completion notification is separate from the per
handle result, so a transfer is only done once both have happened.

libcurl is optional, not required. A console adopter injects its own transport
and must not be forced to build curl, and a studio that vendors dependencies
itself may already have one. LOOTLOCKER_WITH_CURL defaults ON but a missing curl
is a status message rather than an error; the header guards on the same macro so
an adopter without curl gets a clean build and no dangling declaration.

CI now installs libcurl on all three targets, including the module-boundary job.
Without that the curl path would be dead code that only a downstream adopter
discovers is broken.

Verified: 12 test functions, all 9 suites pass, zero warnings. The curl TU was
compiled against a stub header to prove it is syntactically sound, since curl is
not installed on this machine - that is weaker than a real build, and CI is what
closes the gap. Breaking the pollable contract fails 4 assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings November 13, 2026 14:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Approval is blocked by active CMake integration, public-interface, lifetime, global cleanup, and non-blocking DNS issues that prevent reliable and safe use of the curl transport.

5 open issues
What changed in this PR

Adds the T10 pollable HTTP transport seam, including ordered headers, a scriptable fake transport, and an optional libcurl-backed implementation.

Changes:

  • Introduces public IHttpTransport, HeaderMap, FakeHttpTransport, and conditional CurlHttpTransport APIs.
  • Implements header handling and curl_multi transfer lifecycle management.
  • Adds transport tests, CMake integration, optional libcurl detection, CI configuration, and build-directory ignores.
File Description
tests/​test_transport.cpp Adds tests for header behavior and fake transport polling, completion, failure, cancellation, and binary bodies.
tests/​CMakeLists.txt Registers the transport test executable with the test suite.
src/​core/​header_map.cpp Implements ordered, case-insensitive header lookup, replacement, and erasure.
src/​core/​curl_http_transport.cpp Implements the libcurl curl_multi transport and asynchronous transfer lifecycle.
include/​lootlocker/​http_transport.h Defines the public pollable HTTP transport interface and request/response types.
include/​lootlocker/​header_map.h Defines the public ordered, case-insensitive header collection.
include/​lootlocker/​fake_http_transport.h Provides a public scriptable fake transport for tests and adopters.
include/​lootlocker/​curl_http_transport.h Declares the conditional libcurl transport implementation.
CMakeLists.txt Adds transport sources, optional libcurl discovery, and curl target integration.
.gitignore Ignores generated CMake build directories.
.github/​workflows/​build.yml Updates CI configuration for libcurl setup and transport-related build coverage.

Review effort: Lite

Comment thread CMakeLists.txt Resolved
Comment thread src/core/curl_http_transport.cpp Resolved
Comment thread src/core/curl_http_transport.cpp Resolved
Comment thread src/core/curl_http_transport.cpp Resolved
Comment thread CMakeLists.txt Resolved
kirre-bylund and others added 2 commits November 13, 2026 15:30
windows-latest does not ship curl development files, only the CLI, so the
default transport was silently absent on MSVC - the one target where the curl
path was never compiled. vcpkg is the supported way to get headers and a lib on
the runner.

Found by reading the CI log rather than trusting the green check: all six jobs
passed while one of them was building a core without the transport.
LOOTLOCKER_HAVE_CURL was PRIVATE, so a consumer including the public
curl_http_transport.h saw the guard as false and CurlHttpTransport was
undeclared - the documented default transport could not be constructed outside
the library. It is PUBLIC now. Verified with a standalone CMake project that a
PUBLIC define does reach a consumer TU.

The curl target block gated on CURL_FOUND alone, ignoring LOOTLOCKER_WITH_CURL.
CURL_FOUND can already be true in a parent project or a reused configure, so the
documented opt-out did nothing. Both conditions now.

curl_global_init and curl_global_cleanup were per instance. libcurl's global
state is process-wide, so destroying one transport tore it down under any other
one still holding handles - a use-after-free that only appears when two clients
exist. Now refcounted, torn down when the last transport goes.

CURLOPT_POSTFIELDS borrows the pointer, and it pointed into the caller's
Request, which the caller may destroy the moment Start returns. The transfer now
owns a copy for its lifetime. This is the one that would have corrupted request
bodies intermittently and been very hard to diagnose.

The DNS claim was wrong. curl's default resolver is synchronous, so the first
Poll for an uncached hostname can block for the length of the lookup, which
breaks the non-blocking promise the header makes. Rather than pretend, the
contract is narrowed and documented in the interface: the transfer is pollable,
the name resolution is not, and a game that cannot tolerate that should resolve
at startup or inject a transport with an async resolver. A DNS cache timeout is
set so repeat lookups do not pay it again.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@kirre-bylund kirre-bylund left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also miss some documentation/readme information about how the transport is done and how to override it (both build instructions and how the interface should look).

Comment thread CMakeLists.txt Resolved
kirre-bylund and others added 2 commits November 16, 2026 08:49
Introduced when the curl option was added above it. CMake tolerates the
duplicate silently, which is why it survived - the second declaration just
overwrites the first with the same value.
The README had no transport section at all, so an adopter had no way to learn
that the network goes through an injectable interface, how to build without
curl, or what shape an override has to take. That is the seam the whole design
rests on, and it was only discoverable by reading the headers.

Four parts: the interface with its pollable contract, the build instructions
including the opt-out, how to override it, and the DNS caveat.

The interface listing is copied from the real header rather than written from
memory, and the override example was compiled against the actual headers before
committing - a README that does not compile is worse than no README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kirre-bylund

Copy link
Copy Markdown
Contributor Author

I also miss some documentation/readme information about how the transport is done and how to override it (both build instructions and how the interface should look).

You are right, and I missed this because I only read the inline comments and not the review summary. That is a gap in how I was checking feedback, not just a missing section.

Added a ## The transport section to the README with four parts:

  • The interface, with the pollable contract spelled out — Start returns a handle, Poll reports without waiting, Cancel releases, and a transport that blocks in Poll stalls the host frame.
  • Build instructions, including -DLOOTLOCKER_WITH_CURL=OFF and what LOOTLOCKER_HAVE_CURL does to the public header.
  • How to override it, with a worked example assigning to Config::transport.
  • The DNS caveat, since that is a real limitation of the default transport and an adopter should not discover it in production.

Two things I did to stop it drifting: the interface listing is copied from the real header rather than written from memory, and I compiled the override example against the actual headers before committing. A README that does not compile is worse than no README.

Comment thread README.md Resolved
The four seams were raw pointers documented as "not owned". That expressed the
ownership relationship in a comment and nothing more, so an object destroyed
before the client was a dangling dereference the compiler would not catch.

weak_ptr says what was already true: the adopter owns the object, the SDK
observes it. The SDK locks per use and holds the result only for the duration of
that call, so an expired seam is detectable rather than fatal.

weak_ptr rather than shared_ptr, deliberately. A shared_ptr member would keep
the object alive for the client's lifetime, which is ownership rather than
observation - and it would make the expiry check dead code. I got this wrong in
discussion first: locking once at construction and holding the shared_ptr would
have been a shared_ptr with extra steps.

The cost is real and worth stating: an injected object must now be owned by a
shared_ptr, so a stack object no longer works. That is a genuine ergonomic loss
for the common case of a process-lifetime singleton, and it is the price of the
safety.

Config stays a copyable aggregate, which was the reason to avoid unique_ptr and
shared_ptr members - weak_ptr copies fine.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Active critical compatibility and security issues remain, including breaking public injection changes and unsafe redirect handling, alongside untested and incomplete curl error and response-header paths.

2 open issues
3 previously missed issues

In code that hasn't changed since last review

Medium severity Clear response headers when a new status line arrives

src/​core/​curl_http_transport.cpp:42

CURLOPT_HEADERFUNCTION is called for every response in a redirect chain and for interim responses such as 100 Continue. This branch skips each status line but never resets the accumulated map, so headers that existed only on an earlier response remain visible in the final Response (for example, an old Location or proxy header). Clear the response headers when a new status line is received so only the final response's headers are exposed.

Medium severity Add runtime coverage for CurlHttpTransport

src/​core/​curl_http_transport.cpp:146

The new CurlHttpTransport implementation has no runtime coverage: test_transport exercises only FakeHttpTransport and HeaderMap, while CI's curl installation merely causes this translation unit to compile. That leaves real multi-handle completion, response parsing, cancellation, timeout/error paths, and the public curl linkage untested. Add a deterministic loopback-server/injected-resolver test (or equivalent curl-backed integration suite) and run it in CI.

Medium severity Handle curl_multi_perform failures

src/​core/​curl_http_transport.cpp:245

The CURLMcode from curl_multi_perform is ignored. If libcurl reports CURLM_OUT_OF_MEMORY, CURLM_BAD_SOCKET, or another multi-handle error, no CURLMSG_DONE message is guaranteed, so the transfer remains finished == false and every Poll reports it as still running until the executor timeout instead of reporting a transport failure. Handle a non-OK result by completing the affected transfers as failed (and preserving any available error details).

5 resolved since last review

Review effort: Lite

Comment thread include/lootlocker/config.h Resolved
Comment thread src/core/curl_http_transport.cpp Resolved
curl does not scope custom headers to the origin. With CURLOPT_FOLLOWLOCATION
on, every header in the caller's list is replayed to whatever host the Location
header names. The SDK sends x-session-token, so a redirect to an
attacker-controlled host would hand over the session - and a redirect to http://
would do it in clear text.

The API does not redirect in normal operation, so refusing costs a clear error
instead of a silent credential leak. A caller that genuinely needs redirects can
follow them itself, where it can decide which headers survive the hop - only the
caller knows which of its headers are sensitive.

Rejected the alternative of validating Location and stripping sensitive headers
before following: more code, it has to know which headers are sensitive (which
the transport does not), and it fails open if the check is ever wrong. Refusing
fails closed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kirre-bylund
kirre-bylund merged commit a0ea22e into dev Nov 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants