h1: replace Beast with Boost.HTTP (cppalliance/http)

.cspell.json

@@ -1,16 +1,13 @@
 {
     "words": [
-        "dedent",
-        "doxygen",
-        "literalinclude",
-        "toctree",
         "64kminus1",
         "ALTSVC",
         "anyhttp",
         "ASAN",
         "ASIO",
         "autotuning",
         "AWSLC",
+        "BINDIR",
         "boringssl",
         "callgrind",
         "capy",
@@ -22,24 +19,30 @@
         "datagrams",
         "dcid",
         "dcidlen",
+        "dedent",
         "demuxed",
+        "DESTDIR",
         "docroot",
+        "doxygen",
         "ECDH",
         "Eisenlohr",
+        "endforeach",
         "epoll",
         "gencert",
         "gmock",
         "GOAWAY",
         "GTEST",
         "HCAT",
         "HPACK",
+        "INCLUDEDIR",
         "IOCP",
         "IWYU",
         "jemalloc",
         "libcrypto",
         "LIBCXX",
         "linearisation",
         "linearise",
+        "literalinclude",
         "logd",
         "loge",
         "logi",
@@ -48,6 +51,7 @@
         "malloc",
         "marp",
         "mawk",
+        "misaligns",
         "mlogd",
         "mloge",
         "mlogi",
@@ -57,10 +61,12 @@
         "nghttpd",
         "nghttpd's",
         "ngtcp",
+        "noexcept",
         "nullptr",
         "ossl",
         "osslclient",
         "osslserver",
+        "Postconditions",
         "PROFDATA",
         "PROFRAW",
         "psedoc",
@@ -85,8 +91,11 @@
         "syscalls",
         "testcases",
         "TLSEXT",
+        "toctree",
         "TSAN",
-        "WOULDBLOCK"
+        "UBSAN",
+        "WOULDBLOCK",
+        "zstd"
     ],
     "ignorePaths": [
         ".devcontainer/*",

.github/instructions/anyhttp.instructions.md

@@ -4,7 +4,7 @@ applyTo: '**'
 AnyHTTP is an experimental, low-level C++23 HTTP client and server library with a type-erasing
 interface: one API for HTTP/1.1, HTTP/2 and HTTP/3 (QUIC), hence the *any* in the name -- plus
 plain QUIC streams next to HTTP/3. None of the protocols are implemented from scratch; it wraps
-Boost.Beast (parser and serializer), nghttp2, and ngtcp2/nghttp3.
+Boost.HTTP (cppalliance/http: parser, serializer and header fields), nghttp2, and ngtcp2/nghttp3.
 
 Everything is asynchronous and written with C++20 coroutines. Every operation is a coroutine
 (`read_some`, `write`, `write_eof`, `submit`, `get_response`, `connect`, `get`) that yields a
@@ -24,7 +24,7 @@ the runtime only through `anyhttp/runtime.hpp`, `anyhttp/net.hpp` and `anyhttp/f
 includes nothing else from Boost.Asio but `<boost/asio/buffer.hpp>`.
 
 Each protocol has a directory of its own, again under both `include/anyhttp/` and `src/`: `h1/`
-(Beast), `h2/` (nghttp2), `quic/` (ngtcp2) and `h3/` (nghttp3, on top of `quic/`). Only files in
+(Boost.HTTP), `h2/` (nghttp2), `quic/` (ngtcp2) and `h3/` (nghttp3, on top of `quic/`). Only files in
 `h2/` include `<nghttp2/*>`, only `quic/` and `h3/` `<ngtcp2/*>`, only `h3/` `<nghttp3/*>`.
 Generic code reaches a protocol through that protocol's `backend.hpp`. Keep these boundaries.
 

CLAUDE.md

@@ -13,6 +13,15 @@ benchmark with the latter, never with `build/`. Both use clang, which builds aga
 the libraries in `/opt/libc++` (`USE_LIBCXX`, `LIBCXX_ROOT`); `build-gcc/` is GCC with libstdc++
 and `/usr/local`. Keep `build/` on clang: clangd reads its `compile_commands.json`.
 
+HTTP/1.1 is [Boost.HTTP](https://github.com/cppalliance/http) (cppalliance/http, not in a Boost
+release yet), and its `fields` are anyhttp's `Fields`. Every tree FetchContents it, and capy with
+it, at a pinned SHA of `develop`; `cmake/http.cmake` builds its protocol core as `anyhttp_http`,
+with the patches in `cmake/patches/` applied, each of them proposed upstream. Its parser and
+serializer are sans-I/O: `h1/io.hpp` has the loops that drive them on the runtime layer, as
+Boost.HTTP's own (`parser::read()`, `serializer::sink`) need a capy stream. Leave chunking and
+`Content-Length` to the library, even where it copies: a body is copied into the serializer's
+buffer, there is no zero-copy path yet.
+
 `build-corosio/` is the COROSIO I/O runtime (`-DANYHTTP_API=COROSIO -DTLS_LIBRARY=OpenSSL`, see
 [docs/corosio-port-plan.md](docs/corosio-port-plan.md)); capy and corosio are FetchContent'd at
 pinned SHAs. Both runtimes build the same library sources and test files; what belongs to one
@@ -139,7 +148,7 @@ buffer (see `WriteMode`), and `build-h2v2-asan/` is the version 2 ASAN tree for
 
 ## Source layout
 
-Each protocol lives in a directory of its own, `h1/` (Beast/HTTP/1.1), `h2/` (nghttp2), `quic/`
+Each protocol lives in a directory of its own, `h1/` (Boost.HTTP), `h2/` (nghttp2), `quic/`
 (ngtcp2) and `h3/` (nghttp3, on top of `quic/`), in both `include/anyhttp/` and `src/`. **Only
 files in `h2/` may include `<nghttp2/*>`, only files in `h2v2/` `<nghttp2v2/*>`, only files in
 `quic/` and `h3/` `<ngtcp2/*>`, only files in `h3/` `<nghttp3/*>`.** What belongs to one async
@@ -206,8 +215,9 @@ Only objects in `h2/`, `quic/` and `h3/` of the library may appear in the first
 in `h3/` in the second: that is what proves the QUIC transport has no HTTP in it. (Test objects
 may: the h2c upgrade and formatter tests drive nghttp2 by hand.)
 
-**Generic and protocol code include nothing from Boost.Asio but `<boost/asio/buffer.hpp>`**, which
-Beast's parser, serializer and `Fields` bring anyway, and nothing from capy or corosio. They reach
+**Generic and protocol code include nothing from Boost.Asio but `<boost/asio/buffer.hpp>`**, and
+nothing from capy or corosio but what Boost.HTTP brings along: its parser and serializer are
+written in terms of capy's buffers, which `h1/io.hpp` converts at the boundary. They reach
 the runtime through `anyhttp/runtime.hpp`, `anyhttp/net.hpp` and `anyhttp/formatter.hpp`, which
 include the `asio/` or `corosio/` header of the build's runtime. Addresses and endpoints are the
 runtime's own (`IpAddress`, `TcpEndpoint`, `UdpEndpoint`); where the two differ, `net.hpp` declares

CMakeLists.txt

@@ -151,29 +151,40 @@ elseif (ANYHTTP_API STREQUAL "COROSIO")
         message(FATAL_ERROR "ANYHTTP_API=COROSIO needs TLS_LIBRARY=OpenSSL: "
                             "corosio's openssl_stream does not build against AWS-LC")
     endif()
+else()
+    message(FATAL_ERROR "ANYHTTP_API must be ASIO or COROSIO, not '${ANYHTTP_API}'")
+endif()
 
-    #
-    # Pinned: both are pre-release and change their API on 'develop' without notice. Moving a pin
-    # is a commit of its own.
-    #
-    include(FetchContent)
-    foreach (option TESTS EXAMPLES BENCH)
-        set(BOOST_CAPY_BUILD_${option} OFF CACHE BOOL "" FORCE)
-        set(BOOST_COROSIO_BUILD_${option} OFF CACHE BOOL "" FORCE)
-    endforeach()
-    FetchContent_Declare(capy
-        GIT_REPOSITORY https://github.com/cppalliance/capy.git
-        GIT_TAG d62f5de1684389682ff4201a6171eec646165036
-        EXCLUDE_FROM_ALL)
+#
+# capy, Boost.HTTP (cppalliance/http) and, for COROSIO, corosio. Every tree needs the first two:
+# HTTP/1.1 is Boost.HTTP's parser and serializer, and those are written in terms of capy's buffers
+# and errors, whatever the runtime. See cmake/http.cmake.
+#
+# Pinned: all three are pre-release and change their API on 'develop' without notice. Moving a pin
+# is a commit of its own, and Boost.HTTP's has to build against capy's.
+#
+include(FetchContent)
+foreach (option TESTS EXAMPLES BENCH)
+    set(BOOST_CAPY_BUILD_${option} OFF CACHE BOOL "" FORCE)
+    set(BOOST_COROSIO_BUILD_${option} OFF CACHE BOOL "" FORCE)
+endforeach()
+FetchContent_Declare(capy
+    GIT_REPOSITORY https://github.com/cppalliance/capy.git
+    GIT_TAG d62f5de1684389682ff4201a6171eec646165036
+    EXCLUDE_FROM_ALL)
+include(cmake/http.cmake)
+if (ANYHTTP_COROSIO)
     FetchContent_Declare(corosio
         GIT_REPOSITORY https://github.com/cppalliance/corosio.git
         GIT_TAG 13bcf933adf8c5ce8576bd3349ae58d2e46e5c53
         EXCLUDE_FROM_ALL)
-    FetchContent_MakeAvailable(capy corosio)
+    FetchContent_MakeAvailable(capy http corosio)
     message(STATUS "Using I/O runtime: COROSIO (capy/corosio)")
 else()
-    message(FATAL_ERROR "ANYHTTP_API must be ASIO or COROSIO, not '${ANYHTTP_API}'")
+    FetchContent_MakeAvailable(capy http)
+    message(STATUS "Using I/O runtime: ASIO")
 endif()
+anyhttp_add_http()
 
 # --------------------------------------------------------------------------------------------------
 

README.md

@@ -18,7 +18,7 @@ It supports HTTP/1.x, HTTP/2 and HTTP/3 behind a common, type-erasing interface,
 
 None of those protocols are implemented from scratch. Instead, it is a wrapper around the following well-established libraries:
 
-* [beast](https://github.com/boostorg/beast) (its parser and serializer)
+* [Boost.HTTP](https://github.com/cppalliance/http) (its parser, serializer and header fields), pre-release, built from its `develop` branch
 * [nghttp2](https://github.com/nghttp2/nghttp2), or [nghttp2 version 2](https://github.com/nghttp2/nghttp2-v2), its rewrite
 * [ngtcp2](https://github.com/ngtcp2/ngtcp2) / [nghttp3](https://github.com/ngtcp2/nghttp3)
 
@@ -106,7 +106,6 @@ An installed package works as well, on ASIO, see
 
 For now, this section contains just a set of random links collected during development.
 
-* [Beast Example using Type Erasure](https://www.boost.org/doc/libs/develop/boost/beast/http/message_generator.hpp)
 * [Development container](docs/devcontainer.rst) -- how the build environment is put together
 * [Capturing traffic](docs/capture.rst) -- tcpdump and tshark, decrypting TLS and QUIC with a key log
 * [cpp-devcontainer](https://github.com/pgit/cpp-devcontainer) -- the generic C++ base image it builds on

cmake/http.cmake

@@ -0,0 +1,69 @@
+#
+# Boost.HTTP (https://github.com/cppalliance/http), the HTTP/1.1 parser and serializer of h1/ and
+# the header field container that is anyhttp's Fields. Not part of a Boost release yet, so it comes
+# like capy and corosio, by FetchContent at a pinned commit of 'develop'.
+#
+# Only its protocol core is built, as a target of our own, anyhttp_http (anyhttp::http when
+# installed). Upstream's CMakeLists.txt builds all of src/ -- the router, the server, bcrypt and
+# JSON -- and asks for Boost.JSON, which nothing here needs; SOURCE_SUBDIR keeps
+# FetchContent_MakeAvailable() from adding it. The compression codecs are left out as well: the
+# parser and serializer look them up as services at run time and do without them unless asked to
+# (de)compress, which anyhttp never does. Their error categories are part of the core all the same.
+#
+# The patches in cmake/patches/ are proposed upstream. Each one says what it is for.
+#
+#   http-release-buffered-data.patch    parser::release_buffered_data(), a stub on 'develop': the
+#                                       h2c upgrade hands what follows the request to HTTP/2
+#   http-reserve-bytes-alignment.patch  fields_base::reserve_bytes() took any capacity, which
+#                                       misaligns the table of fields at the end of the buffer
+#                                       (UBSAN: "reference binding to misaligned address")
+#
+set(anyhttp_http_patches
+    ${CMAKE_CURRENT_LIST_DIR}/patches/http-release-buffered-data.patch
+    ${CMAKE_CURRENT_LIST_DIR}/patches/http-reserve-bytes-alignment.patch)
+
+#
+# The patch step runs when the source is fetched, and again whenever its command changes -- not when
+# a patch file does, so each command carries the hash of its patch. Every run starts from the
+# pristine files: a patch that has changed does not apply over its earlier version.
+#
+set(anyhttp_http_patch_command COMMAND git checkout --quiet -- .)
+foreach (patch IN LISTS anyhttp_http_patches)
+    file(SHA256 ${patch} anyhttp_http_patch_hash)
+    list(APPEND anyhttp_http_patch_command COMMAND sh -c
+        "git apply '${patch}' # ${anyhttp_http_patch_hash}")
+endforeach()
+unset(anyhttp_http_patch_hash)
+list(POP_FRONT anyhttp_http_patch_command) # the first COMMAND is PATCH_COMMAND itself
+
+FetchContent_Declare(http
+    GIT_REPOSITORY https://github.com/cppalliance/http.git
+    GIT_TAG 1f58e9791afd6c7351805f3074d4d3bbff8189f9
+    PATCH_COMMAND ${anyhttp_http_patch_command}
+    SOURCE_SUBDIR do-not-add-upstream-cmake
+    EXCLUDE_FROM_ALL)
+
+function(anyhttp_add_http)
+    set(dir ${http_SOURCE_DIR})
+    file(GLOB sources
+        ${dir}/src/*.cpp
+        ${dir}/src/core/*.cpp
+        ${dir}/src/detail/*.cpp
+        ${dir}/src/rfc/*.cpp
+        ${dir}/src/rfc/detail/*.cpp
+        ${dir}/src/brotli/error.cpp
+        ${dir}/src/zlib/error.cpp
+        ${dir}/src/zstd/error.cpp)
+    list(FILTER sources EXCLUDE REGEX "/src/(application|datastore)\\.cpp$|/file_win32\\.cpp$")
+
+    add_library(anyhttp_http STATIC ${sources})
+    add_library(anyhttp::http ALIAS anyhttp_http)
+    set_target_properties(anyhttp_http PROPERTIES EXPORT_NAME http)
+    target_compile_features(anyhttp_http PUBLIC cxx_std_20)
+    target_include_directories(anyhttp_http SYSTEM PUBLIC
+        $<BUILD_INTERFACE:${dir}/include>
+        $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
+    target_include_directories(anyhttp_http PRIVATE ${dir})
+    target_compile_definitions(anyhttp_http PUBLIC BOOST_HTTP_STATIC_LINK PRIVATE BOOST_HTTP_SOURCE)
+    target_link_libraries(anyhttp_http PUBLIC Boost::capy Boost::headers Boost::url)
+endfunction()

cmake/install.cmake

@@ -51,7 +51,13 @@ endif()
 
 set(anyhttp_cmake_dir ${CMAKE_INSTALL_LIBDIR}/cmake/anyhttp)
 
-install(TARGETS anyhttp
+#
+# Boost.HTTP's core is a library of its own, see cmake/http.cmake, and its headers come along: Fields
+# is its container. So does capy, which it is written against: both are FetchContent'd, and a
+# subproject added EXCLUDE_FROM_ALL installs nothing of its own, so they are part of this package,
+# as anyhttp::http and anyhttp::capy.
+#
+install(TARGETS anyhttp anyhttp_http boost_capy
     EXPORT anyhttp-targets
     ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT Development
     LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT Development
@@ -77,6 +83,9 @@ install(DIRECTORY ${PROJECT_SOURCE_DIR}/include/anyhttp
     PATTERN "*_impl.hpp" EXCLUDE
     PATTERN "any_async_stream.hpp" EXCLUDE
     PATTERN "buffer_array.hpp" EXCLUDE)
+install(DIRECTORY ${http_SOURCE_DIR}/include/boost ${capy_SOURCE_DIR}/include/boost
+    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} COMPONENT Development
+    FILES_MATCHING PATTERN "*.hpp")
 install(FILES ${PROJECT_BINARY_DIR}/include/anyhttp/config.hpp
     DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/anyhttp COMPONENT Development)
 

cmake/patches/http-release-buffered-data.patch

@@ -0,0 +1,93 @@
+diff --git a/include/boost/http/parser.hpp b/include/boost/http/parser.hpp
+index 2edf97b..dfe0c9c 100644
+--- a/include/boost/http/parser.hpp
++++ b/include/boost/http/parser.hpp
+@@ -336,6 +336,19 @@ public:
+         to retrieve protocol-dependent data that
+         follows the HTTP message.
+
++        The octets are released: they are no longer
++        part of what the parser holds, and a later
++        @ref has_buffered_data returns false. The body
++        of the message, if any, is left as it is.
++
++        @par Preconditions
++        `this->is_complete() == true`, otherwise the
++        returned view is empty.
++
++        @par Postconditions
++        The returned view is invalidated by any
++        modifying member function.
++
+         @return A string view of leftover data.
+
+         @see @ref metadata::upgrade, @ref metadata::connection.
+diff --git a/src/parser.cpp b/src/parser.cpp
+index e73cb3b..29aac77 100644
+--- a/src/parser.cpp
++++ b/src/parser.cpp
+@@ -33,6 +33,7 @@
+ #include "src/detail/buffer_utils.hpp"
+ #include "src/detail/zlib_filter_base.hpp"
+
++#include <algorithm>
+ #include <array>
+ #include <memory>
+
+@@ -1303,6 +1304,45 @@ public:
+             body_avail_);
+     }
+
++    core::string_view
++    release_buffered_data() noexcept
++    {
++        if(state_ != state::complete)
++            return {};
++
++        // the octets past the message are at the end of cb0_,
++        // behind what is left of a plain body
++        auto const skip = is_plain() ? body_avail_ : 0;
++        if(cb0_.size() <= skip)
++            return {};
++
++        // make them contiguous: rotating the storage puts the
++        // second piece of the circular buffer behind the first
++        auto cbp = cb0_.data();
++        if(cbp[1].size() != 0)
++        {
++            auto* base = static_cast<char*>(
++                const_cast<void*>(cbp[1].data()));
++            auto* first = static_cast<char*>(
++                const_cast<void*>(cbp[0].data()));
++            auto const cap = cb0_.max_size();
++            auto const size = cb0_.size();
++            std::rotate(base, first, first + cbp[0].size());
++            cb0_ = { base, cap, size };
++            cbp = cb0_.data();
++        }
++
++        auto const* p = static_cast<char const*>(cbp[0].data());
++        core::string_view leftover(p + skip, cb0_.size() - skip);
++
++        // released: no longer part of what the parser holds, but
++        // the storage stays as it is until the next modifying call.
++        // What is left is the body, and that is all the buffer
++        // may claim: p is not the base of the storage.
++        cb0_ = { const_cast<char*>(p), skip, skip };
++        return leftover;
++    }
++
+     bool
+     has_buffered_data() const noexcept
+     {
+@@ -1564,8 +1604,8 @@ core::string_view
+ parser::
+ release_buffered_data() noexcept
+ {
+-    // TODO
+-    return {};
++    BOOST_ASSERT(impl_);
++    return impl_->release_buffered_data();
+ }
+
+ bool