Skip to content

refactor!: shrink the buffer API and match Asio's iterator names #479

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sgerbino merged 1 commit into from
Oct 30, 2026
Merged

refactor!: shrink the buffer API and match Asio's iterator names #479

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions doc/lint/baseline.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,6 @@
"lint/.docstrings/buffers.hpp.adoc:#4:Vale.Spelling",
"lint/.docstrings/buffers.hpp.adoc:#5:Vale.Spelling",
"lint/.docstrings/buffers.hpp.adoc:#6:Vale.Spelling",
"lint/.docstrings/buffers/buffer_param.hpp.adoc:#1:Vale.Spelling",
"lint/.docstrings/buffers/buffer_slice.hpp.adoc:#1:Vale.Spelling",
"lint/.docstrings/buffers/buffer_slice.hpp.adoc:#2:Vale.Spelling",
"lint/.docstrings/buffers/consuming_buffers.hpp.adoc:#1:Vale.Spelling",
"lint/.docstrings/buffers/consuming_buffers.hpp.adoc:#2:Vale.Spelling",
"lint/.docstrings/buffers/make_buffer.hpp.adoc:#1:Google.LyHyphens",
"lint/.docstrings/buffers/make_buffer.hpp.adoc:#1:Google.OxfordComma",
"lint/.docstrings/buffers/make_buffer.hpp.adoc:#2:Google.LyHyphens",
Expand Down Expand Up @@ -481,12 +476,12 @@
"?:#6:unsupported HTML tag <tt>",
"?:#7:unsupported HTML tag <tt>",
"?:#8:unsupported HTML tag <tt>",
"include/boost/capy/buffers.hpp:#1:begin: variable is undocumented",
"include/boost/capy/buffers.hpp:#1:buffer_empty: variable is undocumented",
"include/boost/capy/buffers.hpp:#1:buffer_sequence_begin: variable is undocumented",
"include/boost/capy/buffers.hpp:#1:buffer_sequence_end: variable is undocumented",
"include/boost/capy/buffers.hpp:#1:buffer_size: variable is undocumented",
"include/boost/capy/buffers.hpp:#1:end: variable is undocumented",
"include/boost/capy/buffers.hpp:#1:buffers_empty: variable is undocumented",
"include/boost/capy/buffers/buffer_copy.hpp:#1:buffer_copy: variable is undocumented",
"include/boost/capy/buffers/front.hpp:#1:front: variable is undocumented",
"include/boost/capy/buffers/buffers_front.hpp:#1:buffers_front: variable is undocumented",
"include/boost/capy/detail/intrusive.hpp:#1:boost::capy::detail::intrusive_list: Failed to resolve reference to 'node'",
"include/boost/capy/ex/executor_ref.hpp:#1:executor_ref: function is undocumented",
"include/boost/capy/ex/recycling_memory_resource.hpp:#1:boost::capy::recycling_memory_resource::allocate_fast: 2nd parameter is unnamed",
Expand Down
6 changes: 2 additions & 4 deletions doc/modules/ROOT/pages/4.coroutines/4g.composition.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,10 +144,8 @@ loop that bound it.

In the snippet above nothing dangles: `items` outlives the cpp:when_all[]. The
hazard is what a caller could do -- passing a temporary would leave the stored
task holding a reference to it. Awaiting a task immediately, as function `read_all`
from section
xref:5.buffers/5d.buffer-algo.adoc#_cursor_interface[Buffer Algorithms] does,
is the case where a reference parameter is safe.
task holding a reference to it. Awaiting a task immediately is the case where a
reference parameter is safe.
====

=== Racing a Deadline
Expand Down
4 changes: 2 additions & 2 deletions doc/modules/ROOT/pages/5.buffers/5.intro.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ designed with the following criteria in mind:
that describe sequences of those handles.
* xref:5.buffers/5c.system-io-integration.adoc[System I/O Integration] --
how a sequence maps onto the structures that native I/O systems expect.
* xref:5.buffers/5d.buffer-algo.adoc[Buffer Algorithms] -- counting bytes,
slicing, copying, and consuming those sequences.
* xref:5.buffers/5d.buffer-algo.adoc[Buffer Algorithms] -- counting bytes
and copying those sequences.
* xref:5.buffers/5e.rationale.adoc[Rationale] -- why the handles use `void*`,
why sequences are concepts, and why those concepts require bidirectional
iteration.
21 changes: 19 additions & 2 deletions doc/modules/ROOT/pages/5.buffers/5b.composite-buffers.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ Algorithms call them inside `noexcept` functions, so a conversion that throws te
Header `<boost/capy/buffers.hpp>` also provides a couple of function objects for navigating buffer sequences.

In order to iterate over contiguous buffers themselves (as opposed to bytes),
use cpp:begin[capy::begin] and cpp:end[capy::end]:
use cpp:buffer_sequence_begin[] and cpp:buffer_sequence_end[]:

[source,cpp]
----
Expand All @@ -150,10 +150,27 @@ These functions treat single contiguous buffers as a one-element range of contig
include::example$snippets/5b_composite_buffers.cpp[tag=begin_end_uniform,indent=0]
----

Function cpp:buffer_length[] says how many contiguous buffers are present in the buffer
Function cpp:buffers_count[] says how many contiguous buffers are present in the buffer
sequence. It is equivalent to:

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=length_equivalent,indent=0]
----

=== Iterating Over Bytes

To treat a buffer sequence as one run of bytes, use cpp:buffers_begin[] and
cpp:buffers_end[]. They return a cpp:buffers_iterator[], a random access
iterator that crosses buffer boundaries and skips empty buffers, so standard
algorithms can scan the data without copying it into contiguous storage:

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=buffers_iterator_include]

include::example$snippets/5b_composite_buffers.cpp[tag=iterate_bytes,indent=0]
----

Moving the iterator by `n` bytes walks the buffers it crosses, so jumps are
linear in the number of buffers. The sequence must outlive the iterators.
4 changes: 2 additions & 2 deletions doc/modules/ROOT/pages/5.buffers/5c.system-io-integration.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ auto platform_write(Buffers const& buffers)
{
iovec iovecs[16]; // fixed on-frame window, never heap-allocated

auto it = begin(buffers);
auto last = end(buffers);
auto it = capy::buffer_sequence_begin(buffers);
auto last = capy::buffer_sequence_end(buffers);
while (it != last)
{
std::size_t count = fill_iovecs(iovecs, it, last, 16); // up to 16
Expand Down
50 changes: 4 additions & 46 deletions doc/modules/ROOT/pages/5.buffers/5d.buffer-algo.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ include::example$snippets/5d_buffer_algo.cpp[tag=buffer_size_example,indent=0]
----


cpp:buffer_empty[] reports whether a sequence carries no data,
cpp:buffers_empty[] reports whether a sequence carries no data,
either because it holds no buffers or because every buffer has size zero.
It is a shorthand for `buffer_size(bufseq) == 0`.

Expand All @@ -43,9 +43,9 @@ This would translate into later using a zero-size buffer.
The following two functions have similar names but return different values:

* cpp:buffer_size[] -- number of bytes across the entire buffer sequence,
* cpp:buffer_length[] -- number of contiguous buffers in the sequence.
* cpp:buffers_count[] -- number of contiguous buffers in the sequence.

A three-buffer sequence of 100 bytes each has a cpp:buffer_size[] of 300 and a cpp:buffer_length[] of 3.
A three-buffer sequence of 100 bytes each has a cpp:buffer_size[] of 300 and a cpp:buffers_count[] of 3.
====


Expand Down Expand Up @@ -82,49 +82,7 @@ include::example$snippets/5d_buffer_algo.cpp[tag=buffer_copy_cross,indent=0]

The algorithm fills target buffers in order, reading from source buffers as needed. It handles a source buffer spanning several targets, and the reverse.


== Slicing

cpp:buffer_slice[] returns a byte sub-range of a buffer sequence, as a value:

[source,cpp]
----
include::example$snippets/5d_buffer_algo.cpp[tag=buffer_slice_include]

include::example$snippets/5d_buffer_algo.cpp[tag=buffer_slice,indent=0]
----

cpp:buffer_slice[]`(seq, offset, length)` returns a value that is itself a buffer sequence, so you can pass it to any operation expecting one. Both `offset` and `length` are optional, which makes it a general byte sub-range primitive. Except in the single-buffer case the result borrows `seq`, so the sequence must outlive the slice.

Capy creates and manages buffer handles on your behalf. Those are the sub-range a cpp:buffer_slice[] produces, and the descriptors a type-erased stream passes to the OS. Each is valid only for the window its API documents.

== Cursor Interface

When transferring data incrementally, cpp:consuming_buffers[] is a cursor that tracks progress:

[source,cpp]
----
include::example$snippets/5d_buffer_algo.cpp[tag=consuming_buffers_include]

include::example$snippets/5d_buffer_algo.cpp[tag=read_all,indent=0]
----

The cursor borrows the underlying sequence and provides:

* cpp:consuming_buffers::data[data]`()` -- buffer sequence view of the remaining bytes
* cpp:consuming_buffers::consume[consume]`(n)` -- advance past `n` transferred bytes, in place

[NOTE]
====
`read_all` takes its stream by reference and its buffers by value. Both are safe
here, because the caller owns the stream and awaits the task immediately.

A coroutine reads its parameters when its body runs, not when the call is
written. A task that is stored and awaited later outlives its call expression,
so a reference parameter can dangle by then. That is why the fan-out example on
xref:4.coroutines/4g.composition.adoc[Concurrent Composition] takes its item by
value: it collects its tasks first, then awaits them together.
====
Capy creates and manages buffer handles on your behalf, such as the descriptors a type-erased stream passes to the OS. Each is valid only for the window its API documents.


== Custom Buffer Types
Expand Down
2 changes: 1 addition & 1 deletion doc/modules/ROOT/pages/5.buffers/5e.rationale.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ At type-erasure boundaries the tradeoff reverses. Virtual functions need concret
The concepts require bidirectional ranges, not merely forward ranges, for two reasons:

1. Some algorithms traverse buffers backwards
2. The slice views from cpp:buffer_slice[] and cpp:consuming_buffers::data[]`()` must adjust the first and last buffers' bounds
2. Composed operations such as cpp:read[] and cpp:write[] resume partway through a sequence, through a view that must adjust the first buffer's bounds
2 changes: 1 addition & 1 deletion doc/modules/ROOT/pages/8.examples/8c.buffer-composition.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Capy's buffer-sequence concepts accept any range of cpp:const_buffer[] or cpp:mu
include::example$buffer-composition/buffer_composition.cpp[tag=multi_buffer,indent=0]
----

For more than two buffers, the same pattern works with a larger `std::array`. Buffer sequences support `buffer_size()` and `buffer_length()` for querying total bytes and buffer count.
For more than two buffers, the same pattern works with a larger `std::array`. Buffer sequences support `buffer_size()` and `buffers_count()` for querying total bytes and buffer count.

=== Scatter/Gather I/O

Expand Down
2 changes: 1 addition & 1 deletion doc/modules/ROOT/pages/9.design/9b.Separation.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ Corosio contains five platform backends, each a substantial body of platform-spe
* *kqueue* on macOS and BSD
* *select* as a POSIX fallback

Merging these into Capy would mean that a developer who wants a cpp:task[task<>] type or a cpp:buffer_slice[] must compile against platform I/O headers. Keeping Capy separate ensures that none of the headers a consumer includes transitively pull in anything from the platform I/O layer.
Merging these into Capy would mean that a developer who wants a cpp:task[task<>] type or a cpp:buffer_copy[] must compile against platform I/O headers. Keeping Capy separate ensures that none of the headers a consumer includes transitively pull in anything from the platform I/O layer.


== Conclusion
Expand Down
2 changes: 1 addition & 1 deletion doc/modules/ROOT/pages/9.design/9c.ReadStream.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Deferral is forced only at the boundary, when the final bytes exactly fill the b

=== The Empty-Buffer Rule

When `buffer_empty(buffers)` is true, `n` is 0. The empty buffer is not itself a cause for error, but `ec` may reflect the state of the stream.
When `buffers_empty(buffers)` is true, `n` is 0. The empty buffer is not itself a cause for error, but `ec` may reflect the state of the stream.

Whether the implementation performs a system call for a zero-length buffer is unspecified. A concrete type that short-circuits with `(!ec, 0)` conforms. A concrete type that forwards the zero-length call to the OS and reports whatever condition arises also conforms.

Expand Down
27 changes: 19 additions & 8 deletions doc/modules/ROOT/pages/9.design/9m.WhyNotAsio.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,34 +88,45 @@ Asio got xref:glossary.adoc#buffer-sequence[buffer sequences] right. The concept

We adopt Asio's buffer sequence model because it works.

But we improve on it. Need to trim bytes from the front of a buffer sequence? Asio makes you work for it. Capy provides cpp:buffer_slice[] and cpp:front[]—byte-range slicing primitives for efficient byte-level manipulation. Need to compose two buffers without copying? Use `std::array<const_buffer, 2>` (or any range of buffers) directly — Capy's buffer-sequence concepts accept arbitrary ranges.
But we improve on it. Capy states the requirements as C++20 concepts rather than type traits, and cpp:MutableBufferSequence[] refines cpp:ConstBufferSequence[]. A function overloaded on both selects the mutable overload for a mutable sequence instead of being ambiguous. Capy also adds small algorithms that Asio leaves out: cpp:buffers_front[], cpp:buffers_count[] and cpp:buffers_empty[].

`std::ranges` cannot help here. `ranges::size` returns the number of buffers, not the total bytes. Range views can drop entire elements, but buffer sequences need byte-level trimming. Buffer sequences need their own concepts.

=== What Capy Offers

* cpp:ConstBufferSequence[], cpp:MutableBufferSequence[] — core concepts (Asio-compatible)
* cpp:buffer_slice[], cpp:front[] — byte-level manipulation utilities
* cpp:buffers_front[] — the first buffer of a sequence
* cpp:buffers_count[] — the number of buffers in a sequence
* cpp:buffers_empty[] — whether a sequence holds any bytes

=== Comparison

Capy adopts cpp:ConstBufferSequence[], cpp:MutableBufferSequence[], cpp:const_buffer[] and cpp:mutable_buffer[] from Asio. Its element requirement is stricter: an element of a mutable buffer sequence must also convert to cpp:const_buffer[]. Every mutable buffer sequence is therefore also a const buffer sequence. What Capy adds is byte-level slicing:
Capy adopts cpp:ConstBufferSequence[], cpp:MutableBufferSequence[], cpp:const_buffer[] and cpp:mutable_buffer[] from Asio. Its element requirement is stricter: an element of a mutable buffer sequence must also convert to cpp:const_buffer[]. Every mutable buffer sequence is therefore also a const buffer sequence. The buffer algorithms compare as follows:

[cols="1,1"]
|===
| Capy | Asio

| cpp:buffer_slice[]
| cpp:buffer_sequence_begin[], cpp:buffer_sequence_end[]
| `buffer_sequence_begin`, `buffer_sequence_end`

| cpp:buffers_begin[], cpp:buffers_end[]
| `buffers_begin`, `buffers_end`

| cpp:buffers_front[]
^| -

| cpp:buffers_count[]
^| -

| cpp:front[]
| cpp:buffers_empty[]
^| -

| cpp:buffer_size[]
| `buffer_size`

| cpp:buffer_copy[]
| `buffer_copy`

| Byte-level trimming
^| -
|===

== Coroutine Execution Model
Expand Down
12 changes: 2 additions & 10 deletions doc/modules/ROOT/pages/9.design/9n.WhyNotCobalt.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,15 +295,7 @@ Cobalt does not provide buffer sequence types. Users who need these features use
| Yes
| Via Asio

| cpp:front[]
| Yes
|

| cpp:buffer_slice[]
| Yes
|

| Byte-level trimming
| cpp:buffers_front[]
| Yes
|
|===
Expand Down Expand Up @@ -531,7 +523,7 @@ Every `co_await ws.write(...)` call creates a xref:glossary.adoc#coroutine-frame
| `cancellation_signal`, automatic, OS-level

| Buffer sequences
| Extended (cpp:buffer_slice[], cpp:front[])
| Extended (cpp:buffers_front[])
| None (use Asio directly)

| Allocator control
Expand Down
3 changes: 1 addition & 2 deletions doc/modules/ROOT/pages/9.design/9p.WhyNotTMC.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,7 @@ NOTE: Neither library prevents the caller from initiating multiple concurrent I/
*Capy provides:*

* xref:glossary.adoc#buffer-sequence[Buffer sequences] (xref:glossary.adoc#scatter-gather-io[scatter/gather I/O] without copying)
* Algorithms: slice, copy, concatenate, consume
* Dynamic buffers that grow as needed
* Algorithms: copy, count, and inspect buffer sequences
* xref:glossary.adoc#type-erased[Type-erased] streams: write code once, use with any stream type

*TMC provides:*
Expand Down
23 changes: 9 additions & 14 deletions doc/mrdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,14 @@ exclude-symbols:
- 'boost::capy::test::blocking_handler_wrapper'
- 'boost::capy::test::run_blocking_wrapper'
- 'boost::capy::test::bufgrind::next_awaitable'
# Private members of buffer_param's anonymous union. `inaccessible-members:
# never` does not reach variant members of an anonymous union in this MrDocs
# build, so these two private fields leak into the reference as "[variant
# member]" rows. They are storage internals, not API. The `**` spans the
# unnamed union's own scope, which sits between the class and the members.
- 'boost::capy::buffer_param::**::dummy_'
- 'boost::capy::buffer_param::**::arr_'
# Same anonymous-union leak in the task and quitter promises: `ep_` is the
# exception_ptr held in a union so it is not constructed or destroyed
# automatically, and it sits in a `private:` section that `inaccessible-members:
# never` would otherwise filter. Its sibling `has_ep_`/`state_` -- private but
# not in a union -- is correctly filtered, which is what identifies the union as
# the cause.
# Private members of anonymous unions. `inaccessible-members: never` does not
# reach variant members of an anonymous union in this MrDocs build, so they
# leak into the reference as "[variant member]" rows. In the task and quitter
# promises, `ep_` is the exception_ptr held in a union so it is not constructed
# or destroyed automatically. Its sibling `has_ep_`/`state_` -- private but not
# in a union -- is correctly filtered, which is what identifies the union as
# the cause. The `**` spans the unnamed union's own scope, which sits between
# the class and the members.
- 'boost::capy::task::**::ep_'
- 'boost::capy::quitter::**::ep_'
implementation-defined:
Expand Down Expand Up @@ -64,7 +59,7 @@ warn-if-doc-error: true
# Two further MrDocs 0.8.0 limitations stay grandfathered in
# doc/lint/baseline.json instead of being fixed here. The reasoning lives next
# to the affected code so it is findable from either end:
# include/boost/capy/buffers/front.hpp -- the six buffer CPO
# include/boost/capy/buffers/buffers_front.hpp -- the six buffer CPO
# "variable is undocumented" findings
# include/boost/capy/ex/run.hpp -- "run: Documented parameter 'alloc'
# does not exist"
Expand Down
6 changes: 3 additions & 3 deletions doc/unlisted/library-buffers.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ task<void> echo(ReadStream auto& in, WriteStream auto& out)
| Scatter/gather I/O with multiple buffers

| xref:../buffers/algorithms.adoc[Buffer Algorithms]
| `buffer_copy`, `buffer_size`, `buffer_empty`
| `buffer_copy`, `buffer_size`, `buffers_empty`
|===

== Key Types
Expand Down Expand Up @@ -107,8 +107,8 @@ task<void> echo(ReadStream auto& in, WriteStream auto& out)
| `<boost/capy/buffers/buffer_copy.hpp>`
| `buffer_copy` algorithm

| `<boost/capy/buffers/buffer_slice.hpp>`
| Byte sub-range slicing algorithm
| `<boost/capy/buffers/buffers_iterator.hpp>`
| `buffers_iterator`, `buffers_begin`, `buffers_end`
|===

== Navigation
Expand Down
4 changes: 2 additions & 2 deletions doc/unlisted/library-streams.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ If `buffer_size(buffers) > 0`:
* If `!ec`: `n >= 1 && n \<= buffer_size(buffers)`. `n` bytes were read.
* If `ec`: `n >= 0 && n \<= buffer_size(buffers)`. `n` is the number of bytes read before the I/O condition arose.

If `buffer_empty(buffers)` is true, `n` is 0. The empty buffer is not itself a cause for error, but `ec` may reflect the state of the stream.
If `buffers_empty(buffers)` is true, `n` is 0. The empty buffer is not itself a cause for error, but `ec` may reflect the state of the stream.

==== Example

Expand Down Expand Up @@ -180,7 +180,7 @@ If `buffer_size(buffers) > 0`:
* If `!ec`: `n >= 1 && n \<= buffer_size(buffers)`. `n` bytes were written.
* If `ec`: `n >= 0 && n \<= buffer_size(buffers)`. `n` is the number of bytes written before the I/O condition arose.

If `buffer_empty(buffers)` is true, `n` is 0. The empty buffer is not itself a cause for error, but `ec` may reflect the state of the stream.
If `buffers_empty(buffers)` is true, `n` is 0. The empty buffer is not itself a cause for error, but `ec` may reflect the state of the stream.

==== Example

Expand Down
Loading
Loading