Repository navigation
Improve IOCP performance - #420
TheLingeringWill wants to merge 6 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
|
|
An automated preview of the documentation is available at https://420.corosio.prtest3.cppalliance.org/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-11-14 04:42:32 UTC |
8d6c47d to
ac722a0
Compare
|
GCOVR code coverage report https://420.corosio.prtest3.cppalliance.org/gcovr/index.html Build time: 2026-11-14 05:01:03 UTC |
ac722a0 to
a508e9d
Compare
Each accept asked the kernel for both endpoints of the new connection, one getsockname and one getpeername, although AcceptEx had already written both addresses into the buffer it was given. The tcp service now looks up GetAcceptExSockaddrs next to AcceptEx and the accept parses the two addresses out of that buffer, which takes no syscall. A provider that does not hand out GetAcceptExSockaddrs leaves the accept asking the kernel as before.
A successful connect called getsockname to cache a local endpoint that most connections never read. It now marks the endpoint pending, and the first local_endpoint() call fetches and caches it, as the io_uring backend does since it stopped fetching it eagerly. One reader claims the fill and a reader racing it fetches its own copy, so concurrent calls never read the cache while it is being written. A fetch that fails caches nothing, and the next call asks again. ConnectEx needs a bound socket, and an endpoint left unfetched says the socket most likely is one. The mark can outlive the socket it described when a late completion races a close and reopen, so a connect over it tries the bind anyway and lets an "already bound" refusal through to ConnectEx. A second connect on a connected socket still fails as it did before, at ConnectEx. The local_endpoint() documentation now says the endpoint may be cached by the first call after a connect rather than at the connect.
Every socket close and cancel asked the wait reactor to drop the socket's wait op, parked or not, and every open() closes first. Once the polling thread exists, each ask took the reactor lock, sent a byte down the wakeup channel, and made the thread wake, search its table and rebuild its poll set, for a socket that in nearly every case had nothing parked. Ops now carry a flag the reactor sets when a register queues them and clears before it completes them. A cancel reads it under the reactor lock and returns when it is clear: the op is not held, so there is nothing to drop. A cancel that comes before the register is still caught there by the op's cancelled flag, and one that races the reactor letting go of the op is dropped by the table search as before. Two fault tests used a cancel with nothing parked as a wake that always reaches the polling thread. testWakeSendFails now parks its wait again after every cancel, and testWaitReactorPollFails wakes the thread with a register.
Removing a fired, cancelled or refused wait erased it from the middle of the table, moving every entry after it down, so k completions out of n parked waits cost O(k*n) moves. The table has no order, so the last entry now fills the hole; the walks that remove as they go run from the back, so the entry moved was already examined and the slots below it keep their poll set entries. Draining the wakeup channel read until recv failed, which took a second recv on every wake just to hear WSAEWOULDBLOCK. Only the sender that turns wake_pending_ from false to true writes a byte, so the channel holds one at a time and a short read has emptied it.
update_timeout read the earliest expiry and set the waitable timer with no lock held. Two threads arming at once could read different expiries and set the timer in the opposite order, leaving the later deadline set, so the earlier timer fired late, at the later one's deadline. A thread inserting a new earliest timer races the run thread re-arming after a fire the same way. The read and the arm now happen under the dispatch lock, as asio arms its timer. A new earliest expiry is published before its notification takes the lock, so the last thread through always arms the earliest one.
The stream and random-access file wrappers passed their by-value stop token on by copy, so every read and write took and dropped a reference on the stop state. They now move it, as the stream handle already does.
a508e9d to
c3e1148
Compare
|
There is something weird with the unit tests in the CI: https://github.com/cppalliance/corosio/actions/runs/43557619651/job/132334044957?pr=420#step:31:1168 |
The two allocation sites #383 names are already gone on develop: 8efad53 made the wait reactor reuse its drain buffers and moved random-access file operations onto a free list (
testRafChurnIsZeroAlloccovers it). This PR takes on the rest of the pass. Compared with asio's IOCP backend, corosio made several syscalls per connection that asio does not, and the Windows benchmark report shows matching gaps against asio coroutines: about 5% on accept_churn and 3% on socket_latency pingpong. The changes below remove those calls and some smaller per-operation costs.Changes
getsocknameandgetpeername, but AcceptEx had already written both addresses into its buffer. The accept now parses them withGetAcceptExSockaddrs, looked up once next to AcceptEx, which takes no syscall. A provider that does not hand out the function keeps the old path. The local-stream acceptor is unchanged.getsocknamefor a local endpoint most connections never read. The firstlocal_endpoint()call now fetches it, as the io_uring backend has done since 0b3830d.local_endpoint()calls never read the cache while it is being written. A fetch that fails caches nothing, and the next call asks again.tcp_socket::local_endpoint()documentation now says the endpoint may be cached by the first call after a connect, rather than at the connect.open()asked it to drop the socket's wait operation. Each ask cost a lock, a byte down the wakeup channel, a wake, a table search and a poll-set rebuild, nearly always for a socket with nothing parked. Operations now carry a flag that the registering thread sets under the reactor lock and the reactor clears before completing the operation. A cancel returns early when the flag is clear.testWakeSendFailsnow parks its wait again after every cancel.testWaitReactorPollFailsnow wakes the thread with a register.recvon every wake just to getWSAEWOULDBLOCK. Only one byte is ever in flight, so a short read is the end.update_timeoutread the earliest expiry and set the waitable timer with no lock held. Two threads arming at once could set the later deadline last, which made the earlier timer fire late. The read and the arm now happen under the dispatch lock, as asio does.Not in this PR
These came up in the audit but are larger, or need Windows profiling to justify:
CancelIoExthat every close makes beforeclosesocket. It would save a syscall per close, and asio's close does without it. Butclosesocketalone may not cancel pending I/O when a child process has inherited the socket handle, and corosio creates its sockets inheritable. It came out of this PR after review, and needs either non-inheritable sockets or a way to tell that nothing is pending.SetWaitableTimerwhen a new earliest timer is due after the deadline already set, which would save a syscall per arm in the per-operation timeout pattern (arm, cancel, arm a later one). It came out of this PR after review. The waitable timer's relative countdown stops while the system sleeps, butsteady_clockkeeps running. After a sleep, a remembered deadline can look already passed while the timer still has time to run, and a timer skipped behind it fires late. Doing this correctly means keeping the deadline in unbiased interrupt time, andQueryUnbiasedInterruptTimeneeds Windows 7, while corosio supports Vista.win_scheduler, plusGetQueuedCompletionStatusExbatching. Everypost()is aPostQueuedCompletionStatusand every handler a separate dequeue.FILE_SKIP_COMPLETION_PORT_ON_SUCCESSfor sockets and files. It needs an IFS-provider check and changes whatrelease()and adoption hand back.IOCTL_AFD_POLLin place of the wait reactor's polling thread.Measurements
Not run yet. The Windows benchmark needs a maintainer to add the
benchmarklabel.Testing
New tests in
iocp_faults:testAcceptReadsAcceptExAddresses: an accept makes nogetsocknameorgetpeernamecall, and both endpoints are correct.testAcceptWithoutSockaddrsParser: with theGetAcceptExSockaddrslookup failed, the accept asks the kernel (onegetsockname, onegetpeername) and the endpoints are still correct.testConnectDefersLocalEndpoint: nogetsocknameat connect, one at the firstlocal_endpoint(), none on a reread.testSecondConnectFailsAtConnectEx: a second connect on a connected socket, whose endpoint was never read, makes onebindand still fails withWSAEISCONN.testDeferredLocalEndpointOutlivesConnection: the deferred endpoint, first read after the peer resets the connection or after both sides close it, equals the address the peer saw. This checks that Windowsgetsocknamestill answers once the connection is gone.testIdleReactorNotWokenByClose: once the polling thread runs, closes and a cancel with no wait parked send nothing down its wakeup channel.testReactorRemovalKeepsMovedWait: cancelling a wait from the middle of the reactor's table leaves the wait moved into its slot parked, and still cancellable. Removal during the ready walk is reached only bytestWaitReactorEntryRefused, on providers that refuse the whole poll.testWakeSendFailsandtestWaitReactorPollFailschanged as described above. The comment ontestStaleWaitCancelsFromDestroyedObjectswas updated for the new cancel rule.No Windows toolchain was available while writing this, so Windows CI on this PR is the first build of the code.
Resolves #383