Block-granular reads for Caterva2, and what they needed - #702
Open
FrancescAlted wants to merge 20 commits into
Open
Block-granular reads for Caterva2, and what they needed#702FrancescAlted wants to merge 20 commits into
FrancescAlted wants to merge 20 commits into
Conversation
Analysis of what it would take to give C2Array the block-granular reads blosc2.open(url, lazy=True) got in #701, and where the work goes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
httpx.get() builds a client, opens a connection and negotiates TLS per call. Against cat2.cloud that is 0.162 s per request against 0.046 s through a pooled client, on the existing chunk path -- 3.5x, and it grows in weight as reads get smaller, which block-granular reads will make them. The client is shared and thread-safe, which is what lets Proxy fan its fetches out, but it never keeps a cookie: auth belongs to the C2Array being read, and arrays with different tokens (or none) share the client. Login keeps a client of its own for the same reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Everything FsspecNDSource knows about taking a Blosc2 frame apart -- the header, the chunk offsets, which chunks are worth reading block by block, where those blocks are and how few requests they fit in -- is the frame format and nothing else. It now lives in a base class whose only abstract method is read_range(offset, size); FsspecNDSource is that base plus four lines of fsspec, and C2Array will be the same base plus HTTP ranges. The frame index is read through read_range too, so a subclass has one seam to implement rather than two. That is what the test churn is: reads made while opening, and the read behind a whole-chunk fetch, now show up in the traffic the tests count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Caterva2 serves a stored dataset with a FileResponse, which implements RFC 7233 on its own: a ranged request is answered 206 with the bytes asked for, seeked to in the file, and the auth cookie composes with it. So no new endpoint is needed for a Proxy over a C2Array to fetch the blocks a slice touches rather than the chunks they live in. On cat2.cloud's kevlar-tomo.b2nd a corner slice costs 0.031 MB instead of 2.723 MB -- 88x fewer bytes, and the same wall time on a fast link, where a round trip and a megabyte cost about the same. C2NDSource is ByteRangeNDSource plus that transport; C2Array grows the five members Proxy looks for and delegates them to it, so every existing Proxy(C2Array(...)) gets this without asking. What it costs when it does not apply is the point of the two gates before it: api/info rules out a dataset the subscriber computes (it reports an expression where a stored one reports a geometry) and one whose chunks are too small to be worth taking apart, both without a request, so those datasets behave exactly as they did. Everything else is decided by the status code of the first range read, once: a streamed dataset answers 200 with the whole body, which read_range refuses without reading it off the socket, and the array keeps to api/chunk for good. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RFC 7233 lets a Range header name several spans, and the answer is a multipart/byteranges body carrying each with its own Content-Range. Starlette builds that, so the blocks a slice wants -- across chunks, since they are all the same file -- can travel in one request instead of one each. No object store offers this; it is what makes a subscriber the better backend for block reads rather than merely an equal one. Measured against cat2.cloud on kevlar-tomo.b2nd, a slice touching ten chunks: 20 requests in 0.334 s becomes 2 in 0.204 s, and 1.008 s becomes 0.141 s against a subscriber reached one request at a time. Both waves collapse: the chunk headers that say where the blocks are, then the blocks. The seam is one method. ByteRangeNDSource grows read_ranges(spans), which by default is one read_range each, and max_ranges to say how many a transport will take; C2NDSource overrides the first and raises the second. A source with neither -- every fsspec backend -- is batched one range at a time, which is the reads it always made. The answer is taken apart by what each part says it holds rather than by trusting the order: Starlette sorts the spans and merges the ones that touch, and answers a plain 206 when they all merge into one. A subscriber that answers with less than was asked for is noticed once and never batched again. Also give C2Array a max_concurrency: Proxy.afetch already used 8 for one, and fetch was serial only for want of somewhere to read the figure from. That alone is what takes the unbatched block path from 1.008 s to 0.334 s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plan said an HDF5 leaf or a .b2z member might report a geometry while being streamed, and that a C2Array over a computed dataset would be caught by the api/info check. Both turned out slightly otherwise, and the plan now says so rather than reading as if it had been right. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bench/ndarray/fsspec-block-granularity.py answers "would blocks beat chunks for this array, and by how much" for an fsspec URL. This is the same question for a Caterva2 dataset, where it has a different shape: whether the dataset serves ranges at all is the first thing to know, and three separate changes -- a pooled client, overlapped fetches, one request carrying many ranges -- decide the rest. It prints, for a dataset of the caller's choosing: whether ranges are served and what finding that out cost, what the frame index costs, the request plan of each mode for six slice shapes, what a pooled connection is worth against a client per request, and the timed slices themselves. The plans come from Proxy._wanted_blocks and the source's own block_plan rather than from a reimplementation, and the chunk sizes are read out of the chunk headers rather than guessed from the gaps between them, so the counts in the plan table are the ones the timed run then produces. No service is needed: a stand-in subscriber serves api/info, api/fetch and api/chunk over loopback from any local .b2nd, with ranges, multipart, and the sort-and-merge Starlette does to the spans it is given. --streamed makes it answer the way a computed dataset is served, which is what the fallback costs. --latency-ms and --bandwidth-mbs put a network back in front of loopback. Against cat2.cloud/demo on kevlar-tomo.b2nd, a line through all ten chunks: chunk mode 10 req 14.43 MB 1.008 s blocks 20 req 0.20 MB 0.202 s multipart 2 req 0.20 MB 0.107 s 9.4x and the two slabs that want every block of their chunk come out identical in all three modes, which is the threshold declining to take a chunk apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Run against a .b2z member, the bench died in httpx: api/chunk resolves a path without its inner key and 404s, so a proxy over a container leaf cannot fetch chunks from it at all. That has never worked and is nothing this branch changed, but a tool whose job is to answer "what would this dataset cost" should answer it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The format asks to be read in four steps -- how long is the header, the header, how long is the offsets chunk, the offsets chunk -- and each of the two questions costs a round trip to learn a number smaller than the answer it asks for. Over a network that is half the cost of opening an array. Both are guessed at instead: 8 KB of the head, which holds any ordinary header (165-320 bytes in practice), and the tail that the frame's own length bounds, capped at 64 KB so that a large trailer is not dragged along with the offsets (compressed offsets are small -- 4 KB for a frame of 100_000 chunks). A guess that falls short is followed by the exact read that would have happened anyway, so the worst case is the cost of today. A frame small enough to arrive whole in the first read costs one request: its offsets chunk is in those bytes already. Measured against cat2.cloud on kevlar-tomo.b2nd, opening a C2Array: as the format asks 0.237 s 4 requests, 303 bytes optimistic 0.138 s 2 requests, 8_306 bytes More bytes, fewer round trips, which is the trade worth making: 8 KB is under a millisecond on any link where a round trip is 45. It is in ByteRangeNDSource, so blosc2.open(url, lazy=True) gets it too. The seek/read shim the reader needed goes away with it -- it reads exact ranges now, which is what the transport underneath was doing all along. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A cache was checked against the source's geometry alone, so a dataset rewritten with the same shape and partitioning was adopted and went on being served from whatever the earlier run had fetched. In block mode that is worse than stale data: the cached chunks were spliced at offsets read from the frame the cache was built against, and the new frame's offsets are somewhere else. C2Array now has a stamp, like FsspecNDSource: api/info's mtime and the compressed size, both already fetched at construction, so it costs no request. The size is in there because a rewrite inside one clock tick is what an mtime cannot see. A subscriber reporting no mtime leaves the array unstamped and the cache checked on geometry alone, as before. The vlmeta entry is renamed fsspec-stamp -> proxy-stamp, since it is no longer only fsspec's. Caches from earlier builds of this development cycle are not adopted; mode="w" once is the way through, and nothing released is affected. Also: do not stamp a cache opened read-only. blosc2.open(path, mode="r") over a persisted proxy hands the Proxy a cache it may not write to, and recording the stamp there raised "Cannot do this action with reading mode" instead of opening it -- reachable only once a C2Array had a stamp to record, which is why the network tests caught it and the local ones did not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four follow-ons (the bench, the container-leaf chunks, the two-request open, the stamp) were only visible as strikethroughs in a list of things left undone, and the headline numbers predated the last two commits. Both are now one run of the bench against cat2.cloud, which is where those numbers should come from anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A .b2z is a zip of stored frames, so a leaf's bytes are a contiguous frame at a fixed offset and blosc2 already computes where. If api/fetch answers a leaf's ranged request by seeking into the container, every client that reads a .b2nd over ranges reads a leaf the same way, knowing nothing about containers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A zip store writes each external leaf uncompressed, so a member's bytes in the file are the Blosc2 frame that leaf would have been written as on its own: self-contained, beginning at a known offset, and readable by anything that reads a frame. The store computes those windows already, for its own reading -- map_tree names the member, offsets says where it is -- but nothing could ask. member_window(key) is that question, and it answers None where there is no window rather than a wrong one: a directory-backed store keeps leaves in files of their own, an embedded leaf lives inside the store's own compressed super-chunk, and a C2Array leaf is a reference rather than bytes. Wanted by a server that would hand a byte-range reader the window instead of rebuilding the leaf per request, which is plans/container-leaf-ranges.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The client side of container-leaf ranges is that there is no client side: a subscriber serves a leaf out of its window in the .b2z, so what arrives is a frame beginning at 0 and everything already written for a .b2nd applies. The stand-in subscriber can now be pointed at a leaf, which is the same six lines: it serves the window instead of the file, exactly as Caterva2 does. The tests then assert what matters -- a Proxy over a leaf fetches blocks rather than chunks, and the leaf is stamped with the container's mtime, since a leaf has none of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Including the one deviation from the plan: the whole-leaf fetch is served from the window too, because the rebuild it replaced re-partitioned the array and so disagreed with the chunks api/info reports for the same leaf. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The stand-in subscriber binds a real ThreadingHTTPServer, and Pyodide serves listen(2) out of node's `ws` module -- absent from the cibuildwheel xbuildenv, so the call takes the whole interpreter down as a fatal error rather than raising something a test could catch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
blosc2.open(url, lazy=True)fetches only the blocks a slice touches (#701).C2Array, the other remote path, still fetched whole chunks. This gives it thesame thing, and the pieces that turned out to be needed on the way.
Companion server-side PR: ironArray/Caterva2#289. Neither needs
the other to be useful; together they are what the numbers below measure.
Plans, with the analysis and the results:
plans/cat2-block-granularity.mdandplans/container-leaf-ranges.md.What is in it
C2Array.httpx.get()opened a connection andnegotiated TLS per call: 0.162 s against 0.046 s pooled, on the existing chunk
path. Auth stays per request, and the client keeps no cookies.
ByteRangeNDSource: everythingFsspecNDSourceknew about taking a frameapart, with one abstract
read_range.FsspecNDSourceis that plus fourlines of fsspec;
C2NDSourceis that plus HTTP ranges with the Caterva2cookie.
C2Array, decided by two gates that cost nothing(
api/inforules out computed datasets, and chunks under a megabyte are notworth an extra round trip) and then by the status code of one range read,
once, never retried.
object store offers. 32 spans in 0.136 s against 0.208 s as 32 requests eight
at a time.
four: both reads that only measured the next one are guessed at instead.
C2Array, so a cache built against a dataset that was sincereplaced raises instead of being served stale.
DictStore.member_window(key), which says where a leaf's frame liesinside a
.b2z— what lets the server hand a byte-range reader the window.bench/ndarray/cat2-block-granularity.py, the fsspec bench's questionasked of a subscriber; it needs no service to point at.
Numbers
bench/ndarray/cat2-block-granularity.py @public/examples/kevlar-tomo.b2nd --urlbase https://cat2.cloud/demo, median of 5:1.4% of the bytes for a slice landing in a corner of a chunk; the slabs that
want every block of theirs are untouched, which is the threshold declining to
take a chunk apart. Around the slice: one request costs 42.3 ms pooled against
163.3 ms unpooled, and opening a
C2Array0.138 s against 0.237 s.Worth knowing before merging
fsspec-stamp->proxy-stamp, socaches from earlier builds of this development cycle are not adopted;
mode="w"once is the way through. Nothing released carries the old key.C2Arraygainedmax_concurrency = 8:Proxy.fetchover one was serial,where
afetchalready used that figure. It is what takes the unbatched blockpath from 1.008 s to 0.334 s.
Tests
9450 local and 90 network tests pass. The C2 block path is covered against a
stand-in subscriber that answers Caterva2's three read endpoints over loopback,
ranges, multipart and Starlette's sort-and-merge included, so none of it needs a
service to run.
🤖 Generated with Claude Code