diff --git a/packages/zarr-storage/DESIGN.md b/packages/zarr-storage/DESIGN.md index 46edd3d635..0867b09478 100644 --- a/packages/zarr-storage/DESIGN.md +++ b/packages/zarr-storage/DESIGN.md @@ -27,26 +27,33 @@ Sources, and what each contributed: ``` Backend capability protocols over backend paths; owns shared state and lifecycle ▲ -_View (backend, root, read_only, owner): validates keys, applies the root, relativizes - ▲ listings, enforces read-only; `store / "a"` is a view on the same backend +_View (backend, root, read_only, context, owner): validates keys, applies the root, + ▲ relativizes listings, enforces read-only, limits and schedules every operation with + its context's resources; `store / "a"` is a view on the same backend Store the operations as sync methods; `close()`, `with`; `as_async()` AsyncStore the same names as coroutines; `aclose()`, `async with`; `as_sync()` ``` A **backend** is an object that implements some capability protocols (`Readable`, `Writable`, `Listable`, `WritableIfMatch`, `DeletableIfMatch`, and their async twins) on *backend paths*: -`/`-separated strings, already joined with the root. Backends never see relative keys and never join or strip paths. They hold everything that must exist once: the dict, -the filesystem root, a client, locks, concurrency limits. +`/`-separated strings, already joined with the root. Backends never see relative keys and never +join or strip paths. They hold everything that must exist once per backend: the dict, the +filesystem root, a client, locks. They hold no thread pools or limits: a backend names the +`resource_key` its operations are limited under, and the store draws the resources from its +context (see "Contexts"). Backends must be thread-safe. A **`Store`** or **`AsyncStore`** is the only object users hold. Each is a view: a backend plus a -root. All key validation, root joining, listing relativization and read-only enforcement happen in -their shared base, once, so no backend can get them wrong on its own (the source of most bugs in the legacy API: #2190, #2467, #3639, #3922, #4204, +root, under a context. All key validation, root joining, listing relativization, read-only +enforcement and resource limiting happen in their shared base, once, so no backend can get them +wrong on its own (the source of most bugs in the legacy API: #2190, #2467, #3639, #3922, #4204, #4452). **Sync and async are two classes** (as zarrs has separate sync and async storage traits): `Store` -calls a backend's sync protocols and `AsyncStore` its async ones, under the same method names, so -`store.get(k)` and `await astore.get(k)` read the same way and neither class carries the other's -methods. A backend with both surfaces serves both; `store.as_async()` and `astore.as_sync()` are +calls a backend's sync protocols, and `AsyncStore` its async ones where the backend has them and +its sync ones on the context's thread pool otherwise (zarrs's `SyncToAsyncStorageAdapter`, with +the pool in place of its `spawn_blocking`), under the same method names, so `store.get(k)` and +`await astore.get(k)` read the same way and neither class carries the other's methods. A backend +that only blocks, such as a filesystem, needs no async code. Every backend serves both; `store.as_async()` and `astore.as_sync()` are views on the same backend that don't own it. Lifecycle differs too: `Store` has `close()` and `with`, `AsyncStore` has `await aclose()` and `async with`, which awaits a `ClosableAsync` backend's `aclose` (a backend whose cleanup awaits a client session) and calls `close` on any other. @@ -121,13 +128,16 @@ landed with their results and whose `remaining` holds the requests that failed o (`except*` keeps both). Conditional operations have no batch form: compare-and-swap coordinates single documents, and a batch of it belongs to transactions. -`Store` provides every batch method for any backend that can read or write, running the single -operation one at a time; a backend that has a batch capability (`ReadableMany`, -`ReadableIntoMany[O]`, `WritableMany`, and async twins) is used instead. The rule for such -fallbacks: `Store` may fall back when only speed differs, never when behavior does. A serial batch -returns what a concurrent one would; a `get_into` that read and then copied would not put the bytes -where the backend's buffer type says, which is why `get_into` has no fallback. The batch runners in -`zarr_storage._batch` implement the windowing and failure rule once, for `Store` and backends alike. +`Store` provides every batch method for any backend that can read or write, running single +operations concurrently under the view's context: `Store` on the context's thread pool, `AsyncStore` +as tasks, at most the context limiter's `limit` at once, each holding its own slot. A backend with +a batch capability (`ReadableMany`, `ReadableIntoMany[O]`, `WritableMany`, and async twins) is +used instead, for batching the store itself can do (one request for many keys, one GPU +submission); it receives the context so it can draw on the same resources. The rule for any +fallback: `Store` may fall back when only speed differs, never when behavior does. A batch of single +operations returns what a native batch would; a `get_into` that read and then copied would not put +the bytes where the backend's buffer type says, which is why `get_into` has no fallback. The batch +runners in `zarr_storage._batch` implement the windowing and failure rule once. Listing arguments are checked when the method is called; the items are produced lazily, and taking one after the store closes raises `StoreClosedError`. @@ -141,8 +151,9 @@ raises `InvalidByteRangeError`; a suffix longer than the value returns the whole ## Views, lifecycle, identity -- `store / "a"`, `store.with_read_only(True)` and `store.as_async()` (or `as_sync()`) return views - on the same backend. A view of a +- `store / "a"`, `store.with_read_only(True)`, `store.with_context(ctx)`, `store.limited(n)` and + `store.as_async()` (or `as_sync()`) return views on the same backend. A view keeps its context + unless it is given another. A view of a read-only store is read-only; `with_read_only(False)` on one raises `ReadOnlyError`. - The store the user constructed owns the backend. `close()` (`aclose()` on an `AsyncStore`) on it closes the backend; on a derived view it does nothing. After the backend closes, every view raises `StoreClosedError`. @@ -166,10 +177,10 @@ key (`a/b` when `a` is a key): that reads as absent. | `LocalBackend` | `(dev, inode, mtime_ns, size)`, with each write stamping a unique ns mtime | yes / yes, on filesystems that keep ns times | no (`KeyConflictError`) | the root | | `ObjectStoreBackend` | ETag and version (obstore's `LocalStore` ETag can repeat on ext4) | if the store supports it / not implemented | as the store does | as the store does | -Every backend has `put_if_absent` and `get_if_changed`. `LocalBackend` runs batches, sync and -async, `concurrency` operations at once on its threads; `ObjectStoreBackend` keeps `concurrency` -requests of an async batch in flight and runs sync batches one request at a time; `MemoryBackend` -uses `Store`'s serial batches, which are as fast for it. +Every backend has `put_if_absent` and `get_if_changed`. `LocalBackend` is sync only; `AsyncStore` +runs its operations on the context's pool. `MemoryBackend` and `ObjectStoreBackend` are also +natively async. Their resource keys are `"local"`, `"memory"`, and `"obstore."` plus the obstore +store's class name, so every S3 backend in a process shares one limit by default. `LocalBackend` stages each write in a reserved directory under the root and renames it into place, holding the lock of the key's directory; the lock lives in that directory, so overlapping roots and @@ -210,25 +221,144 @@ step, each stored key reads back as the model says, through `get` and `get_many` `get_if_changed` with its generation is `Unchanged`. The obstore tests also fail any async run that makes a blocking obstore call. +## Contexts + +Resource limits are part of a store's context, as in TensorStore. zarrs has no I/O limits of its +own (its per-call `Resources` carry only a compute target, from Rayon's global pool); it bounds +object-store requests by wrapping the store in object_store's `LimitStore` and sharing that store. + +- A **`Context`** holds a `ThreadPool` and a `Limiter` per resource key. It is immutable; + `ctx.child(threads=..., limits={...})` overrides some resources and takes the rest from its + parent, and a context with no parent takes the rest from the process-wide defaults + (`Context.default()`): `ThreadPool.default()` of `max(4, cpu_count)` threads (TensorStore's + `file_io_concurrency` default) and one `Limiter(32)` per resource key (its + `s3_request_concurrency` default). A default is shared state, but it changes only speed, never + what an operation does. +- **Every view carries a context**, `Context.default()` unless given one (`Store(backend, + context=...)`, `store.with_context(ctx)`). Every operation through the view holds a slot of + `context.limiter(backend.resource_key)`; blocking work (a sync backend called through an + `AsyncStore`, and the operations of a `Store` batch) runs on `context.threads`. Stores sharing a + context share its limits, whatever their backends. Like TensorStore's, a context is not part of a + store's identity: `shares_backend` and `overlaps` ignore it. +- **`store.limited(n)`** is a view whose operations, together, run at most `n` at once, for a burst + of work: its context is a child whose limiter for the backend's key is `Limiter(n, within=...)` + the parent's. A limiter `within` another holds a slot of both, so a scoped limit never exceeds the + limit it narrows. Here the design departs from TensorStore, where a child context's resource + replaces the parent's: replacing would let a scoped limit escape the process-wide one. +- **Slots are taken innermost first**, so work queued on a narrow limit holds none of the wide one, + and **where the work runs**: a sync backend's operation called through an `AsyncStore` takes its + slot on the pool thread that runs it. No operation holds a slot while it waits for a thread, so + pool threads waiting for slots and slot holders waiting for threads can't block each other. +- `limiter.slot()` is a sync or async context manager, and a slot is not re-entrant, as with a lock: + stores take exactly one slot per operation, and never call each other while holding one. An + earlier draft made slots re-entrant through a context variable; that leaked into every task and + thread inheriting the context (`asyncio.gather`, and new threads on free-threaded Python 3.14), + letting a fan-out run unbounded under one slot. +- A batch started on one of its pool's own threads runs on that thread, one operation at a time: + waiting there for work queued on the same pool could wait forever once every thread does it. +- Whoever creates a resource owns it; the default pool can't be shut down. Pickling a context + pickles its resources' settings, and pickle stores each object once, so stores that shared a + context or a resource share one after a round trip; a default unpickles as the receiving + process's default. +- Pools for I/O and for compute should stay separate (TensorStore's `file_io_concurrency` and + `data_copy_concurrency`): compute waiting on I/O in one pool starves it. A context holds the I/O + pool; zarr-python's codec pipeline would take its own. + +## Caching + +Planned for iteration 3; not built yet. A cache is a resource of the store's context, like +TensorStore's `cache_pool`. It is neither a store class that wraps another store (zarr-python's +`CacheStore`) nor a hook on store methods: + +- **A wrapper can't get its capabilities right.** A Python class can't have exactly the capabilities + of the backend it wraps. It would need one class per combination, or it would claim every + capability and raise at runtime, losing the typed capabilities. +- **A hook that caches is a wrapper.** To serve a hit, a hook has to skip the read and supply the + result, and then it is a wrapper. + +The rule: wrapping is for layers that change what keys mean (zip, sharding). Policies that leave +the meaning alone (limits, caching, retries) are context resources. + +- **A `CachePool(tier, total_bytes)` is a context resource**, set with `Context(cache=...)` or + `ctx.child(cache=...)`. The default pool holds 0 bytes, so caching is off until a context gives it + a budget. Stores sharing a context share its pool: one byte budget, one eviction order. +- **The cached bytes live in a store, the tier.** The tier is a memory store by default, a local + store for a disk cache, or any other writable store. Each cached value is a value in the tier, so + the cache gets the following from the tier without new code: + - byte-range reads and `get_ranges`, served from one cached whole value; + - batches, for the hits of a batch; + - thread safety and both surfaces (a memory tier answers an `AsyncStore` inline); + - the conformance suite, as the tier's tests. + + zarr-python's `CacheStore` keeps its bytes in a store too; what changes is where the cache sits. +- **A cache is more than a store.** Beside the tier, the pool keeps an index in memory for what a + store can't hold. Per entry, the index records: + - the source's generation (the tier mints its own); + - the source's `read_at`; + - the entry's size and recency; + - absences: a `Missing` from the source has no bytes to store. + + The index also runs eviction, and it combines concurrent misses on one key into one source read. +- **Tier keys are opaque and written once.** The index names each entry's bytes with a fresh key + under a random per-pool prefix. Tier keys don't come from source keys, because the key grammar + has no way to name the source backend. A refresh writes a new key and points the index at it. + Eviction removes the index entry, then deletes the key. So no reader sees a tier value change + under it, and a tier read that returns `Missing` (evicted, or a disk tier someone else cleaned) is + a miss. Pools sharing a disk tier don't collide. +- **Entries are keyed by backend instance and path**, the identity `shares_backend` uses. Closing a + backend drops its entries. +- **Each view decides how stale a read may be**, with `store.with_max_age(seconds)`, default `0`. + TensorStore makes the same split (`recheck_cached_metadata`, `recheck_cached_data`), because + metadata and chunks need different answers. + - A hit younger than `max_age` returns without touching the source. + - An older hit revalidates with `get_if_changed(key, generation)`. On object stores that's a + conditional GET: one round trip, no bytes. + - `math.inf` trusts the cache forever, for values that are never rewritten. + - An absence ages like a value, but only a full read can revalidate it, so caching absences pays + only when `max_age > 0`: zarr's metadata probes for format versions, for example. +- **Writes keep a pool coherent.** A write or delete through a view whose context holds the pool + drops that key's entry, and `delete_dir` drops every entry under its prefix. Writes also bump a + per-key epoch, and a read that started before a write to its key returns its result but + doesn't fill the cache. So a view + sees its pool's own writes whatever its `max_age`. Writes from other processes, other pools, or + straight to the backend show up only through revalidation, as in TensorStore. +- **Hits hold no source slot.** The tier runs under a context of the pool's own, so a hit waits on + neither the source's limiter nor the process-wide `memory` limiter. A revalidation or a miss holds + a source slot like any other read. +- Listings are never cached, and neither is a value larger than the budget. +- Pickling a pool pickles its budget and its tier's configuration, never its contents. The + receiver starts with an empty index and an empty tier. +- `get_into` into a buffer type the tier can't write (device memory, with a host tier) bypasses + the pool. +- Every `StoreSuite` runs again through a cached view (a pool with a budget, `max_age=0`), where the + cache must be invisible. The state machine adds writes made straight to the backend, which only + revalidation can catch. + ## Configuration -No global configuration. Every knob (local I/O concurrency, how many requests an obstore batch -keeps in flight) is a constructor parameter with its default in the signature. +No global configuration. Every knob is a constructor parameter with its default in the signature; +resources are passed in a store's context. ## Open - Durability: no backend syncs to disk; a `sync` option on `LocalBackend` would. - Staged files left by a crash are never read but are never removed either. - Windows rules are implemented and unit-tested, not yet run on Windows. -- Sync batches over obstore run one request at a time; a thread pool, or a native batch call, - would overlap them. +- Listings hold no slot of the context's limiter. - `get_ranges_into`, `put_from`, and a GPU direct storage backend whose `get_into_many` submits one cuFile batch. +- Caching: range reads of values the pool doesn't hold whole go to the source. Shard indexes are the + case that matters, and they belong to the codec pipeline, not here: the sharding codec knows where + an index is and can cache it decoded, keyed by store, key and generation. The storage layer gives + it what it needs: the generation of every read, and `get_if_changed` with a byte range to + revalidate one. +- Caching: a disk tier starts cold in every process, because the index lives in memory. +- Caching: a pool over a memory backend copies values for no gain; nothing stops one yet. ## Iterations 1. **Core:** types, `Store`, the memory and local backends, the conformance suite in `zarr_storage.testing`, pickling. 2. **Remote and archive backends:** obstore (S3, GCS, Azure, HTTP; async-native), fsspec, zip. -3. **Composition:** range coalescing, caching, retry, a sync bridge for async-only backends, URL specs. +3. **Composition:** range coalescing, caching (see Caching), retry, a sync bridge for async-only backends, URL specs. 4. **Transactions:** staged writes committed with generations, memory first. diff --git a/packages/zarr-storage/changes/464.feature.md b/packages/zarr-storage/changes/464.feature.md new file mode 100644 index 0000000000..20d405e110 --- /dev/null +++ b/packages/zarr-storage/changes/464.feature.md @@ -0,0 +1 @@ +Add `Context`, the resources a `zarr-storage` store's operations draw on: a `ThreadPool` and a `Limiter` per backend resource key, with process-wide defaults. Every store view carries a context (`Store(backend, context=...)`, `with_context`), stores sharing a context share its limits, and `store.limited(n)` bounds a burst of operations within the context's limit. `AsyncStore` now runs a sync-only backend on the context's thread pool, and batches run concurrently under the context. diff --git a/packages/zarr-storage/src/zarr_storage/__init__.py b/packages/zarr-storage/src/zarr_storage/__init__.py index 510c1e6ef0..c5ac6b8a9a 100644 --- a/packages/zarr-storage/src/zarr_storage/__init__.py +++ b/packages/zarr-storage/src/zarr_storage/__init__.py @@ -3,8 +3,8 @@ A `Store` is a view of a backend at a root directory, used through sync methods; an `AsyncStore` is the same view used through coroutines, with the same method names. Backends implement capability protocols (`Readable`, `Writable`, `Listable`, their batch variants, and async twins) over backend -paths; the stores validate keys, apply their root, enforce read-only access, and are the only -objects most callers need. +paths; the stores validate keys, apply their root, enforce read-only access, schedule and limit +operations with the resources of their `Context`, and are the only objects most callers need. """ from zarr_storage._errors import ( @@ -39,6 +39,7 @@ WritableMany, WritableManyAsync, ) +from zarr_storage._resources import Context, Limiter, ThreadPool from zarr_storage._store import AsyncStore, Store from zarr_storage._types import ( ByteRange, @@ -67,6 +68,7 @@ "BatchError", "ByteRange", "ClosableAsync", + "Context", "Copied", "DeletableIfMatch", "DeletableIfMatchAsync", @@ -79,6 +81,7 @@ "InvalidKeyError", "Key", "KeyConflictError", + "Limiter", "ListItem", "Listable", "ListableAsync", @@ -101,6 +104,7 @@ "Store", "StoreClosedError", "SuffixRequest", + "ThreadPool", "Unchanged", "UnsupportedOperationError", "Writable", diff --git a/packages/zarr-storage/src/zarr_storage/_batch.py b/packages/zarr-storage/src/zarr_storage/_batch.py index e850519e57..c3a9145599 100644 --- a/packages/zarr-storage/src/zarr_storage/_batch.py +++ b/packages/zarr-storage/src/zarr_storage/_batch.py @@ -14,7 +14,9 @@ if TYPE_CHECKING: from collections.abc import AsyncIterator, Callable, Coroutine, Iterator, Sequence - from concurrent.futures import Executor, Future + from concurrent.futures import Future + + from zarr_storage._resources import ThreadPool __all__ = ["run_many", "run_many_async"] @@ -27,13 +29,17 @@ def run_many( fn: Callable[[Unpack[Ts]], R], requests: Sequence[tuple[*Ts]], - executor: Executor | None = None, + pool: ThreadPool | None = None, window: int = 1, ) -> Iterator[tuple[int, R]]: - """Call `fn(*request)` for each request: one at a time, or on `executor`, `window` at once.""" - if executor is None: + """Call `fn(*request)` for each request: one at a time, or on `pool`, `window` at once. + + Called from one of `pool`'s own threads, it runs one at a time on that thread: waiting there + for work queued behind it on the same pool could wait forever. + """ + if pool is None or pool.in_worker(): return _run_serially(fn, requests) - return _run_on(executor, fn, requests, window) + return _run_on(pool, fn, requests, window) def _run_serially( @@ -48,7 +54,7 @@ def _run_serially( def _run_on( - executor: Executor, + pool: ThreadPool, fn: Callable[[Unpack[Ts]], R], requests: Sequence[tuple[*Ts]], window: int, @@ -59,7 +65,7 @@ def _run_on( try: while pending or (not errors and started < len(requests)): while not errors and started < len(requests) and len(pending) < window: - pending[executor.submit(fn, *requests[started])] = started + pending[pool.submit(fn, *requests[started])] = started started += 1 done, _ = wait(pending, return_when=FIRST_COMPLETED) for future in done: diff --git a/packages/zarr-storage/src/zarr_storage/_protocols.py b/packages/zarr-storage/src/zarr_storage/_protocols.py index 7058b87ca7..dd06555970 100644 --- a/packages/zarr-storage/src/zarr_storage/_protocols.py +++ b/packages/zarr-storage/src/zarr_storage/_protocols.py @@ -8,15 +8,22 @@ `/`. Each capability has a sync protocol and an async twin whose methods end in `_async`. A backend -implements whichever it can. `Store` calls the sync protocols and `AsyncStore` the async ones, and -their methods are typed so that calling one the backend lacks is a type error. Compare-and-swap on a generation (`WritableIfMatch`, `DeletableIfMatch`) is a capability -of its own, since many stores can't do it; `put_if_absent` is part of `Writable`. +implements whichever it can. `Store` calls the sync protocols. `AsyncStore` calls the async twin +where the backend has it and otherwise runs the sync method on its context's thread pool, so a +backend that only blocks (a filesystem) needs no async code. Methods are typed so that calling one +the backend lacks is a type error. Compare-and-swap on a generation (`WritableIfMatch`, +`DeletableIfMatch`) is a capability of its own, since many stores can't do it; `put_if_absent` is +part of `Writable`. + +Backends hold no thread pools or limits: the store schedules and limits every operation with the +resources of its `Context`, under the backend's `resource_key`. Backends must be thread-safe, since +a store runs their sync methods on many threads at once. The batch capabilities (`ReadableMany`, `ReadableIntoMany`, `WritableMany` and their async twins) -are optional and change only speed: `Store.get_many` and its siblings work on any backend that -can read or write, one operation at a time, and use a backend's batch methods when it has them. A -batch method yields `(index, result)` pairs as operations finish and follows the failure rule of -`zarr_storage._batch`, whose runners implement it. +are for batching the store itself can do (one request for many keys, one GPU submission): without +them, `Store.get_many` and its siblings run single operations concurrently under the context's +resources. A batch method receives that context, so it can draw on the same resources; it yields +`(index, result)` pairs as operations finish and follows the failure rule of `zarr_storage._batch`. """ from __future__ import annotations @@ -26,6 +33,7 @@ if TYPE_CHECKING: from collections.abc import AsyncIterator, Iterator, Sequence + from zarr_storage._resources import Context from zarr_storage._types import ( ByteRange, Copied, @@ -70,7 +78,17 @@ class Backend(Protocol): - """What every backend provides: a lifecycle and whether it accepts writes at all.""" + """What every backend provides: a lifecycle, whether it accepts writes, and the key its + operations are limited under.""" + + @property + def resource_key(self) -> str: + """The key of the `Limiter` that stores take from their context for this backend. + + Backends that should share a limit by default share a key: every obstore S3 backend uses + `"obstore.S3Store"`, so a process's S3 requests are bounded together. + """ + ... @property def closed(self) -> bool: @@ -171,7 +189,7 @@ def delete_if_match(self, path: str, generation: Generation) -> Deleted | Reject @runtime_checkable class ReadableMany(Backend, Protocol): def get_many( - self, requests: Sequence[tuple[str, ByteRange | None]] + self, requests: Sequence[tuple[str, ByteRange | None]], context: Context ) -> Iterator[tuple[int, Found | Missing]]: """`get` for each `(path, byte_range)`, yielding `(index, result)` as each finishes.""" ... @@ -180,7 +198,7 @@ def get_many( @runtime_checkable class ReadableIntoMany(Backend, Protocol[O_contra]): def get_into_many( - self, requests: Sequence[tuple[str, O_contra, ByteRange | None]] + self, requests: Sequence[tuple[str, O_contra, ByteRange | None]], context: Context ) -> Iterator[tuple[int, Copied | Missing]]: """`get_into` for each `(path, out, byte_range)`, yielding `(index, result)`.""" ... @@ -188,11 +206,15 @@ def get_into_many( @runtime_checkable class WritableMany(Backend, Protocol): - def put_many(self, requests: Sequence[tuple[str, memoryview]]) -> Iterator[tuple[int, Written]]: + def put_many( + self, requests: Sequence[tuple[str, memoryview]], context: Context + ) -> Iterator[tuple[int, Written]]: """`put` for each `(path, value)`, yielding `(index, result)` as each finishes.""" ... - def delete_many(self, paths: Sequence[str]) -> Iterator[tuple[int, Deleted | Missing]]: + def delete_many( + self, paths: Sequence[str], context: Context + ) -> Iterator[tuple[int, Deleted | Missing]]: """`delete` for each path, yielding `(index, result)` as each finishes.""" ... @@ -207,6 +229,7 @@ def list_dir(self, directory: str) -> Iterator[ListItem]: ... +@runtime_checkable class ReadableAsync(Backend, Protocol): async def get_async(self, path: str, byte_range: ByteRange | None) -> Found | Missing: ... @@ -221,12 +244,14 @@ async def get_ranges_async( async def head_async(self, path: str) -> ObjectMetadata | Missing: ... +@runtime_checkable class ReadableIntoAsync(Backend, Protocol[O_contra]): async def get_into_async( self, path: str, out: O_contra, byte_range: ByteRange | None ) -> Copied | Missing: ... +@runtime_checkable class WritableAsync(Backend, Protocol): async def put_async(self, path: str, value: memoryview) -> Written: ... @@ -237,12 +262,14 @@ async def delete_async(self, path: str) -> Deleted | Missing: ... async def delete_dir_async(self, directory: str) -> None: ... +@runtime_checkable class WritableIfMatchAsync(Backend, Protocol): async def put_if_match_async( self, path: str, value: memoryview, generation: Generation ) -> Written | Rejected | Missing: ... +@runtime_checkable class DeletableIfMatchAsync(Backend, Protocol): async def delete_if_match_async( self, path: str, generation: Generation @@ -252,28 +279,29 @@ async def delete_if_match_async( @runtime_checkable class ReadableManyAsync(Backend, Protocol): def get_many_async( - self, requests: Sequence[tuple[str, ByteRange | None]] + self, requests: Sequence[tuple[str, ByteRange | None]], context: Context ) -> AsyncIterator[tuple[int, Found | Missing]]: ... @runtime_checkable class ReadableIntoManyAsync(Backend, Protocol[O_contra]): def get_into_many_async( - self, requests: Sequence[tuple[str, O_contra, ByteRange | None]] + self, requests: Sequence[tuple[str, O_contra, ByteRange | None]], context: Context ) -> AsyncIterator[tuple[int, Copied | Missing]]: ... @runtime_checkable class WritableManyAsync(Backend, Protocol): def put_many_async( - self, requests: Sequence[tuple[str, memoryview]] + self, requests: Sequence[tuple[str, memoryview]], context: Context ) -> AsyncIterator[tuple[int, Written]]: ... def delete_many_async( - self, paths: Sequence[str] + self, paths: Sequence[str], context: Context ) -> AsyncIterator[tuple[int, Deleted | Missing]]: ... +@runtime_checkable class ListableAsync(Backend, Protocol): def list_async(self, directory: str) -> AsyncIterator[str]: ... diff --git a/packages/zarr-storage/src/zarr_storage/_resources.py b/packages/zarr-storage/src/zarr_storage/_resources.py new file mode 100644 index 0000000000..ca9c9e8ec5 --- /dev/null +++ b/packages/zarr-storage/src/zarr_storage/_resources.py @@ -0,0 +1,519 @@ +"""The resources a store's operations draw on: thread pools, limits on operations in flight, and the +context that holds them. + +Modeled on TensorStore's `Context`: a context maps resource keys to resource objects, a store view +carries one, and every operation through the view takes its resources from it. A backend names the +key its operations are limited under (`Backend.resource_key`) and never holds resources itself. +Resources change how fast operations run, never what they do, so a context is not part of a store's +identity. + +A context left out is `Context.default()`, which resolves every key to a process-wide default, as +TensorStore's thread pools and request limits are shared across independent contexts by default. + +Pickling a resource pickles its settings. Pickle stores each object once, so stores that shared a +resource before pickling share one new resource after; a default unpickles as the receiving +process's default. +""" + +from __future__ import annotations + +import asyncio +import contextvars +import os +import threading +from collections import deque +from concurrent.futures import ThreadPoolExecutor +from functools import partial +from types import MappingProxyType +from typing import TYPE_CHECKING, Final, TypeVar, TypeVarTuple + +if TYPE_CHECKING: + from collections.abc import Callable, Mapping + from concurrent.futures import Future + from types import TracebackType + +__all__ = [ + "Context", + "Limiter", + "Slot", + "ThreadPool", + "default_context", + "default_limiter", + "default_thread_pool", +] + +R = TypeVar("R") +Ts = TypeVarTuple("Ts") + +DEFAULT_THREADS: Final = max(4, os.cpu_count() or 1) +"""Threads in the default pool, as TensorStore's `file_io_concurrency` default.""" + +DEFAULT_LIMIT: Final = 32 +"""Operations in flight under a default limiter, as TensorStore's `s3_request_concurrency`.""" + + +def _check_size(name: str, value: int) -> None: + if type(value) is not int: + raise TypeError(f"{name} must be an int; got {type(value).__name__}.") + if value < 1: + raise ValueError(f"{name} must be >= 1; got {value}.") + + +# ---------------------------------------------------------------------------- thread pools + +_worker = threading.local() +"""Per thread: the `ThreadPool` the thread works for, if any.""" + + +def _mark_worker(pool: ThreadPool) -> None: + _worker.pool = pool + + +class ThreadPool: + """Threads that run blocking operations, such as file I/O. + + The threads start when first needed. Work runs in a copy of the submitter's `contextvars` + context, as `asyncio.to_thread` does. Whoever creates a pool owns it and may `shutdown` it once + nothing needs it; the default pool lives as long as the process. + + Parameters + ---------- + max_workers + The most threads the pool runs at once. + """ + + def __init__(self, max_workers: int) -> None: + _check_size("max_workers", max_workers) + self._max_workers = max_workers + self._lock = threading.Lock() + self._executor: ThreadPoolExecutor | None = None + + @classmethod + def default(cls) -> ThreadPool: + """The process-wide pool, of `DEFAULT_THREADS` threads.""" + return default_thread_pool() + + @property + def max_workers(self) -> int: + return self._max_workers + + def __repr__(self) -> str: + return f"{type(self).__name__}(max_workers={self._max_workers})" + + def __reduce__(self) -> tuple[object, ...]: + return (ThreadPool, (self._max_workers,)) + + def _pool(self) -> ThreadPoolExecutor: + with self._lock: + if self._executor is None: + self._executor = ThreadPoolExecutor( + self._max_workers, + thread_name_prefix="zarr-storage", + initializer=_mark_worker, + initargs=(self,), + ) + return self._executor + + def in_worker(self) -> bool: + """Whether the calling thread is one of this pool's. + + Work started from one of the pool's own threads must not wait for other work queued on the + pool, which may never start; stores run such work inline. + """ + return getattr(_worker, "pool", None) is self + + def submit(self, fn: Callable[[*Ts], R], /, *args: *Ts) -> Future[R]: + """Run `fn(*args)` on one of the pool's threads.""" + context = contextvars.copy_context() + return self._pool().submit(context.run, fn, *args) + + async def run(self, fn: Callable[[*Ts], R], /, *args: *Ts) -> R: + """Await `fn(*args)`, run on one of the pool's threads.""" + context = contextvars.copy_context() + loop = asyncio.get_running_loop() + return await loop.run_in_executor(self._pool(), partial(context.run, fn, *args)) + + def shutdown(self, *, wait: bool = True) -> None: + """Stop the pool's threads; with `wait`, after the work already given them. + + The pool starts new threads if it is used again. Raises `ValueError` for the default pool, + which other code may be using. + """ + with self._lock: + executor, self._executor = self._executor, None + if executor is not None: + executor.shutdown(wait=wait) + + +class _DefaultThreadPool(ThreadPool): + """The process-wide pool: never shut down, and unpickled as the receiver's own default.""" + + def __reduce__(self) -> tuple[object, ...]: + return (default_thread_pool, ()) + + def shutdown(self, *, wait: bool = True) -> None: + raise ValueError("The default thread pool is shared by the whole process.") + + +_defaults_lock = threading.Lock() +_default_pool: list[ThreadPool] = [] + + +def default_thread_pool() -> ThreadPool: + """The process-wide `ThreadPool`, of `DEFAULT_THREADS` threads.""" + with _defaults_lock: + if not _default_pool: + _default_pool.append(_DefaultThreadPool(DEFAULT_THREADS)) + return _default_pool[0] + + +# ---------------------------------------------------------------------------- limiters + + +class _ThreadWaiter: + """A thread blocked on a limiter.""" + + def __init__(self) -> None: + self._event = threading.Event() + + def grant(self) -> bool: + self._event.set() + return True + + def wait(self) -> None: + self._event.wait() + + +def _resolve(future: asyncio.Future[None]) -> None: + if not future.done(): + future.set_result(None) + + +class _TaskWaiter: + """A task blocked on a limiter, on its own event loop. Fields change under the limiter's lock.""" + + def __init__(self, loop: asyncio.AbstractEventLoop) -> None: + self.loop = loop + self.future: asyncio.Future[None] = loop.create_future() + self.granted = False + self.cancelled = False + + def grant(self) -> bool: + if self.cancelled: + return False + try: + self.loop.call_soon_threadsafe(_resolve, self.future) + except RuntimeError: # the loop is closed; nothing waits any more + return False + self.granted = True + return True + + +def _limiter(limit: int, within: Limiter | None) -> Limiter: + return Limiter(limit, within=within) + + +class Limiter: + """A limit on how many operations run at once. + + Threads and tasks on any event loop wait in one first-come, first-served queue. A limiter + `within` another takes a slot of that one too, so a scoped limit can't exceed the limit it + narrows. + + Parameters + ---------- + limit + The most operations that run at once. + within + A limiter whose slots this one's operations also hold. + """ + + def __init__(self, limit: int, *, within: Limiter | None = None) -> None: + _check_size("limit", limit) + self._limit = limit + self._within = within + self._lock = threading.Lock() + self._in_use = 0 + self._waiters: deque[_ThreadWaiter | _TaskWaiter] = deque() + + @property + def limit(self) -> int: + return self._limit + + @property + def within(self) -> Limiter | None: + """The limiter whose slots this one's operations also hold, if any.""" + return self._within + + @property + def in_use(self) -> int: + """How many slots are held now.""" + return self._in_use + + def __repr__(self) -> str: + return f"{type(self).__name__}(limit={self._limit}, in_use={self._in_use})" + + def __reduce__(self) -> tuple[object, ...]: + return (_limiter, (self._limit, self._within)) + + def chain(self) -> tuple[Limiter, ...]: + """This limiter and each it lies within, innermost first.""" + chain: list[Limiter] = [] + current: Limiter | None = self + while current is not None: + chain.append(current) + current = current.within + return tuple(chain) + + def slot(self) -> Slot: + """A context manager, sync (`with`) or async (`async with`), holding a slot of this limiter + and of each limiter it lies within. + + Slots are taken innermost first, so an operation waiting on a narrow limit holds none of the + wider one it narrows. A slot is not re-entrant: code holding one that starts work waiting + for another slot of the same limiter can wait forever, as with a lock. Stores take one slot + per operation; to bound a burst of operations, use a limited view (`Store.limited`). + """ + return Slot(self) + + def _take(self) -> bool: + """Take a free slot if no one is waiting; called under the lock.""" + if self._in_use < self._limit and not self._waiters: + self._in_use += 1 + return True + return False + + def acquire(self) -> None: + """Take a slot of this limiter alone, blocking the calling thread until one is free.""" + with self._lock: + if self._take(): + return + waiter = _ThreadWaiter() + self._waiters.append(waiter) + waiter.wait() + + async def acquire_async(self) -> None: + """Take a slot of this limiter alone, waiting without blocking the event loop.""" + with self._lock: + if self._take(): + return + waiter = _TaskWaiter(asyncio.get_running_loop()) + self._waiters.append(waiter) + try: + await waiter.future + except asyncio.CancelledError: + with self._lock: + granted = waiter.granted + if not granted: + waiter.cancelled = True + self._waiters.remove(waiter) + if granted: + self.release() + raise + + def release(self) -> None: + """Give back a slot of this limiter, to the longest waiter if there is one. + + Raises `RuntimeError` if no slot is held. + """ + with self._lock: + while self._waiters: + if self._waiters.popleft().grant(): + return # the slot passes to the waiter + if self._in_use == 0: + raise RuntimeError("Released a limiter slot that was not held.") + self._in_use -= 1 + + +def _release_all(limiters: list[Limiter]) -> None: + for limiter in reversed(limiters): + limiter.release() + + +class Slot: + """One use of a `Limiter`: holds a slot of it and of each limiter it lies within, innermost + first, from entry to exit.""" + + __slots__ = ("_chain", "_taken") + + def __init__(self, limiter: Limiter) -> None: + self._chain = limiter.chain() + self._taken: list[Limiter] = [] + + def __enter__(self) -> None: + taken: list[Limiter] = [] + try: + for limiter in self._chain: + limiter.acquire() + taken.append(limiter) + except BaseException: + _release_all(taken) + raise + self._taken = taken + + def __exit__( + self, + exc_type: type[BaseException] | None, + exc: BaseException | None, + tb: TracebackType | None, + ) -> None: + taken, self._taken = self._taken, [] + _release_all(taken) + + async def __aenter__(self) -> None: + taken: list[Limiter] = [] + try: + for limiter in self._chain: + await limiter.acquire_async() + taken.append(limiter) + except BaseException: + _release_all(taken) + raise + self._taken = taken + + async def __aexit__( + self, + exc_type: type[BaseException] | None, + exc: BaseException | None, + tb: TracebackType | None, + ) -> None: + taken, self._taken = self._taken, [] + _release_all(taken) + + +class _DefaultLimiter(Limiter): + """A process-wide limiter for one key; unpickled as the receiver's limiter for that key.""" + + def __init__(self, key: str) -> None: + super().__init__(DEFAULT_LIMIT) + self._key = key + + def __reduce__(self) -> tuple[object, ...]: + return (default_limiter, (self._key,)) + + +_default_limiters: dict[str, Limiter] = {} + + +def default_limiter(key: str) -> Limiter: + """The process-wide `Limiter` for `key`, of `DEFAULT_LIMIT`.""" + with _defaults_lock: + limiter = _default_limiters.get(key) + if limiter is None: + limiter = _default_limiters[key] = _DefaultLimiter(key) + return limiter + + +# ---------------------------------------------------------------------------- contexts + + +def _context( + threads: ThreadPool | None, limits: Mapping[str, Limiter], parent: Context | None +) -> Context: + return Context(threads=threads, limits=limits, parent=parent) + + +class Context: + """The resources operations draw on: a thread pool, and a limiter per resource key. + + A context is immutable. `child` makes one that overrides some resources and takes the rest from + its parent; a context with no parent takes the rest from the process-wide defaults. Every store + view carries a context (`Store.context`); `with_context` and `limited` return views under + another one. + + Parameters + ---------- + threads + The pool that runs blocking work: a sync backend's operations called through an + `AsyncStore`, and the operations of batches. Left out, the parent's. + limits + A `Limiter` for each resource key it names (a backend's `resource_key`). Keys left out + resolve through the parent. + parent + The context to take unnamed resources from. Left out, the process-wide defaults. + """ + + __slots__ = ("_limits", "_parent", "_threads") + + def __init__( + self, + *, + threads: ThreadPool | None = None, + limits: Mapping[str, Limiter] | None = None, + parent: Context | None = None, + ) -> None: + own = {} if limits is None else dict(limits) + for key, limiter in own.items(): + if not isinstance(limiter, Limiter): # pyright: ignore[reportUnnecessaryIsInstance] + raise TypeError(f"limits[{key!r}] must be a Limiter; got {type(limiter).__name__}.") + if threads is not None and not isinstance(threads, ThreadPool): # pyright: ignore[reportUnnecessaryIsInstance] + raise TypeError(f"threads must be a ThreadPool; got {type(threads).__name__}.") + self._threads = threads + self._limits: Mapping[str, Limiter] = MappingProxyType(own) + self._parent = parent + + @classmethod + def default(cls) -> Context: + """The process-wide context: every key resolves to its process-wide default.""" + return default_context() + + @property + def parent(self) -> Context | None: + return self._parent + + @property + def threads(self) -> ThreadPool: + """The pool that runs blocking work under this context.""" + context: Context | None = self + while context is not None: + if context._threads is not None: + return context._threads + context = context._parent + return default_thread_pool() + + def limiter(self, key: str) -> Limiter: + """The limiter for resource key `key` under this context.""" + context: Context | None = self + while context is not None: + limiter = context._limits.get(key) + if limiter is not None: + return limiter + context = context._parent + return default_limiter(key) + + def child( + self, *, threads: ThreadPool | None = None, limits: Mapping[str, Limiter] | None = None + ) -> Context: + """A context with these resources, taking the rest from this one.""" + return Context(threads=threads, limits=limits, parent=self) + + def limited(self, key: str, limit: int) -> Context: + """A child whose limiter for `key` allows `limit` operations, within this context's.""" + return self.child(limits={key: Limiter(limit, within=self.limiter(key))}) + + def __repr__(self) -> str: + own = ", ".join(f"{key!r}: {limiter!r}" for key, limiter in self._limits.items()) + return f"{type(self).__name__}(threads={self._threads!r}, limits={{{own}}})" + + def __reduce__(self) -> tuple[object, ...]: + return (_context, (self._threads, dict(self._limits), self._parent)) + + +class _DefaultContext(Context): + """The process-wide context; unpickled as the receiver's own.""" + + __slots__ = () + + def __reduce__(self) -> tuple[object, ...]: + return (default_context, ()) + + +_default_context: list[Context] = [] + + +def default_context() -> Context: + """The process-wide `Context`.""" + with _defaults_lock: + if not _default_context: + _default_context.append(_DefaultContext()) + return _default_context[0] diff --git a/packages/zarr-storage/src/zarr_storage/_store.py b/packages/zarr-storage/src/zarr_storage/_store.py index de7dedc71b..6d443a84bb 100644 --- a/packages/zarr-storage/src/zarr_storage/_store.py +++ b/packages/zarr-storage/src/zarr_storage/_store.py @@ -1,13 +1,21 @@ -"""`Store` and `AsyncStore`: views of a backend at a root, through sync methods or coroutines. +"""`Store` and `AsyncStore`: views of a backend at a root, under a context, through sync methods or +coroutines. They are where the key grammar, the root, read-only access and the closed check are enforced, -once, for every backend. A backend only sees backend paths. +once, for every backend; a backend only sees backend paths. They are also where operations are +scheduled and limited: every operation holds a slot of the view's context's limiter for the +backend's `resource_key`, and blocking work runs on the context's thread pool. + +A slot is taken where the work runs. A sync backend's operation called through an `AsyncStore` +takes its slot on the pool thread that runs it, so no operation ever holds a slot while it waits +for a thread: pool threads waiting for slots and slot holders waiting for threads can't block each +other. """ from __future__ import annotations from collections.abc import AsyncGenerator, Generator -from typing import TYPE_CHECKING, Final, Generic, Self, TypeVar, cast, final +from typing import TYPE_CHECKING, Final, Generic, Self, TypeVar, TypeVarTuple, cast, final from zarr_storage._batch import run_many, run_many_async from zarr_storage._buffers import byte_view @@ -16,35 +24,37 @@ from zarr_storage._protocols import ( Backend, ClosableAsync, + DeletableIfMatchAsync, + ListableAsync, + ReadableAsync, + ReadableIntoAsync, ReadableIntoMany, ReadableIntoManyAsync, ReadableMany, ReadableManyAsync, + WritableAsync, + WritableIfMatchAsync, WritableMany, WritableManyAsync, ) +from zarr_storage._resources import default_context from zarr_storage._types import Missing if TYPE_CHECKING: - from collections.abc import AsyncIterator, Iterable, Iterator, Sequence + from collections.abc import AsyncIterator, Callable, Iterable, Iterator, Sequence from types import TracebackType from typing_extensions import Buffer from zarr_storage._protocols import ( DeletableIfMatch, - DeletableIfMatchAsync, Listable, - ListableAsync, Readable, - ReadableAsync, ReadableInto, - ReadableIntoAsync, Writable, - WritableAsync, WritableIfMatch, - WritableIfMatchAsync, ) + from zarr_storage._resources import Context, Limiter, Slot from zarr_storage._types import ( ByteRange, Copied, @@ -65,19 +75,40 @@ O = TypeVar("O") T = TypeVar("T") R = TypeVar("R") +Ts = TypeVarTuple("Ts") Q = TypeVar("Q", bound="str | tuple[str, ByteRange | None]") """A `get_many` request: a key, or a key and the part of its value to read.""" +def _limited(limiter: Limiter, fn: Callable[[*Ts], R], *args: *Ts) -> R: + """`fn(*args)`, holding a slot of `limiter`, on the calling thread.""" + with limiter.slot(): + return fn(*args) + + +def _list_all(fn: Callable[[str], Iterator[T]], directory: str) -> list[T]: + return list(fn(directory)) + + class _View(Generic[B_co]): - """What `Store` and `AsyncStore` share: a backend, a root, read-only access, ownership.""" + """What `Store` and `AsyncStore` share: a backend, a root, read-only access, a context, and + ownership.""" - __slots__ = ("_backend", "_owner", "_read_only", "_root") + __slots__ = ("_backend", "_context", "_limiter", "_owner", "_read_only", "_root") - def __init__(self, backend: B_co, *, root: str = "", read_only: bool = False) -> None: + def __init__( + self, + backend: B_co, + *, + root: str = "", + read_only: bool = False, + context: Context | None = None, + ) -> None: self._backend: Final[B_co] = backend self._root: Final[str] = normalize_directory(root) self._read_only: Final[bool] = read_only or not backend.writable + self._context: Final[Context] = default_context() if context is None else context + self._limiter: Final[Limiter] = self._context.limiter(backend.resource_key) self._owner = True # ----------------------------------------------------------------- views and identity @@ -97,20 +128,27 @@ def read_only(self) -> bool: """Whether writes through this store are refused.""" return self._read_only + @property + def context(self) -> Context: + """The resources this store's operations draw on.""" + return self._context + @property def closed(self) -> bool: """Whether the backend has been closed.""" return self._backend.closed - def _view(self, *, root: str, read_only: bool) -> Self: - view = type(self)(self._backend, root=root, read_only=read_only) + def _view(self, *, root: str, read_only: bool, context: Context) -> Self: + view = type(self)(self._backend, root=root, read_only=read_only, context=context) view._owner = False return view def __truediv__(self, directory: str) -> Self: """A view on the same backend, rooted at `directory` under this store's root.""" return self._view( - root=join(self._root, normalize_directory(directory)), read_only=self._read_only + root=join(self._root, normalize_directory(directory)), + read_only=self._read_only, + context=self._context, ) def with_read_only(self, read_only: bool = True) -> Self: @@ -121,7 +159,19 @@ def with_read_only(self, read_only: bool = True) -> Self: """ if self._read_only and not read_only: raise ReadOnlyError(f"{self!r} is read-only; its views are read-only too.") - return self._view(root=self._root, read_only=read_only) + return self._view(root=self._root, read_only=read_only, context=self._context) + + def with_context(self, context: Context) -> Self: + """A view on the same backend and root whose operations draw on `context`.""" + return self._view(root=self._root, read_only=self._read_only, context=context) + + def limited(self, limit: int) -> Self: + """A view whose operations, together, run at most `limit` at once. + + The limit narrows this view's: an operation holds a slot of both. Batches through the view + start at most `limit` operations at once. + """ + return self.with_context(self._context.limited(self._backend.resource_key, limit)) def shares_backend(self, other: _View[Backend]) -> bool: """Whether `other` reads and writes through the same backend object.""" @@ -138,7 +188,10 @@ def __repr__(self) -> str: return f"{type(self).__name__}({self._backend!r}, root={self._root!r}{flags})" def __reduce__(self) -> tuple[object, ...]: - return (_restore_store, (type(self), self._backend, self._root, self._read_only)) + return ( + _restore_store, + (type(self), self._backend, self._root, self._read_only, self._context), + ) # ----------------------------------------------------------------- path handling @@ -154,16 +207,8 @@ def _directory(self, directory: str) -> str: self._check_open() return join(self._root, normalize_directory(directory)) - def _open_items(self, items: Iterator[T]) -> Iterator[T]: - """`items`, refusing to continue once the backend closes.""" - for item in items: - self._check_open() - yield item - - async def _open_items_async(self, items: AsyncIterator[T]) -> AsyncIterator[T]: - async for item in items: - self._check_open() - yield item + def _slot(self) -> Slot: + return self._limiter.slot() def _writable_path(self, key: str) -> str: path = self._path(key) @@ -207,20 +252,33 @@ def _writable_paths(self, keys: Sequence[str]) -> list[str]: seen.add(path) return paths + def _put_requests( + self, items: Iterable[tuple[str, Buffer]] + ) -> tuple[list[str], list[tuple[str, memoryview]]]: + batch = list(items) + keys = [key for key, _ in batch] + paths = self._writable_paths(keys) + views = [byte_view(value, writable=False) for _, value in batch] + return keys, list(zip(paths, views, strict=True)) + @final class Store(_View[B_co]): - """A backend seen from a root directory, through sync methods. + """A backend seen from a root directory, under a context, through sync methods. Construct a store from a backend; the store you construct owns the backend and closes it. - `store / "a"`, `with_read_only()` and `as_async()` return views on the same backend, which - share its state and never close it. A view of a read-only store is read-only too. + `store / "a"`, `with_read_only()`, `with_context()`, `limited()` and `as_async()` return views + on the same backend, which share its state and never close it. A view of a read-only store is + read-only too. Which methods a store offers depends on its backend's capabilities: a method whose capability the backend lacks is a type error. `AsyncStore` offers the same operations as coroutines. - `backend` gives direct access to the backend, which bypasses the key grammar, the root, and - read-only and closed checks. + Every operation holds a slot of the context's limiter for the backend's `resource_key`; batches + run their operations on the context's thread pool, at most the limiter's limit at once. + + `backend` gives direct access to the backend, which bypasses the key grammar, the root, the + read-only and closed checks, and the context's limits. Stores have no value equality: `==` is identity. Use `shares_backend` and `overlaps` to compare stores. @@ -234,16 +292,27 @@ class Store(_View[B_co]): read_only Refuse writes through this store. A backend that can't write makes every store over it read-only. + context + The resources operations draw on. Left out, `Context.default()`. """ __slots__ = () - def __init__(self, backend: B_co, *, root: str = "", read_only: bool = False) -> None: - super().__init__(backend, root=root, read_only=read_only) + def __init__( + self, + backend: B_co, + *, + root: str = "", + read_only: bool = False, + context: Context | None = None, + ) -> None: + super().__init__(backend, root=root, read_only=read_only, context=context) def as_async(self) -> AsyncStore[B_co]: - """An `AsyncStore` view on the same backend and root; it doesn't own the backend.""" - view = AsyncStore(self._backend, root=self._root, read_only=self._read_only) + """An `AsyncStore` view on the same backend, root and context; it doesn't own the backend.""" + view = AsyncStore( + self._backend, root=self._root, read_only=self._read_only, context=self._context + ) view._owner = False return view @@ -265,7 +334,13 @@ def __exit__( ) -> None: self.close() - # ----------------------------------------------------------------- batch results + # ----------------------------------------------------------------- batch plumbing + + def _run_many( + self, fn: Callable[[*Ts], R], requests: Sequence[tuple[*Ts]] + ) -> Iterator[tuple[int, R]]: + """`fn` over `requests` on the context's pool, at most the limiter's limit at once.""" + return run_many(fn, requests, self._context.threads, self._limiter.limit) def _streamed( self, batch: Sequence[T], results: Iterator[tuple[int, R]] @@ -299,13 +374,33 @@ def _finished(batch: Sequence[T], results: Iterator[tuple[int, R]]) -> list[tupl raise _batch_error(group, batch, completed, completed.keys()) from None return [(request, completed[index]) for index, request in enumerate(batch)] + # ----------------------------------------------------------------- operations on backend paths + + def _get(self: Store[Readable], path: str, byte_range: ByteRange | None) -> Found | Missing: + with self._slot(): + return self._backend.get(path, byte_range) + + def _get_into( + self: Store[ReadableInto[O]], path: str, out: O, byte_range: ByteRange | None + ) -> Copied | Missing: + with self._slot(): + return self._backend.get_into(path, out, byte_range) + + def _put(self: Store[Writable], path: str, value: memoryview) -> Written: + with self._slot(): + return self._backend.put(path, value) + + def _delete(self: Store[Writable], path: str) -> Deleted | Missing: + with self._slot(): + return self._backend.delete(path) + # ----------------------------------------------------------------- reads def get( self: Store[Readable], key: str, *, byte_range: ByteRange | None = None ) -> Found | Missing: """The value at `key`, or the part `byte_range` selects.""" - return self._backend.get(self._path(key), byte_range) + return self._get(self._path(key), byte_range) def get_if_changed( self: Store[Readable], @@ -318,13 +413,17 @@ def get_if_changed( This revalidates a cached copy, as an HTTP `If-None-Match` request does. """ - return self._backend.get_if_changed(self._path(key), generation, byte_range) + path = self._path(key) + with self._slot(): + return self._backend.get_if_changed(path, generation, byte_range) def get_ranges( self: Store[Readable], key: str, ranges: Sequence[ByteRange] ) -> FoundRanges | Missing: """Several parts of the value at `key`, all from one generation.""" - return self._backend.get_ranges(self._path(key), ranges) + path = self._path(key) + with self._slot(): + return self._backend.get_ranges(path, ranges) def get_into( self: Store[ReadableInto[O]], key: str, out: O, *, byte_range: ByteRange | None = None @@ -335,11 +434,13 @@ def get_into( for the built-in backends, device memory for a GPU direct storage backend. Raises `ValueError` if `out` is shorter than the bytes to copy or can't be written. """ - return self._backend.get_into(self._path(key), out, byte_range) + return self._get_into(self._path(key), out, byte_range) def head(self: Store[Readable], key: str) -> ObjectMetadata | Missing: """The size and generation of the value at `key`.""" - return self._backend.head(self._path(key)) + path = self._path(key) + with self._slot(): + return self._backend.head(path) def exists(self: Store[Readable], key: str) -> bool: """Whether `key` holds a value.""" @@ -349,13 +450,14 @@ def exists(self: Store[Readable], key: str) -> bool: def put(self: Store[Writable], key: str, value: Buffer) -> Written: """Write `value` at `key`.""" - return self._backend.put(self._writable_path(key), byte_view(value, writable=False)) + return self._put(self._writable_path(key), byte_view(value, writable=False)) def put_if_absent(self: Store[Writable], key: str, value: Buffer) -> Written | Rejected: """Write `value` at `key` if the key holds no value; `Rejected` if it holds one.""" - return self._backend.put_if_absent( - self._writable_path(key), byte_view(value, writable=False) - ) + path = self._writable_path(key) + view = byte_view(value, writable=False) + with self._slot(): + return self._backend.put_if_absent(path, view) def put_if_match( self: Store[WritableIfMatch], key: str, value: Buffer, generation: Generation @@ -364,23 +466,28 @@ def put_if_match( `Rejected` if the key holds another generation, `Missing` if it holds no value. """ - return self._backend.put_if_match( - self._writable_path(key), byte_view(value, writable=False), generation - ) + path = self._writable_path(key) + view = byte_view(value, writable=False) + with self._slot(): + return self._backend.put_if_match(path, view, generation) def delete(self: Store[Writable], key: str) -> Deleted | Missing: """Delete the key `key`. A directory is not a key: deleting one finds it `Missing`.""" - return self._backend.delete(self._writable_path(key)) + return self._delete(self._writable_path(key)) def delete_if_match( self: Store[DeletableIfMatch], key: str, generation: Generation ) -> Deleted | Rejected | Missing: """Delete the key `key` if its value has `generation`.""" - return self._backend.delete_if_match(self._writable_path(key), generation) + path = self._writable_path(key) + with self._slot(): + return self._backend.delete_if_match(path, generation) def delete_dir(self: Store[Writable], directory: str = "") -> None: """Delete every key under `directory` (by default, every key in this store).""" - self._backend.delete_dir(self._writable_directory(directory)) + path = self._writable_directory(directory) + with self._slot(): + self._backend.delete_dir(path) # ----------------------------------------------------------------- batches @@ -391,9 +498,10 @@ def get_many( A request is a key, or a `(key, byte_range)` pair, and comes back as it was given, so `dict(store.get_many(keys))` maps keys to results. Keys are checked when `get_many` is - called; the reads happen as the pairs are taken, in no particular order. A batch is not - atomic: each read is its own `get`, from whatever generation it finds (`get_ranges` reads - several parts of one value from one generation). + called; the reads run on the context's thread pool as the pairs are taken, at most the + limiter's limit at once, in no particular order. A batch is not atomic: each read is its own + `get`, from whatever generation it finds (`get_ranges` reads several parts of one value from + one generation). If reads raise, the iterator yields the reads already running that succeed, then raises `BatchError` with the requests left in `remaining`; its `completed` is empty, since those @@ -404,8 +512,8 @@ def get_many( paths = [self._read_request(request) for request in batch] backend = self._backend if isinstance(backend, ReadableMany): - return self._streamed(batch, backend.get_many(paths)) - return self._streamed(batch, run_many(backend.get, paths)) + return self._streamed(batch, backend.get_many(paths, self._context)) + return self._streamed(batch, self._run_many(self._get, paths)) def get_into_many( self: Store[ReadableInto[O]], @@ -421,8 +529,8 @@ def get_into_many( backend = self._backend if isinstance(backend, ReadableIntoMany): many = cast("ReadableIntoMany[O]", backend) - return self._streamed(batch, many.get_into_many(paths)) - return self._streamed(batch, run_many(backend.get_into, paths)) + return self._streamed(batch, many.get_into_many(paths, self._context)) + return self._streamed(batch, self._run_many(self._get_into, paths)) def put_many( self: Store[Writable], items: Iterable[tuple[str, Buffer]] @@ -434,17 +542,11 @@ def put_many( finish, no others start, and `BatchError` reports the keys written (`completed`, with their results) and the rest (`remaining`). """ - batch = list(items) - keys = [key for key, _ in batch] - paths = self._writable_paths(keys) - requests = [ - (path, byte_view(value, writable=False)) - for path, (_, value) in zip(paths, batch, strict=True) - ] + keys, requests = self._put_requests(items) backend = self._backend if isinstance(backend, WritableMany): - return self._finished(keys, backend.put_many(requests)) - return self._finished(keys, run_many(backend.put, requests)) + return self._finished(keys, backend.put_many(requests, self._context)) + return self._finished(keys, self._run_many(self._put, requests)) def delete_many( self: Store[Writable], keys: Iterable[str] @@ -454,16 +556,22 @@ def delete_many( paths = self._writable_paths(batch) backend = self._backend if isinstance(backend, WritableMany): - return self._finished(batch, backend.delete_many(paths)) - return self._finished(batch, run_many(backend.delete, [(path,) for path in paths])) + return self._finished(batch, backend.delete_many(paths, self._context)) + return self._finished(batch, self._run_many(self._delete, [(path,) for path in paths])) # ----------------------------------------------------------------- listing + def _open_items(self, items: Iterator[T]) -> Iterator[T]: + """`items`, refusing to continue once the backend closes.""" + for item in items: + self._check_open() + yield item + def list(self: Store[Listable], directory: str = "") -> Iterator[str]: """Every key under `directory`, relative to this store's root, in no particular order. The directory is checked when `list` is called; iterating after the store closes raises - `StoreClosedError`. + `StoreClosedError`. Listings hold no slot of the context's limiter. """ paths = self._backend.list(self._directory(directory)) return self._open_items(relativize(path, self._root) for path in paths) @@ -479,12 +587,13 @@ def list_dir(self: Store[Listable], directory: str = "") -> Iterator[ListItem]: @final class AsyncStore(_View[B_co]): - """A backend seen from a root directory, through coroutines. + """A backend seen from a root directory, under a context, through coroutines. - The async twin of `Store`: the same operations under the same names, awaited, over the - backend's async capabilities (`ReadableAsync`, `WritableAsync`, ...). Listings and lazy batches - are async iterators. Construct one from a backend, or take `store.as_async()`; `as_sync()` - goes back. Close it with `await store.aclose()` or `async with`. + The async twin of `Store`: the same operations under the same names, awaited. A backend's async + methods are used where it has them; otherwise its sync methods run on the context's thread + pool, so a backend that only blocks needs no async code. Listings and lazy batches are async + iterators. Construct one from a backend, or take `store.as_async()`; `as_sync()` goes back. + Close it with `await store.aclose()` or `async with`. Parameters ---------- @@ -495,16 +604,27 @@ class AsyncStore(_View[B_co]): read_only Refuse writes through this store. A backend that can't write makes every store over it read-only. + context + The resources operations draw on. Left out, `Context.default()`. """ __slots__ = () - def __init__(self, backend: B_co, *, root: str = "", read_only: bool = False) -> None: - super().__init__(backend, root=root, read_only=read_only) + def __init__( + self, + backend: B_co, + *, + root: str = "", + read_only: bool = False, + context: Context | None = None, + ) -> None: + super().__init__(backend, root=root, read_only=read_only, context=context) def as_sync(self) -> Store[B_co]: - """A `Store` view on the same backend and root; it doesn't own the backend.""" - view = Store(self._backend, root=self._root, read_only=self._read_only) + """A `Store` view on the same backend, root and context; it doesn't own the backend.""" + view = Store( + self._backend, root=self._root, read_only=self._read_only, context=self._context + ) view._owner = False return view @@ -534,7 +654,11 @@ async def __aexit__( ) -> None: await self.aclose() - # ----------------------------------------------------------------- batch results + # ----------------------------------------------------------------- batch plumbing + + async def _in_thread(self, fn: Callable[[*Ts], R], *args: *Ts) -> R: + """`fn(*args)` on the context's pool, holding a slot taken on the pool thread.""" + return await self._context.threads.run(_limited, self._limiter, fn, *args) async def _streamed( self, batch: Sequence[T], results: AsyncIterator[tuple[int, R]] @@ -567,102 +691,195 @@ async def _finished( raise _batch_error(group, batch, completed, completed.keys()) from None return [(request, completed[index]) for index, request in enumerate(batch)] + # ----------------------------------------------------------------- operations on backend paths + + async def _get( + self: AsyncStore[ReadableAsync] | AsyncStore[Readable], + path: str, + byte_range: ByteRange | None, + ) -> Found | Missing: + backend = self._backend + if isinstance(backend, ReadableAsync): + async with self._slot(): + return await backend.get_async(path, byte_range) + return await self._in_thread(backend.get, path, byte_range) + + async def _get_into( + self: AsyncStore[ReadableIntoAsync[O]] | AsyncStore[ReadableInto[O]], + path: str, + out: O, + byte_range: ByteRange | None, + ) -> Copied | Missing: + backend = self._backend + if isinstance(backend, ReadableIntoAsync): + async with self._slot(): + return await backend.get_into_async(path, out, byte_range) + return await self._in_thread(backend.get_into, path, out, byte_range) + + async def _put( + self: AsyncStore[WritableAsync] | AsyncStore[Writable], path: str, value: memoryview + ) -> Written: + backend = self._backend + if isinstance(backend, WritableAsync): + async with self._slot(): + return await backend.put_async(path, value) + return await self._in_thread(backend.put, path, value) + + async def _delete( + self: AsyncStore[WritableAsync] | AsyncStore[Writable], path: str + ) -> Deleted | Missing: + backend = self._backend + if isinstance(backend, WritableAsync): + async with self._slot(): + return await backend.delete_async(path) + return await self._in_thread(backend.delete, path) + # ----------------------------------------------------------------- reads async def get( - self: AsyncStore[ReadableAsync], key: str, *, byte_range: ByteRange | None = None + self: AsyncStore[ReadableAsync] | AsyncStore[Readable], + key: str, + *, + byte_range: ByteRange | None = None, ) -> Found | Missing: """As `Store.get`.""" - return await self._backend.get_async(self._path(key), byte_range) + return await self._get(self._path(key), byte_range) async def get_if_changed( - self: AsyncStore[ReadableAsync], + self: AsyncStore[ReadableAsync] | AsyncStore[Readable], key: str, generation: Generation, *, byte_range: ByteRange | None = None, ) -> Found | Missing | Unchanged: """As `Store.get_if_changed`.""" - return await self._backend.get_if_changed_async(self._path(key), generation, byte_range) + path = self._path(key) + backend = self._backend + if isinstance(backend, ReadableAsync): + async with self._slot(): + return await backend.get_if_changed_async(path, generation, byte_range) + return await self._in_thread(backend.get_if_changed, path, generation, byte_range) async def get_ranges( - self: AsyncStore[ReadableAsync], key: str, ranges: Sequence[ByteRange] + self: AsyncStore[ReadableAsync] | AsyncStore[Readable], + key: str, + ranges: Sequence[ByteRange], ) -> FoundRanges | Missing: """As `Store.get_ranges`.""" - return await self._backend.get_ranges_async(self._path(key), ranges) + path = self._path(key) + backend = self._backend + if isinstance(backend, ReadableAsync): + async with self._slot(): + return await backend.get_ranges_async(path, ranges) + return await self._in_thread(backend.get_ranges, path, ranges) async def get_into( - self: AsyncStore[ReadableIntoAsync[O]], + self: AsyncStore[ReadableIntoAsync[O]] | AsyncStore[ReadableInto[O]], key: str, out: O, *, byte_range: ByteRange | None = None, ) -> Copied | Missing: """As `Store.get_into`.""" - return await self._backend.get_into_async(self._path(key), out, byte_range) + return await self._get_into(self._path(key), out, byte_range) - async def head(self: AsyncStore[ReadableAsync], key: str) -> ObjectMetadata | Missing: + async def head( + self: AsyncStore[ReadableAsync] | AsyncStore[Readable], key: str + ) -> ObjectMetadata | Missing: """As `Store.head`.""" - return await self._backend.head_async(self._path(key)) + path = self._path(key) + backend = self._backend + if isinstance(backend, ReadableAsync): + async with self._slot(): + return await backend.head_async(path) + return await self._in_thread(backend.head, path) - async def exists(self: AsyncStore[ReadableAsync], key: str) -> bool: + async def exists(self: AsyncStore[ReadableAsync] | AsyncStore[Readable], key: str) -> bool: """As `Store.exists`.""" return not isinstance(await self.head(key), Missing) # ----------------------------------------------------------------- writes - async def put(self: AsyncStore[WritableAsync], key: str, value: Buffer) -> Written: + async def put( + self: AsyncStore[WritableAsync] | AsyncStore[Writable], key: str, value: Buffer + ) -> Written: """As `Store.put`.""" - return await self._backend.put_async( - self._writable_path(key), byte_view(value, writable=False) - ) + return await self._put(self._writable_path(key), byte_view(value, writable=False)) async def put_if_absent( - self: AsyncStore[WritableAsync], key: str, value: Buffer + self: AsyncStore[WritableAsync] | AsyncStore[Writable], key: str, value: Buffer ) -> Written | Rejected: """As `Store.put_if_absent`.""" - return await self._backend.put_if_absent_async( - self._writable_path(key), byte_view(value, writable=False) - ) + path = self._writable_path(key) + view = byte_view(value, writable=False) + backend = self._backend + if isinstance(backend, WritableAsync): + async with self._slot(): + return await backend.put_if_absent_async(path, view) + return await self._in_thread(backend.put_if_absent, path, view) async def put_if_match( - self: AsyncStore[WritableIfMatchAsync], key: str, value: Buffer, generation: Generation + self: AsyncStore[WritableIfMatchAsync] | AsyncStore[WritableIfMatch], + key: str, + value: Buffer, + generation: Generation, ) -> Written | Rejected | Missing: """As `Store.put_if_match`.""" - return await self._backend.put_if_match_async( - self._writable_path(key), byte_view(value, writable=False), generation - ) - - async def delete(self: AsyncStore[WritableAsync], key: str) -> Deleted | Missing: + path = self._writable_path(key) + view = byte_view(value, writable=False) + backend = self._backend + if isinstance(backend, WritableIfMatchAsync): + async with self._slot(): + return await backend.put_if_match_async(path, view, generation) + return await self._in_thread(backend.put_if_match, path, view, generation) + + async def delete( + self: AsyncStore[WritableAsync] | AsyncStore[Writable], key: str + ) -> Deleted | Missing: """As `Store.delete`.""" - return await self._backend.delete_async(self._writable_path(key)) + return await self._delete(self._writable_path(key)) async def delete_if_match( - self: AsyncStore[DeletableIfMatchAsync], key: str, generation: Generation + self: AsyncStore[DeletableIfMatchAsync] | AsyncStore[DeletableIfMatch], + key: str, + generation: Generation, ) -> Deleted | Rejected | Missing: """As `Store.delete_if_match`.""" - return await self._backend.delete_if_match_async(self._writable_path(key), generation) + path = self._writable_path(key) + backend = self._backend + if isinstance(backend, DeletableIfMatchAsync): + async with self._slot(): + return await backend.delete_if_match_async(path, generation) + return await self._in_thread(backend.delete_if_match, path, generation) - async def delete_dir(self: AsyncStore[WritableAsync], directory: str = "") -> None: + async def delete_dir( + self: AsyncStore[WritableAsync] | AsyncStore[Writable], directory: str = "" + ) -> None: """As `Store.delete_dir`.""" - await self._backend.delete_dir_async(self._writable_directory(directory)) + path = self._writable_directory(directory) + backend = self._backend + if isinstance(backend, WritableAsync): + async with self._slot(): + await backend.delete_dir_async(path) + return + await self._in_thread(backend.delete_dir, path) # ----------------------------------------------------------------- batches def get_many( - self: AsyncStore[ReadableAsync], requests: Iterable[Q] + self: AsyncStore[ReadableAsync] | AsyncStore[Readable], requests: Iterable[Q] ) -> AsyncIterator[tuple[Q, Found | Missing]]: - """As `Store.get_many`.""" + """As `Store.get_many`; the reads run as tasks, at most the limiter's limit at once.""" batch = list(requests) self._check_open() paths = [self._read_request(request) for request in batch] backend = self._backend if isinstance(backend, ReadableManyAsync): - return self._streamed(batch, backend.get_many_async(paths)) - return self._streamed(batch, run_many_async(backend.get_async, paths)) + return self._streamed(batch, backend.get_many_async(paths, self._context)) + return self._streamed(batch, run_many_async(self._get, paths, self._limiter.limit)) def get_into_many( - self: AsyncStore[ReadableIntoAsync[O]], + self: AsyncStore[ReadableIntoAsync[O]] | AsyncStore[ReadableInto[O]], requests: Iterable[tuple[str, O] | tuple[str, O, ByteRange | None]], ) -> AsyncIterator[tuple[tuple[str, O] | tuple[str, O, ByteRange | None], Copied | Missing]]: """As `Store.get_into_many`.""" @@ -672,48 +889,69 @@ def get_into_many( backend = self._backend if isinstance(backend, ReadableIntoManyAsync): many = cast("ReadableIntoManyAsync[O]", backend) - return self._streamed(batch, many.get_into_many_async(paths)) - return self._streamed(batch, run_many_async(backend.get_into_async, paths)) + return self._streamed(batch, many.get_into_many_async(paths, self._context)) + return self._streamed(batch, run_many_async(self._get_into, paths, self._limiter.limit)) async def put_many( - self: AsyncStore[WritableAsync], items: Iterable[tuple[str, Buffer]] + self: AsyncStore[WritableAsync] | AsyncStore[Writable], + items: Iterable[tuple[str, Buffer]], ) -> list[tuple[str, Written]]: """As `Store.put_many`.""" - batch = list(items) - keys = [key for key, _ in batch] - paths = self._writable_paths(keys) - requests = [ - (path, byte_view(value, writable=False)) - for path, (_, value) in zip(paths, batch, strict=True) - ] + keys, requests = self._put_requests(items) backend = self._backend if isinstance(backend, WritableManyAsync): - return await self._finished(keys, backend.put_many_async(requests)) - return await self._finished(keys, run_many_async(backend.put_async, requests)) + return await self._finished(keys, backend.put_many_async(requests, self._context)) + return await self._finished(keys, run_many_async(self._put, requests, self._limiter.limit)) async def delete_many( - self: AsyncStore[WritableAsync], keys: Iterable[str] + self: AsyncStore[WritableAsync] | AsyncStore[Writable], keys: Iterable[str] ) -> list[tuple[str, Deleted | Missing]]: """As `Store.delete_many`.""" batch = list(keys) paths = self._writable_paths(batch) backend = self._backend if isinstance(backend, WritableManyAsync): - return await self._finished(batch, backend.delete_many_async(paths)) + return await self._finished(batch, backend.delete_many_async(paths, self._context)) + requests = [(path,) for path in paths] return await self._finished( - batch, run_many_async(backend.delete_async, [(path,) for path in paths]) + batch, run_many_async(self._delete, requests, self._limiter.limit) ) # ----------------------------------------------------------------- listing - def list(self: AsyncStore[ListableAsync], directory: str = "") -> AsyncIterator[str]: + async def _open_items(self, items: AsyncIterator[T]) -> AsyncIterator[T]: + async for item in items: + self._check_open() + yield item + + async def _listed_in_thread( + self, fn: Callable[[str], Iterator[T]], directory: str + ) -> AsyncIterator[T]: + """A sync listing, taken whole on the context's pool.""" + for item in await self._context.threads.run(_list_all, fn, directory): + yield item + + def list( + self: AsyncStore[ListableAsync] | AsyncStore[Listable], directory: str = "" + ) -> AsyncIterator[str]: """As `Store.list`.""" - paths = self._backend.list_async(self._directory(directory)) - return self._open_items_async(_relativized(paths, self._root)) + path = self._directory(directory) + backend = self._backend + if isinstance(backend, ListableAsync): + paths = backend.list_async(path) + else: + paths = self._listed_in_thread(backend.list, path) + return self._open_items(_relativized(paths, self._root)) - def list_dir(self: AsyncStore[ListableAsync], directory: str = "") -> AsyncIterator[ListItem]: + def list_dir( + self: AsyncStore[ListableAsync] | AsyncStore[Listable], directory: str = "" + ) -> AsyncIterator[ListItem]: """As `Store.list_dir`.""" - return self._open_items_async(self._backend.list_dir_async(self._directory(directory))) + path = self._directory(directory) + backend = self._backend + if isinstance(backend, ListableAsync): + return self._open_items(backend.list_dir_async(path)) + return self._open_items(self._listed_in_thread(backend.list_dir, path)) def _batch_error( @@ -737,6 +975,6 @@ async def _relativized(paths: AsyncIterator[str], root: str) -> AsyncIterator[st def _restore_store( - cls: type[_View[Backend]], backend: Backend, root: str, read_only: bool + cls: type[_View[Backend]], backend: Backend, root: str, read_only: bool, context: Context ) -> _View[Backend]: - return cls(backend, root=root, read_only=read_only) + return cls(backend, root=root, read_only=read_only, context=context) diff --git a/packages/zarr-storage/src/zarr_storage/backends/_local.py b/packages/zarr-storage/src/zarr_storage/backends/_local.py index e6a22f68eb..9521876845 100644 --- a/packages/zarr-storage/src/zarr_storage/backends/_local.py +++ b/packages/zarr-storage/src/zarr_storage/backends/_local.py @@ -2,7 +2,6 @@ from __future__ import annotations -import asyncio import contextlib import errno import os @@ -12,18 +11,14 @@ import threading import time import uuid -from concurrent.futures import ThreadPoolExecutor -from functools import partial from pathlib import Path -from typing import TYPE_CHECKING, ParamSpec, TypedDict, TypeVar +from typing import TYPE_CHECKING, TypedDict, TypeVar -from zarr_storage._batch import run_many, run_many_async from zarr_storage._buffers import byte_view from zarr_storage._byte_range import resolve from zarr_storage._errors import ( InvalidKeyError, KeyConflictError, - StoreClosedError, UnsupportedOperationError, ) from zarr_storage._types import ( @@ -42,7 +37,7 @@ from zarr_storage.backends._common import IfMatch, always, if_absent if TYPE_CHECKING: - from collections.abc import AsyncIterator, Callable, Generator, Iterator, Sequence + from collections.abc import Callable, Generator, Iterator, Sequence from io import BufferedReader from typing_extensions import Buffer @@ -73,14 +68,12 @@ ) _WINDOWS_FORBIDDEN_CHARACTERS = frozenset('<>:"\\|?*') | frozenset(chr(i) for i in range(32)) -P = ParamSpec("P") R = TypeVar("R") T = TypeVar("T") class _State(TypedDict): root: str - concurrency: int closed: bool @@ -396,24 +389,19 @@ class LocalBackend: create Create `root` if it doesn't exist. With `create=False`, a missing root raises `FileNotFoundError`. - concurrency - The number of threads that run the async methods' file operations, and the number of - operations a batch (`get_many`, `put_many`, ...) runs at once. """ def __init__( - self, root: str | os.PathLike[str], *, create: bool = True, concurrency: int = 16 + self, + root: str | os.PathLike[str], + *, + create: bool = True, ) -> None: - if concurrency < 1: - raise ValueError(f"concurrency must be >= 1; got {concurrency}.") self._root = Path(root).resolve() if create: self._root.mkdir(parents=True, exist_ok=True) elif not self._root.is_dir(): raise FileNotFoundError(f"The root directory {str(self._root)!r} does not exist.") - self._concurrency = concurrency - self._executor: ThreadPoolExecutor | None = None - self._executor_lock = threading.Lock() self._probe_lock = threading.Lock() self._exact_times: bool | None = None self._closed = False @@ -427,19 +415,20 @@ def __repr__(self) -> str: return f"LocalBackend({str(self._root)!r})" def __getstate__(self) -> _State: - return {"root": str(self._root), "concurrency": self._concurrency, "closed": self._closed} + return {"root": str(self._root), "closed": self._closed} def __setstate__(self, state: _State) -> None: self._root = Path(state["root"]) - self._concurrency = state["concurrency"] - self._executor = None - self._executor_lock = threading.Lock() self._probe_lock = threading.Lock() self._exact_times = None self._closed = state["closed"] # --- Backend + @property + def resource_key(self) -> str: + return "local" + @property def closed(self) -> bool: return self._closed @@ -449,15 +438,8 @@ def writable(self) -> bool: return True def close(self) -> None: - with self._executor_lock: - self._closed = True - executor, self._executor = self._executor, None - if executor is not None: - executor.shutdown(wait=True) - - async def aclose(self) -> None: - """`close` on a worker thread: it waits for running file operations to finish.""" - await asyncio.to_thread(self.close) + """Refuse further operations; operations already running finish.""" + self._closed = True # --- paths @@ -710,115 +692,6 @@ def list_dir(self, directory: str) -> Iterator[ListItem]: return iter(()) return _classify(entries) - # --- batches: the operations above, `concurrency` at once on the backend's threads - - def get_many( - self, requests: Sequence[tuple[str, ByteRange | None]] - ) -> Iterator[tuple[int, Found | Missing]]: - return run_many(self.get, requests, self._pool(), self._concurrency) - - def get_into_many( - self, requests: Sequence[tuple[str, Buffer, ByteRange | None]] - ) -> Iterator[tuple[int, Copied | Missing]]: - return run_many(self.get_into, requests, self._pool(), self._concurrency) - - def put_many(self, requests: Sequence[tuple[str, memoryview]]) -> Iterator[tuple[int, Written]]: - return run_many(self.put, requests, self._pool(), self._concurrency) - - def delete_many(self, paths: Sequence[str]) -> Iterator[tuple[int, Deleted | Missing]]: - return run_many(self.delete, [(path,) for path in paths], self._pool(), self._concurrency) - - # --- async twins: run the sync methods on the backend's threads - - def _pool(self) -> ThreadPoolExecutor: - with self._executor_lock: - if self._closed: - raise StoreClosedError(f"{self!r} is closed.") - if self._executor is None: - self._executor = ThreadPoolExecutor( - max_workers=self._concurrency, thread_name_prefix="zarr-storage-local" - ) - return self._executor - - async def _run(self, fn: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -> T: - loop = asyncio.get_running_loop() - return await loop.run_in_executor(self._pool(), partial(fn, *args, **kwargs)) - - async def get_async(self, path: str, byte_range: ByteRange | None) -> Found | Missing: - return await self._run(self.get, path, byte_range) - - async def get_if_changed_async( - self, path: str, generation: Generation, byte_range: ByteRange | None - ) -> Found | Missing | Unchanged: - return await self._run(self.get_if_changed, path, generation, byte_range) - - async def get_ranges_async( - self, path: str, ranges: Sequence[ByteRange] - ) -> FoundRanges | Missing: - return await self._run(self.get_ranges, path, ranges) - - async def head_async(self, path: str) -> ObjectMetadata | Missing: - return await self._run(self.head, path) - - async def get_into_async( - self, path: str, out: Buffer, byte_range: ByteRange | None - ) -> Copied | Missing: - return await self._run(self.get_into, path, out, byte_range) - - async def put_async(self, path: str, value: memoryview) -> Written: - return await self._run(self.put, path, value) - - async def put_if_absent_async(self, path: str, value: memoryview) -> Written | Rejected: - return await self._run(self.put_if_absent, path, value) - - async def put_if_match_async( - self, path: str, value: memoryview, generation: Generation - ) -> Written | Rejected | Missing: - return await self._run(self.put_if_match, path, value, generation) - - async def delete_async(self, path: str) -> Deleted | Missing: - return await self._run(self.delete, path) - - async def delete_if_match_async( - self, path: str, generation: Generation - ) -> Deleted | Rejected | Missing: - return await self._run(self.delete_if_match, path, generation) - - async def delete_dir_async(self, directory: str) -> None: - await self._run(self.delete_dir, directory) - - def get_many_async( - self, requests: Sequence[tuple[str, ByteRange | None]] - ) -> AsyncIterator[tuple[int, Found | Missing]]: - return run_many_async(self.get_async, requests, self._concurrency) - - def get_into_many_async( - self, requests: Sequence[tuple[str, Buffer, ByteRange | None]] - ) -> AsyncIterator[tuple[int, Copied | Missing]]: - return run_many_async(self.get_into_async, requests, self._concurrency) - - def put_many_async( - self, requests: Sequence[tuple[str, memoryview]] - ) -> AsyncIterator[tuple[int, Written]]: - return run_many_async(self.put_async, requests, self._concurrency) - - def delete_many_async( - self, paths: Sequence[str] - ) -> AsyncIterator[tuple[int, Deleted | Missing]]: - return run_many_async(self.delete_async, [(path,) for path in paths], self._concurrency) - - async def list_async(self, directory: str) -> AsyncIterator[str]: - for path in await self._run(_materialize, self.list(directory)): - yield path - - async def list_dir_async(self, directory: str) -> AsyncIterator[ListItem]: - for item in await self._run(_materialize, self.list_dir(directory)): - yield item - - -def _materialize(items: Iterator[T]) -> list[T]: - return list(items) - def _walk(directory: Path, prefix: str) -> Iterator[str]: try: diff --git a/packages/zarr-storage/src/zarr_storage/backends/_memory.py b/packages/zarr-storage/src/zarr_storage/backends/_memory.py index e7b9f7e335..69b5f0b231 100644 --- a/packages/zarr-storage/src/zarr_storage/backends/_memory.py +++ b/packages/zarr-storage/src/zarr_storage/backends/_memory.py @@ -79,6 +79,10 @@ def __setstate__(self, state: _State) -> None: # --- Backend + @property + def resource_key(self) -> str: + return "memory" + @property def closed(self) -> bool: return self._closed diff --git a/packages/zarr-storage/src/zarr_storage/backends/_obstore.py b/packages/zarr-storage/src/zarr_storage/backends/_obstore.py index 0bdb8fbae4..e2038b06a3 100644 --- a/packages/zarr-storage/src/zarr_storage/backends/_obstore.py +++ b/packages/zarr-storage/src/zarr_storage/backends/_obstore.py @@ -24,7 +24,6 @@ ) from obstore.store import HTTPStore, LocalStore -from zarr_storage._batch import run_many_async from zarr_storage._buffers import byte_view, copy_into from zarr_storage._byte_range import resolve from zarr_storage._errors import InvalidKeyError, KeyConflictError, UnsupportedOperationError @@ -267,18 +266,15 @@ class ObjectStoreBackend: The obstore store. writable Whether to accept writes. Defaults to `False` for `HTTPStore`, `True` otherwise. - concurrency - The number of requests an async batch (`get_many_async`, `put_many_async`, ...) keeps in - flight. Sync batches run one request at a time. """ def __init__( - self, store: ObjectStore, *, writable: bool | None = None, concurrency: int = 32 + self, + store: ObjectStore, + *, + writable: bool | None = None, ) -> None: - if concurrency < 1: - raise ValueError(f"concurrency must be >= 1; got {concurrency}.") self._store = store - self._concurrency = concurrency self._writable = not isinstance(store, HTTPStore) if writable is None else writable # Only a filesystem refuses a path that lies under a file; elsewhere a refusal is an error. self._filesystem = isinstance(store, LocalStore) @@ -289,6 +285,12 @@ def store(self) -> ObjectStore: """The obstore store.""" return self._store + @property + def resource_key(self) -> str: + """`"obstore."` and the store's class name, so that, for instance, every S3 backend shares + one limit by default, as TensorStore's `s3_request_concurrency` does.""" + return f"obstore.{type(self._store).__name__}" + def __repr__(self) -> str: return f"ObjectStoreBackend({self._store!r})" @@ -716,28 +718,6 @@ async def delete_dir_async(self, directory: str) -> None: with contextlib.suppress(FileNotFoundError): await obstore.delete_async(self._store, path) - # --- async batches: the async operations, `concurrency` at once - - def get_many_async( - self, requests: Sequence[tuple[str, ByteRange | None]] - ) -> AsyncIterator[tuple[int, Found | Missing]]: - return run_many_async(self.get_async, requests, self._concurrency) - - def get_into_many_async( - self, requests: Sequence[tuple[str, Buffer, ByteRange | None]] - ) -> AsyncIterator[tuple[int, Copied | Missing]]: - return run_many_async(self.get_into_async, requests, self._concurrency) - - def put_many_async( - self, requests: Sequence[tuple[str, memoryview]] - ) -> AsyncIterator[tuple[int, Written]]: - return run_many_async(self.put_async, requests, self._concurrency) - - def delete_many_async( - self, paths: Sequence[str] - ) -> AsyncIterator[tuple[int, Deleted | Missing]]: - return run_many_async(self.delete_async, [(path,) for path in paths], self._concurrency) - # --- Listable def _paths(self, directory: str) -> Iterator[str]: diff --git a/packages/zarr-storage/src/zarr_storage/testing/_api.py b/packages/zarr-storage/src/zarr_storage/testing/_api.py index 49ca4415e6..e4f1a5cbd4 100644 --- a/packages/zarr-storage/src/zarr_storage/testing/_api.py +++ b/packages/zarr-storage/src/zarr_storage/testing/_api.py @@ -13,15 +13,10 @@ from zarr_storage._protocols import ( Listable, - ListableAsync, Readable, - ReadableAsync, ReadableInto, - ReadableIntoAsync, Writable, - WritableAsync, WritableIfMatch, - WritableIfMatchAsync, ) from zarr_storage._store import AsyncStore @@ -59,15 +54,12 @@ class FullBackend( Writable, WritableIfMatch, Listable, - ReadableAsync, - ReadableIntoAsync[Buffer], - WritableAsync, - WritableIfMatchAsync, - ListableAsync, Protocol, ): - """A backend with every capability the suite needs of all backends, sync and async. + """A backend with every capability the suite needs of all backends. + Only the sync protocols: `AsyncStore` uses a backend's async twins where it has them and runs + its sync methods on its context's thread pool otherwise, so the suite's async run covers both. `DeletableIfMatch` is left out, since object stores lack it; `delete_if_match` reaches it. """ diff --git a/packages/zarr-storage/tests/test_batch.py b/packages/zarr-storage/tests/test_batch.py index 40b6b43c9b..c88d7918bc 100644 --- a/packages/zarr-storage/tests/test_batch.py +++ b/packages/zarr-storage/tests/test_batch.py @@ -5,13 +5,13 @@ import asyncio import threading import time -from concurrent.futures import ThreadPoolExecutor from typing import TYPE_CHECKING import pytest from zarr_storage import BatchError from zarr_storage._batch import run_many, run_many_async +from zarr_storage._resources import ThreadPool if TYPE_CHECKING: from collections.abc import AsyncIterator @@ -64,8 +64,11 @@ def _run(tracker: _Tracker, mode: str, window: int, n: int, into: list[tuple[int if mode == "serial": into.extend(run_many(tracker, requests)) elif mode == "threads": - with ThreadPoolExecutor(8) as executor: - into.extend(run_many(tracker, requests, executor, window)) + pool = ThreadPool(8) + try: + into.extend(run_many(tracker, requests, pool, window)) + finally: + pool.shutdown() else: asyncio.run(_collect(run_many_async(tracker.call_async, requests, window), into)) diff --git a/packages/zarr-storage/tests/test_local.py b/packages/zarr-storage/tests/test_local.py index a041d7a058..2699181b86 100644 --- a/packages/zarr-storage/tests/test_local.py +++ b/packages/zarr-storage/tests/test_local.py @@ -145,11 +145,6 @@ def test_overlong_key_raises(tmp_path: Path) -> None: Store(LocalBackend(tmp_path)).put("a" * 1000, b"x") -def test_zero_concurrency_raises(tmp_path: Path) -> None: - with pytest.raises(ValueError, match="concurrency"): - LocalBackend(tmp_path, concurrency=0) - - def _create( store: Store[LocalBackend], key: str, barrier: threading.Barrier, results: list[bool] ) -> None: diff --git a/packages/zarr-storage/tests/test_resources.py b/packages/zarr-storage/tests/test_resources.py new file mode 100644 index 0000000000..aa6d925bb9 --- /dev/null +++ b/packages/zarr-storage/tests/test_resources.py @@ -0,0 +1,431 @@ +"""Contexts and their resources, alone and as stores draw on them.""" + +from __future__ import annotations + +import asyncio +import pickle +import threading +import time +from functools import partial +from typing import TYPE_CHECKING + +import pytest +from obstore.store import MemoryStore + +from zarr_storage import Context, Found, Limiter, Missing, Store, ThreadPool +from zarr_storage._resources import default_limiter +from zarr_storage.backends import LocalBackend, MemoryBackend, ObjectStoreBackend + +if TYPE_CHECKING: + from collections.abc import Callable + from pathlib import Path + + from zarr_storage import ByteRange + from zarr_storage.testing import FullBackend + + +class _Peak: + """Counts callers inside a region, and the most there ever were at once.""" + + def __init__(self) -> None: + self._lock = threading.Lock() + self._inside = 0 + self.most = 0 + self.done = 0 + + def enter(self) -> None: + with self._lock: + self._inside += 1 + self.most = max(self.most, self._inside) + + def leave(self) -> None: + with self._lock: + self._inside -= 1 + self.done += 1 + + +def _hold_sync(limiter: Limiter, peaks: list[_Peak]) -> None: + with limiter.slot(): + for peak in peaks: + peak.enter() + time.sleep(0.002) + for peak in peaks: + peak.leave() + + +async def _hold_async(limiter: Limiter, peak: _Peak) -> None: + async with limiter.slot(): + peak.enter() + await asyncio.sleep(0.002) + peak.leave() + + +async def _many_async(limiter: Limiter, peak: _Peak, n: int) -> None: + await asyncio.gather(*(_hold_async(limiter, peak) for _ in range(n))) + + +def _run_loop(limiter: Limiter, peak: _Peak, n: int) -> None: + asyncio.run(_many_async(limiter, peak, n)) + + +def _finishes(fn: Callable[[], object]) -> bool: + """Whether `fn()` returns within ten seconds; a deadlock fails a test instead of hanging it.""" + worker = threading.Thread(target=fn, daemon=True) + worker.start() + worker.join(10) + return not worker.is_alive() + + +def _join_all(threads: list[threading.Thread]) -> None: + for thread in threads: + thread.start() + for thread in threads: + thread.join() + + +# ---------------------------------------------------------------------------- Limiter + + +@pytest.mark.parametrize("limit", [1, 3]) +def test_limiter_bounds_threads_and_tasks_together(limit: int) -> None: + """Threads and tasks on two event loops never hold more than `limit` slots at once.""" + limiter, peak = Limiter(limit), _Peak() + workers = [threading.Thread(target=_hold_sync, args=(limiter, [peak])) for _ in range(8)] + workers += [threading.Thread(target=_run_loop, args=(limiter, peak, 8)) for _ in range(2)] + _join_all(workers) + assert (peak.done, peak.most, limiter.in_use) == (24, limit, 0) + + +def test_nested_limiter_holds_slots_of_both() -> None: + """A limiter `within` another is bounded by its own limit, and shares the wider one.""" + wide = Limiter(3) + narrow = Limiter(2, within=wide) + narrow_peak, wide_peak = _Peak(), _Peak() + workers = [ + threading.Thread(target=_hold_sync, args=(narrow, [narrow_peak, wide_peak])) + for _ in range(8) + ] + workers += [threading.Thread(target=_hold_sync, args=(wide, [wide_peak])) for _ in range(8)] + _join_all(workers) + assert (narrow_peak.most, wide_peak.most, wide_peak.done) == (2, 3, 16) + assert (narrow.in_use, wide.in_use) == (0, 0) + + +def _wait_for_slot(limiter: Limiter) -> None: + with limiter.slot(): + pass + + +def test_waiting_on_a_narrow_limit_holds_none_of_the_wide_one() -> None: + """Slots are taken innermost first, so a burst queued on a narrow limit leaves the wide one + free for everyone else.""" + wide = Limiter(2) + narrow = Limiter(1, within=wide) + with narrow.slot(): + waiter = threading.Thread(target=_wait_for_slot, args=(narrow,)) + waiter.start() + waiter.join(0.05) + assert waiter.is_alive() + assert wide.in_use == 1 + waiter.join(10) + assert (narrow.in_use, wide.in_use) == (0, 0) + + +async def _take_in_order(limiter: Limiter, name: str, order: list[str]) -> None: + async with limiter.slot(): + order.append(name) + + +async def _first_come_first_served() -> list[str]: + limiter, order = Limiter(1), [] + await limiter.acquire_async() + tasks = [asyncio.create_task(_take_in_order(limiter, name, order)) for name in "abc"] + await asyncio.sleep(0) + limiter.release() + await asyncio.gather(*tasks) + return order + + +def test_limiter_serves_waiters_in_order() -> None: + assert asyncio.run(_first_come_first_served()) == ["a", "b", "c"] + + +async def _cancel_while_waiting(granted_first: bool) -> int: + limiter = Limiter(1) + await limiter.acquire_async() + waiting = asyncio.create_task(limiter.acquire_async()) + await asyncio.sleep(0) + if granted_first: + limiter.release() # the slot passes to the waiter, which is then cancelled + waiting.cancel() + with pytest.raises(asyncio.CancelledError): + await waiting + if not granted_first: + limiter.release() + return limiter.in_use + + +@pytest.mark.parametrize("granted_first", [False, True]) +def test_cancelled_waiter_gives_back_its_slot(granted_first: bool) -> None: + assert asyncio.run(_cancel_while_waiting(granted_first)) == 0 + + +async def _cancel_between_slots() -> tuple[int, int]: + wide = Limiter(1) + narrow = Limiter(1, within=wide) + await wide.acquire_async() + entering = asyncio.create_task(_wait_for_slot_async(narrow)) + await asyncio.sleep(0) # holds the narrow slot, waits for the wide one + entering.cancel() + with pytest.raises(asyncio.CancelledError): + await entering + wide.release() + return narrow.in_use, wide.in_use + + +async def _wait_for_slot_async(limiter: Limiter) -> None: + async with limiter.slot(): + pass + + +def test_cancelled_while_taking_a_nested_slot_gives_back_the_inner_one() -> None: + assert asyncio.run(_cancel_between_slots()) == (0, 0) + + +@pytest.mark.parametrize(("limit", "error"), [(0, ValueError), (1.5, TypeError), (True, TypeError)]) +def test_limiter_of_a_bad_limit_raises(limit: object, error: type[Exception]) -> None: + with pytest.raises(error): + Limiter(limit) # pyright: ignore[reportArgumentType] + + +def test_releasing_a_slot_not_held_raises() -> None: + with pytest.raises(RuntimeError, match="not held"): + Limiter(1).release() + + +# ---------------------------------------------------------------------------- ThreadPool + + +def _worker_view(pool: ThreadPool, other: ThreadPool) -> tuple[bool, bool]: + return pool.in_worker(), other.in_worker() + + +def test_thread_pool_knows_its_own_threads() -> None: + pool, other = ThreadPool(1), ThreadPool(1) + try: + assert not pool.in_worker() + assert pool.submit(_worker_view, pool, other).result() == (True, False) + finally: + pool.shutdown() + other.shutdown() + + +@pytest.mark.parametrize(("size", "error"), [(0, ValueError), (2.0, TypeError)]) +def test_thread_pool_of_a_bad_size_raises(size: object, error: type[Exception]) -> None: + with pytest.raises(error): + ThreadPool(size) # pyright: ignore[reportArgumentType] + + +def test_shutting_down_the_default_pool_raises() -> None: + with pytest.raises(ValueError, match="shared"): + ThreadPool.default().shutdown() + + +# ---------------------------------------------------------------------------- Context + + +def test_context_resolves_resources_through_its_parents() -> None: + """Own resources first, then the parent's, then the process-wide defaults.""" + pool, a, b = ThreadPool(2), Limiter(4), Limiter(5) + default = Context.default() + assert default.threads is ThreadPool.default() + assert default.limiter("x") is default_limiter("x") + own = Context(threads=pool, limits={"a": a}) + assert (own.threads, own.limiter("a"), own.limiter("x")) == (pool, a, default_limiter("x")) + child = own.child(limits={"b": b}) + assert (child.threads, child.limiter("a"), child.limiter("b")) == (pool, a, b) + assert child.child(threads=ThreadPool.default()).threads is ThreadPool.default() + narrowed = child.limited("a", 2).limiter("a") + assert (narrowed.limit, narrowed.within) == (2, a) + + +@pytest.mark.parametrize( + "make", + [ + partial(Context, limits={"a": 4}), # pyright: ignore[reportArgumentType] + partial(Context, threads=4), # pyright: ignore[reportArgumentType] + ], + ids=["limit-not-a-limiter", "threads-not-a-pool"], +) +def test_context_of_a_bad_resource_raises(make: Callable[[], Context]) -> None: + with pytest.raises(TypeError): + make() + + +def test_views_carry_their_context() -> None: + """Views keep their context; `with_context` and `limited` change it and nothing else.""" + context = Context(limits={"memory": Limiter(8)}) + store = Store(MemoryBackend(), root="r", context=context) + for view in (store / "a", store.with_read_only(), store.as_async(), store.as_async().as_sync()): + assert view.context is context + other = Context() + assert store.with_context(other).context is other + limited = store.limited(2) + assert limited.context.parent is context + assert limited.context.limiter("memory").limit == 2 + assert limited.context.limiter("memory").within is context.limiter("memory") + assert (limited.root, limited.shares_backend(store)) == ("r", True) + + +def test_pickling_keeps_resources_shared(tmp_path: Path) -> None: + """Stores that shared a context share one after a round trip; defaults stay defaults.""" + wide = Limiter(8) + context = Context(threads=ThreadPool(2), limits={"local": Limiter(3, within=wide)}) + stores = ( + Store(LocalBackend(tmp_path / "a"), context=context), + Store(LocalBackend(tmp_path / "b"), context=context), + Store(LocalBackend(tmp_path / "c")), + ) + a, b, c = pickle.loads(pickle.dumps(stores)) + assert a.context is b.context + assert a.context is not context + limiter = a.context.limiter("local") + assert (a.context.threads.max_workers, limiter.limit, limiter.within.limit) == (2, 3, 8) + assert c.context is Context.default() + + +# ---------------------------------------------------------------------------- stores under limits + + +def _get_in_thread(store: Store[FullBackend], results: list[object]) -> None: + results.append(store.get("k")) + + +async def _get_async_while_held(store: Store[FullBackend]) -> bool: + try: + await asyncio.wait_for(store.as_async().get("k"), 0.05) + except TimeoutError: + return True + return False + + +@pytest.mark.parametrize("backend", ["local", "obstore"]) +def test_operations_wait_for_the_contexts_limiter(tmp_path: Path, backend: str) -> None: + """Sync and async operations, on native-async and sync-only backends, wait for a slot.""" + raw: FullBackend = ( + LocalBackend(tmp_path) if backend == "local" else ObjectStoreBackend(MemoryStore()) + ) + pool, limiter = ThreadPool(2), Limiter(1) + store = Store(raw, context=Context(threads=pool, limits={raw.resource_key: limiter})) + store.put("k", b"x") + limiter.acquire() # some other operation under this context holds the only slot + results: list[object] = [] + reader = threading.Thread(target=_get_in_thread, args=(store, results)) + reader.start() + reader.join(0.05) + assert reader.is_alive() + assert asyncio.run(_get_async_while_held(store)) + limiter.release() + reader.join(10) + pool.shutdown() # the timed-out async read's thread finishes too + assert isinstance(results[0], Found) + assert limiter.in_use == 0 + + +class _PeakBackend(MemoryBackend): + """A memory backend whose reads take a moment and record how many overlap.""" + + def __init__(self) -> None: + super().__init__() + self.peak = _Peak() + + def get(self, path: str, byte_range: ByteRange | None) -> Found | Missing: + self.peak.enter() + time.sleep(0.005) + self.peak.leave() + return super().get(path, byte_range) + + +def test_limited_view_bounds_its_batches() -> None: + backend = _PeakBackend() + store = Store(backend, context=Context(threads=ThreadPool(8))) + keys = [f"k{i}" for i in range(24)] + store.put_many((key, b"x") for key in keys) + assert len(list(store.limited(2).get_many(keys))) == 24 + assert backend.peak.most == 2 + backend.peak.most = 0 + assert len(list(store.get_many(keys))) == 24 + assert backend.peak.most > 2 + + +class _RecordingBackend(LocalBackend): + """A local backend that records which threads its reads run on.""" + + def __init__(self, root: Path) -> None: + super().__init__(root) + self.read_on: set[int] = set() + + def get(self, path: str, byte_range: ByteRange | None) -> Found | Missing: + self.read_on.add(threading.get_ident()) + return super().get(path, byte_range) + + +def _batch_here(store: Store[_RecordingBackend], keys: list[str]) -> int: + assert all(isinstance(result, Found) for _, result in store.get_many(keys)) + return threading.get_ident() + + +def test_batch_from_a_thread_of_its_own_pool_runs_on_that_thread(tmp_path: Path) -> None: + """A batch started on one of its pool's threads must not wait for the pool's other threads, + which may all be waiting the same way; it runs its reads on the thread that started it.""" + pool = ThreadPool(2) + backend = _RecordingBackend(tmp_path) + store = Store(backend, context=Context(threads=pool)) + keys = [f"k{i}" for i in range(4)] + store.put_many((key, b"x") for key in keys) + backend.read_on.clear() + try: + worker = pool.submit(_batch_here, store, keys).result(timeout=10) + finally: + pool.shutdown(wait=False) + assert backend.read_on == {worker} + + +def _read_batch(store: Store[LocalBackend], results: list[object]) -> None: + results.extend(result for _, result in store.get_many(["k"])) + + +def _read_async(store: Store[LocalBackend], results: list[object]) -> None: + results.append(asyncio.run(store.as_async().get("k"))) + + +def _settle() -> None: + time.sleep(0.05) + + +def test_async_read_of_a_sync_backend_holds_no_slot_while_it_waits_for_a_thread( + tmp_path: Path, +) -> None: + """One thread, one slot. A sync batch's read is queued on the thread, then an async read of the + sync backend is queued behind it. Had the async read taken the slot before its turn on the + thread, the batch's read, which runs first, would wait for that slot forever.""" + pool, limiter = ThreadPool(1), Limiter(1) + store = Store(LocalBackend(tmp_path), context=Context(threads=pool, limits={"local": limiter})) + store.put("k", b"x") + gate = threading.Event() + pool.submit(gate.wait) # keeps the only thread busy while the reads queue up + batch: list[object] = [] + single: list[object] = [] + batch_reader = threading.Thread(target=_read_batch, args=(store, batch), daemon=True) + async_reader = threading.Thread(target=_read_async, args=(store, single), daemon=True) + batch_reader.start() + _settle() # the batch's read is queued + async_reader.start() + _settle() # the async read is queued behind it + gate.set() + batch_reader.join(10) + async_reader.join(10) + assert not batch_reader.is_alive() + assert not async_reader.is_alive() + assert [type(r) for r in batch + single] == [Found, Found] + assert limiter.in_use == 0 diff --git a/packages/zarr-storage/tests/test_store.py b/packages/zarr-storage/tests/test_store.py index 63582e5066..15d1f4eaa1 100644 --- a/packages/zarr-storage/tests/test_store.py +++ b/packages/zarr-storage/tests/test_store.py @@ -29,7 +29,7 @@ if TYPE_CHECKING: from collections.abc import Iterator, Sequence - from zarr_storage import ByteRange + from zarr_storage import ByteRange, Context class _ReadOnlyBackend: @@ -38,6 +38,10 @@ class _ReadOnlyBackend: def __init__(self, values: dict[str, bytes]) -> None: self._values = values + @property + def resource_key(self) -> str: + return "test" + @property def closed(self) -> bool: return False @@ -117,17 +121,17 @@ def test_get_into_many_device_buffers() -> None: """Without a batch method of its own, the backend's `get_into` serves `get_into_many`.""" store = Store(_DeviceBackend({"a/k": b"0123", "a/j": b"45"})) / "a" whole, part = _DeviceBuffer(4), _DeviceBuffer(1) - results = [ - (request[0], result) + results = { + request[0]: result for request, result in store.get_into_many( [("k", whole), ("j", part, SuffixRequest(1)), ("missing", _DeviceBuffer(1))] ) - ] - assert results == [ - ("k", Copied(4, Generation(0))), - ("j", Copied(1, Generation(0))), - ("missing", Missing()), - ] + } + assert results == { + "k": Copied(4, Generation(0)), + "j": Copied(1, Generation(0)), + "missing": Missing(), + } assert (whole.download(4), part.download(1)) == (b"0123", b"5") @@ -139,7 +143,7 @@ def __init__(self, values: dict[str, bytes]) -> None: self.asked: list[tuple[str, ByteRange | None]] = [] def get_many( - self, requests: Sequence[tuple[str, ByteRange | None]] + self, requests: Sequence[tuple[str, ByteRange | None]], context: Context ) -> Iterator[tuple[int, Found | Missing]]: self.asked.extend(requests) for index in reversed(range(len(requests))): @@ -179,7 +183,7 @@ def get(self, path: str, byte_range: ByteRange | None) -> Found | Missing: def test_put_many_failure_stops_the_batch_and_reports_it() -> None: backend = _FailingBackend() - store = Store(backend) / "r" + store = Store(backend).limited(1) / "r" # one at a time, so the order is known with pytest.raises(BatchError) as caught: store.put_many([("ok", b"1"), ("bad", b"2"), ("later", b"3")]) error = caught.value @@ -192,7 +196,7 @@ def test_put_many_failure_stops_the_batch_and_reports_it() -> None: def test_get_many_failure_raises_after_the_reads_before_it() -> None: backend = _FailingBackend() - store = Store(backend) / "r" + store = Store(backend).limited(1) / "r" # one at a time, so the order is known store.put("ok", b"1") backend.attempted.clear() results = store.get_many(["ok", "bad", "later"]) @@ -334,4 +338,5 @@ async def _static_surface_checks( store.get_async("k") # pyright: ignore[reportAttributeAccessIssue] store.aclose() # pyright: ignore[reportAttributeAccessIssue] astore.close() # pyright: ignore[reportAttributeAccessIssue] - AsyncStore(_ReadOnlyBackend({})).get("k") # pyright: ignore[reportAttributeAccessIssue] + await AsyncStore(_ReadOnlyBackend({})).get("k") # a sync backend serves async callers + await AsyncStore(_ReadOnlyBackend({})).put("k", b"") # pyright: ignore[reportAttributeAccessIssue]