Feature/caches

docs/adr/AGENTS.md

@@ -32,4 +32,5 @@ Read this index first, then open the records relevant to the change. Before impl
 | [024](024-installer-controller-module.md) | 2026-10-19 | Installer controllers in their own module | `fastapi-startkit init` generates `HomeController` in `app/http/controllers/home_controller.py` (JSON or Inertia variant) instead of inside `routes/api.py`; one shared routes stub imports and registers it, matching the PingCRM layout. |
 | [025](025-model-lifecycle-events.md) | 2026-10-20 | Model lifecycle events, automatic timestamps and datetime casts | `save()` fires `saving → creating/updating → query → created/updated → saved`. Observer changes are persisted, and `updating` fires only when the model is dirty. A new `Model.delete()` fires `deleting → DELETE → deleted`. `observe_events` is async and awaits async handlers. Mass query-builder writes fire no events. Laravel-style auto timestamps: `created_at`/`updated_at` are filled on create unless supplied, `updated_at` moves on update, and `__timestamps__ = False` disables both. `DateCast.set` returns a UTC datetime; `Connection.prepare_binding` passes it unchanged to Postgres/MySQL, and SQLite converts it to a naive UTC string. Postgres registers an asyncpg `timestamp` codec that stores aware values as naive UTC; `timestamptz` keeps asyncpg's codec. `Annotated[T, Cast]` (a `BaseCast` subclass or instance) overrides a column's cast, and `Attribute.__casts__` is removed. |
 | [026](026-schema-column-change.md) | 2026-10-20 | Altering existing columns with `.change()` | `.change()` fully redefines the column, the way Laravel does. MySQL (`MODIFY`) and SQLite (table rebuild) were already correct. Postgres now emits `TYPE ... USING` casts, correct lengths, quoted defaults or `DROP DEFAULT`, `SET`/`DROP NOT NULL`, and replaces the enum check constraint. MSSQL drops the current default constraint (looked up in `sys.default_constraints`), runs one `ALTER COLUMN` per column, then adds `[DF_<table>_<col>]`. Identifiers are bracket- or double-quote-escaped and the MSSQL drop target is built server-side with `QUOTENAME`; changing a Postgres serial column drops its `nextval` default. |
+| [029](029-cache-module.md) | 2026-10-21 | Cache module with remember, memo, locks and tags | New `fastapi_startkit.cache`: async `Cache.get/put/has/forget/flush`, `get` with a callable default, and `remember(key, ttl, factory)` that runs a sync or async factory only on a miss; `remember(key, ttl)` without a factory is a decorator keyed by `key` plus a hash of the call arguments. Stores implement an async contract with a `MISSING` sentinel (cached `None` is a hit): array, file (pickled per-key files), database (masoniteorm table) and redis (named connection from the existing `RedisManager`, `SET PX` TTL, flush by `SCAN` of prefixed keys, never `FLUSHDB`). TTL in seconds or `timedelta`, `None` forever, <=0 forgets. `CacheConfig` via `resolve_config`; `CacheProvider` binds `cache`; facade `remember` resolves lazily. `Cache.memo()` memoizes reads per asyncio task through a `ContextVar` owned by the task, so requests never share memo state. `Cache.lock(name, seconds, owner)` gives atomic, owner-checked, non-re-entrant locks with `get`, `block` (250 ms polling, `LockTimeoutError`), `release`, `force_release` and `async with`, per store: event-loop dict, `fcntl.flock` file, `lock:<name>` row with update-if-expired, Redis `SET NX PX` plus a Lua owner-checked delete. `Cache.tags([...])` uses Laravel's namespaced keys: random tag ids at `<prefix>:tag:<name>:key`, entries at `<prefix>:sha1(ids):<key>`, flush rewrites the ids; tags are sorted and deduplicated. Shipped in slices into `feature/caches`; the former memo, lock and tags ADRs (030, 031, 034) are folded in. |
 | [032](032-migrate-seed-option.md) | 2026-10-21 | db:migrate:fresh and db:migrate:refresh --seed run seeders in the same event loop | `--seed` called the unregistered `seed:run` through Cleo `call` (which also drops the first argument) inside a running event loop, where `db:seed` would call `asyncio.run` again. The seeder selection moves into async `DBSeedCommand.run_seeders`, which `db:seed`, `db:migrate:fresh` and `db:migrate:refresh` all await. `db:migrate:refresh` is now registered. |

fastapi_startkit/src/fastapi_startkit/cache/__init__.py

@@ -0,0 +1,30 @@
+from .array_store import ArrayStore
+from .config import ArrayStoreConfig, CacheConfig, DatabaseStoreConfig, FileStoreConfig, RedisStoreConfig
+from .file_store import FileStore
+from .lock import Lock, LockTimeoutError
+from .manager import CacheManager
+from .memo_store import MemoStore
+from .provider import CacheProvider
+from .repository import Repository
+from .store import MISSING, Store
+from .tagged_store import TaggedStore, TagSet
+
+__all__ = [
+    "MISSING",
+    "ArrayStore",
+    "ArrayStoreConfig",
+    "CacheConfig",
+    "CacheManager",
+    "CacheProvider",
+    "DatabaseStoreConfig",
+    "FileStore",
+    "FileStoreConfig",
+    "Lock",
+    "LockTimeoutError",
+    "MemoStore",
+    "RedisStoreConfig",
+    "Repository",
+    "Store",
+    "TagSet",
+    "TaggedStore",
+]

fastapi_startkit/src/fastapi_startkit/cache/array_store.py

@@ -0,0 +1,62 @@
+from typing import Any
+
+from .lock import Lock
+from .store import MISSING, expired, expires_at
+
+
+class ArrayStore:
+    def __init__(self) -> None:
+        self.storage: dict[str, tuple[float | None, Any]] = {}
+        self.locks: dict[str, tuple[str, float | None]] = {}
+
+    async def get(self, key: str) -> Any:
+        entry = self.storage.get(key)
+        if entry is None:
+            return MISSING
+
+        expiration, value = entry
+        if expired(expiration):
+            del self.storage[key]
+            return MISSING
+        return value
+
+    async def put(self, key: str, value: Any, seconds: float | None) -> bool:
+        self.storage[key] = (expires_at(seconds), value)
+        return True
+
+    async def forget(self, key: str) -> bool:
+        return self.storage.pop(key, MISSING) is not MISSING
+
+    async def flush(self) -> bool:
+        self.storage.clear()
+        return True
+
+    def lock(self, name: str, seconds: float = 0, owner: str | None = None) -> "ArrayLock":
+        return ArrayLock(self, name, seconds, owner)
+
+
+class ArrayLock(Lock):
+    def __init__(self, store: ArrayStore, name: str, seconds: float = 0, owner: str | None = None):
+        super().__init__(name, seconds, owner)
+        self.store = store
+
+    def holder(self) -> str | None:
+        entry = self.store.locks.get(self.name)
+        if entry is None or expired(entry[1]):
+            return None
+        return entry[0]
+
+    async def acquire(self) -> bool:
+        if self.holder() is not None:
+            return False
+        self.store.locks[self.name] = (self.owner, expires_at(self.ttl))
+        return True
+
+    async def release(self) -> bool:
+        if self.holder() != self.owner:
+            return False
+        del self.store.locks[self.name]
+        return True
+
+    async def force_release(self) -> None:
+        self.store.locks.pop(self.name, None)

fastapi_startkit/src/fastapi_startkit/cache/config/__init__.py

@@ -0,0 +1,45 @@
+from dataclasses import dataclass, field
+
+from fastapi_startkit.configuration import Config
+from fastapi_startkit.environment import env
+
+
+@dataclass
+class ArrayStoreConfig(Config):
+    driver: str = "array"
+
+
+@dataclass
+class FileStoreConfig(Config):
+    driver: str = "file"
+    path: str = env("CACHE_FILE_PATH", "storage/framework/cache/data")
+
+
+@dataclass
+class DatabaseStoreConfig(Config):
+    driver: str = "database"
+    connection: str | None = env("DB_CACHE_CONNECTION", None, cast=False)
+    table: str = env("DB_CACHE_TABLE", "caches")
+
+
+@dataclass
+class RedisStoreConfig(Config):
+    driver: str = "redis"
+    connection: str = "cache"
+
+
+@dataclass
+class CacheConfig(Config):
+    default: str = env("CACHE_CONNECTION", "database")
+    prefix: str = env("CACHE_PREFIX", "cache")
+    stores: dict[str, Config] = field(
+        default_factory=lambda: {
+            "array": ArrayStoreConfig(),
+            "file": FileStoreConfig(),
+            "database": DatabaseStoreConfig(),
+            "redis": RedisStoreConfig(),
+        }
+    )
+
+
+__all__ = ["ArrayStoreConfig", "CacheConfig", "DatabaseStoreConfig", "FileStoreConfig", "RedisStoreConfig"]

fastapi_startkit/src/fastapi_startkit/cache/database_store.py

@@ -0,0 +1,94 @@
+import base64
+import math
+import pickle
+import time
+from collections.abc import Callable
+from typing import Any
+
+from sqlalchemy.exc import IntegrityError
+
+from fastapi_startkit.masoniteorm.connections.connection import Connection
+from fastapi_startkit.masoniteorm.models.builder import QueryBuilder
+
+from .lock import Lock
+from .store import MISSING, expired, expires_at
+
+
+def encode(value: Any) -> str:
+    return base64.b64encode(pickle.dumps(value)).decode()
+
+
+class DatabaseStore:
+    def __init__(self, connection: Callable[[], Connection], table: str = "caches"):
+        self.connection = connection
+        self.table = table
+
+    def query(self) -> QueryBuilder[Any]:
+        return self.connection().query().table(self.table)
+
+    async def get(self, key: str) -> Any:
+        builder = self.query().where("key", key).limit(1)
+        row = await builder.connection.select_one(builder.to_qmark(), list(builder.get_bindings()))
+        if row is None:
+            return MISSING
+
+        try:
+            value = pickle.loads(base64.b64decode(row["value"]))
+        except Exception:
+            await self.forget(key)
+            return MISSING
+
+        if expired(row["expiration"]):
+            await self.forget(key)
+            return MISSING
+        return value
+
+    async def put(self, key: str, value: Any, seconds: float | None) -> bool:
+        expiration = expires_at(seconds)
+        record = {"value": encode(value), "expiration": None if expiration is None else math.ceil(expiration)}
+        if await self.query().where("key", key).update(record):
+            return True
+        try:
+            await self.query().insert({"key": key, **record})
+        except IntegrityError:
+            await self.query().where("key", key).update(record)
+        return True
+
+    async def forget(self, key: str) -> bool:
+        return bool(await self.query().where("key", key).delete())
+
+    async def flush(self) -> bool:
+        await self.query().delete()
+        return True
+
+    def lock(self, name: str, seconds: float = 0, owner: str | None = None) -> "DatabaseLock":
+        return DatabaseLock(self, name, seconds, owner)
+
+
+class DatabaseLock(Lock):
+    def __init__(self, store: DatabaseStore, name: str, seconds: float = 0, owner: str | None = None):
+        super().__init__(name, seconds, owner)
+        self.store = store
+        self.key = f"lock:{name}"
+
+    async def acquire(self) -> bool:
+        expiration = expires_at(self.ttl)
+        record = {"value": encode(self.owner), "expiration": None if expiration is None else math.ceil(expiration)}
+        try:
+            await self.store.query().insert({"key": self.key, **record})
+        except IntegrityError:
+            expired_lock = self.store.query().where("key", self.key).where("expiration", "<=", time.time())
+            return bool(await expired_lock.update(record))
+        return True
+
+    async def release(self) -> bool:
+        held = (
+            self.store.query()
+            .where("key", self.key)
+            .where("value", encode(self.owner))
+            .where(lambda query: query.where_null("expiration").or_where("expiration", ">", time.time()))
+        )
+        return bool(await held.delete())
+
+    async def force_release(self) -> None:
+        await self.store.query().where("key", self.key).delete()

fastapi_startkit/src/fastapi_startkit/cache/file_store.py

@@ -0,0 +1,135 @@
+import asyncio
+import hashlib
+import json
+import os
+import pickle
+import re
+import shutil
+import tempfile
+from collections.abc import Callable
+from pathlib import Path
+from typing import Any
+
+from .lock import Lock
+from .store import MISSING, Missing, expired, expires_at
+
+SHARD = re.compile(r"^[0-9a-f]{2}$")
+Holder = tuple[str, float | None] | None
+Decision = tuple[bool, Holder | Missing]
+
+
+class FileStore:
+    def __init__(self, directory: str | Path):
+        self.directory = Path(directory)
+
+    def path(self, key: str) -> Path:
+        digest = hashlib.sha1(key.encode()).hexdigest()
+        return self.directory / digest[:2] / digest[2:4] / digest
+
+    async def get(self, key: str) -> Any:
+        return await asyncio.to_thread(self.read, self.path(key))
+
+    async def put(self, key: str, value: Any, seconds: float | None) -> bool:
+        payload = pickle.dumps((expires_at(seconds), value))
+        await asyncio.to_thread(self.write, self.path(key), payload)
+        return True
+
+    async def forget(self, key: str) -> bool:
+        return await asyncio.to_thread(self.delete, self.path(key))
+
+    async def flush(self) -> bool:
+        await asyncio.to_thread(self.clear)
+        return True
+
+    def read(self, path: Path) -> Any:
+        try:
+            payload = path.read_bytes()
+        except FileNotFoundError:
+            return MISSING
+        try:
+            expiration, value = pickle.loads(payload)
+        except Exception:
+            self.delete(path)
+            return MISSING
+
+        if expired(expiration):
+            self.delete(path)
+            return MISSING
+        return value
+
+    def write(self, path: Path, payload: bytes) -> None:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        descriptor, temporary = tempfile.mkstemp(dir=path.parent, prefix=".tmp-")
+        try:
+            with os.fdopen(descriptor, "wb") as file:
+                file.write(payload)
+            os.replace(temporary, path)
+        except BaseException:
+            Path(temporary).unlink(missing_ok=True)
+            raise
+
+    def delete(self, path: Path) -> bool:
+        try:
+            path.unlink()
+        except FileNotFoundError:
+            return False
+        return True
+
+    def clear(self) -> None:
+        if not self.directory.is_dir():
+            return
+        for shard in self.directory.iterdir():
+            if shard.is_dir() and SHARD.match(shard.name):
+                shutil.rmtree(shard, ignore_errors=True)
+
+    def lock(self, name: str, seconds: float = 0, owner: str | None = None) -> "FileLock":
+        return FileLock(self, name, seconds, owner)
+
+
+class FileLock(Lock):
+    def __init__(self, store: FileStore, name: str, seconds: float = 0, owner: str | None = None):
+        super().__init__(name, seconds, owner)
+        self.path = store.directory / "locks" / hashlib.sha1(name.encode()).hexdigest()
+
+    async def acquire(self) -> bool:
+        def decide(holder: Holder) -> Decision:
+            if holder is not None:
+                return False, MISSING
+            return True, (self.owner, expires_at(self.ttl))
+
+        return await asyncio.to_thread(self.exclusive, decide)
+
+    async def release(self) -> bool:
+        def decide(holder: Holder) -> Decision:
+            if holder is None or holder[0] != self.owner:
+                return False, MISSING
+            return True, None
+
+        return await asyncio.to_thread(self.exclusive, decide)
+
+    async def force_release(self) -> None:
+        await asyncio.to_thread(self.exclusive, lambda holder: (True, None))
+
+    def exclusive(self, decide: Callable[[Holder], Decision]) -> bool:
+        import fcntl
+
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        with open(self.path, "a+b") as file:
+            fcntl.flock(file, fcntl.LOCK_EX)
+            file.seek(0)
+            result, holder = decide(self.read(file.read()))
+            if holder is not MISSING:
+                file.truncate(0)
+                if holder is not None:
+                    file.write(json.dumps(holder).encode())
+                file.flush()
+            return result
+
+    def read(self, payload: bytes) -> Holder:
+        try:
+            owner, expiration = json.loads(payload)
+        except (ValueError, TypeError):
+            return None
+        if not isinstance(owner, str) or not isinstance(expiration, int | float | None) or expired(expiration):
+            return None
+        return owner, expiration