feat(client): scikit-learn estimators, API keys and table files

CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ## [Unreleased]
 ### Added
+- **scikit-learn estimators** (#63): `TabctxClassifier` and `TabctxRegressor` (`tabctx/estimators.py`, `from tabctx import ...`; `pip install "tabctx[client]"` adds scikit-learn and pyarrow). They take DataFrames, numpy arrays or lists of rows with text columns, dates and missing values as they are; labels are checked as scikit-learn checks them and come back exactly as given (sent as class indices); `predict_proba` follows `classes_`; clone, pipelines and model selection work. Test columns match the training columns by name when both have names, else by position (with a warning); a column that was text at fit stays text at predict, whatever its new values look like. Each table is one Parquet file (`tabctx/table_files.py`, float32 numbers, timestamps and durations as such); predicts over `max_request_bytes` (default 29 MiB, under the Vector gateway's 30 MiB) are split until each part fits, and a training table over it is refused before sending. The context id comes from the table, labels, model and task; a predict that finds its context gone (the server's "no cached context") fits it again from the kept training file and retries once, with the model it was fit with.
+- **Client:** `TabctxClient(api_key=...)` (sent as `Authorization: Bearer`, for the Vector gateway); `base_url` and `api_key` default to `TABCTX_BASE_URL` and `TABCTX_API_KEY`. `fit_file()` / `predict_file()` send a table file (path, streamed, or bytes) in the request as multipart/form-data; `predict(model=...)` for gateways, which route predicts by model. Redirects are refused rather than followed (urllib would carry the key to the new host, and turn a POST into a GET). The key never appears in the client's repr and is dropped from pickles: an unpickled client (a saved model, a worker with `n_jobs > 1`) reads `TABCTX_API_KEY`, and a 401 without a key says so; copies (sklearn's clone) share the client. 403 maps to `PermissionError` like 401.
 - **A table file in the fit or predict request itself.** `/v1/tabctx/fit` and `/v1/tabctx/predict` take multipart/form-data as well as JSON: the same fields as form fields, the table as a `file` part (`curl -F dataset_id=churn -F target=churned -F file=@train.parquet`), so a file is one call instead of upload-then-reference. The file streams to the replica's disk under the upload size cap, is read, and is deleted. OpenAPI documents both body types.
 - **Parquet and compressed CSV.** Files (sent with the request or through `/v1/tabctx/upload`) may be CSV (plain, gzip or zstd) or Parquet, detected from the content (`serve/files.py`, pyarrow, replacing the numpy CSV parser). A 5,000 x 500 float table is 51 MB as JSON and 13 MB as Parquet; reading it takes 16 ms. Dates and times become seconds, stored from the training minimum so float32 keeps sub-second resolution (`TableSchema.offsets`, for any float64 column past 2^24); header names are stripped; decimal targets are numbers. Labels keep their text as written (`"1.0"`, `"01"`, `"true"` are classes of their own; `"NA"` can be a class among text labels); a missing value among numeric labels, in any spelling, is a 422 naming the row, and regression with text labels is a 422. Reading a CSV peaks at about 3x its size in host memory (arrow's parser; Parquet much less), which matters only for multi-GB direct uploads. `pyarrow` and `python-multipart` join the `serve` extra.
 - **Multipart bodies stream to disk under the upload cap** (`serve/multipart.py`): the file part goes straight to the upload writer as it arrives, so an oversized body stops at the cap (or is refused from its `Content-Length` before any is read), a good one is written once, and disk writes run off the event loop. Tenant checks run before the body is read.

README.md

@@ -29,11 +29,45 @@ endpoint**, chat-completions style -- requests pick a model by exact id
 one memory budget. A protocol seam (`backends/base.py`) keeps adding
 models cheap.
 
+## Quickstart: as a scikit-learn model
+
+```bash
+pip install "tabctx[client]"
+export TABCTX_BASE_URL=https://proxy.vectorinstitute.ai   # or your own deployment
+export TABCTX_API_KEY=...                                  # a gateway key, if it needs one
+```
+
+```python
+from sklearn.datasets import fetch_openml
+from sklearn.model_selection import train_test_split
+from tabctx import TabctxClassifier
+
+X, y = fetch_openml("credit-g", version=1, as_frame=True, return_X_y=True)
+X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
+
+clf = TabctxClassifier(model="tabpfn-3.5").fit(X_train, y_train)
+clf.score(X_test, y_test)          # the training table stays cached on a GPU
+clf.predict_proba(X_test)
+```
+
+`TabctxClassifier` and `TabctxRegressor` (`tabctx/estimators.py`) take
+DataFrames, arrays or lists of rows with text columns, dates and missing
+values as they are, return labels as given, and work with clone,
+pipelines and `cross_val_score`. Each table travels as one Parquet file;
+large predicts are split under the request limit (`max_request_bytes`),
+and a context the server dropped is fit again from the training file
+the estimator keeps. The API key never appears in a repr and is left out
+of pickles, so with `n_jobs > 1` or a saved model set `TABCTX_API_KEY`.
+
+The lower-level client speaks the HTTP API directly:
+
 ```python
 client = TabctxClient("http://localhost:8000")
 client.models()                                    # -> tabicl-v2, tabpfn-3.5
 client.fit(X, y, dataset_id="churn", model="tabpfn-3.5")
 client.predict("churn", X_test)                    # served by tabpfn-3.5
+client.fit_file("train.parquet", target="churned", dataset_id="churn", model="tabicl-v2")
+client.predict_file("churn", "test.csv", model="tabicl-v2")
 ```
 
 Deploy one model (`TABCTX_BACKEND=tabicl`) or several
@@ -90,7 +124,9 @@ deployment is exactly fast enough.
 
 *Figure: a request enters through Ray Serve's consistent-hash proxy
 (affinity on the session header) into a `TabctxService` replica, which
-resolves the tenant, parses inline JSON or an uploaded CSV, and coalesces
+resolves the tenant, reads inline JSON rows or a table file (CSV or
+Parquet, sent with the request or uploaded first), encodes text columns
+with the context's schema, and coalesces
 same-context predicts. With several GPUs, a `MultiDeviceEngine` places
 each fit on one GPU and routes predicts to the GPU holding the context,
 from lock-free snapshots; each GPU's engine runs in its own worker
@@ -106,8 +142,9 @@ The unit everything revolves around is a **context**: one training
 table, encoded once by one model, addressed by `dataset_id` (scoped per
 tenant) and reused by every later `predict()` against it. A context
 carries its payload (the fitted estimator with its kv cache), the model
-that fit it, its shape and, when it came from a CSV, its column names,
-plus a measured byte cost. All models deployed on a device share one
+that fit it, its shape, its schema (column names, and the codes of its
+text columns, so new rows are encoded the same way), plus a measured
+byte cost. All models deployed on a device share one
 cache and one memory budget, so admission control sees the whole
 device's usage regardless of which model a request picks.
 

pyproject.toml

@@ -48,6 +48,9 @@ serve = [
     "pyarrow>=14",
     "python-multipart>=0.0.9",
 ]
+# The scikit-learn-style estimators (tabctx.estimators); pyarrow sends
+# tables as Parquet (without it, gzip CSV).
+client = ["scikit-learn>=1.6", "pyarrow>=14"]
 dev = ["pytest>=7.0", "pytest-cov", "httpx"]
 
 [dependency-groups]

src/tabctx/__init__.py

@@ -24,6 +24,27 @@
 )
 from tabctx.types import EngineStats, PredictOutcome, Task
 
+# The scikit-learn estimators need scikit-learn, which tabctx itself
+# doesn't: imported on first use (`from tabctx import TabctxClassifier`).
+_ESTIMATORS = ("TabctxClassifier", "TabctxRegressor")
+
+
+def __getattr__(name: str):
+    if name in _ESTIMATORS:
+        try:
+            from tabctx import estimators
+        except ImportError as e:
+            raise ImportError(
+                f"{name} needs scikit-learn and pyarrow: pip install 'tabctx[client]'"
+            ) from e
+        return getattr(estimators, name)
+    raise AttributeError(f"module 'tabctx' has no attribute {name!r}")
+
+
+def __dir__() -> list[str]:
+    return sorted([*globals(), *_ESTIMATORS])
+
+
 __all__ = [
     "AdmissionRejected",
     "BackendComputeError",