Skip to content

Add concurrent generation and cancellation on the vLLM backends, and a per-request provider mode - #40

Merged
emiehling merged 1 commit into
generative-computing:mainfrom
emiehling:feat/continuous-batching
Oct 27, 2026
Merged

emiehling merged 1 commit into
generative-computing:mainfrom
emiehling:feat/continuous-batching

Add per-request generation on the vLLM backends and an opt-in per-req…

8899c9c
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 warning and 1 notice
pytest
succeeded Oct 27, 2026 in 3m 44s