Skip to content

feat: collapse the vLLM-Omni presets into one local service - #325

Open
Xingkai98 wants to merge 4 commits into
ThinkFlowLab:mainfrom
Xingkai98:feat/vllm-omni-local-preset
Open

Xingkai98 wants to merge 4 commits into
ThinkFlowLab:mainfrom
Xingkai98:feat/vllm-omni-local-preset

Conversation

@Xingkai98

@Xingkai98 Xingkai98 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Rebased onto 838f8d7a (#266 merged); the capability-check controls added by #266 are unchanged.

Summary

Issue #224 V1: one vLLM-Omni service preset instead of seven per-stage presets.

The seven presets (image, speech, music, score, video, segmentation, depth) each carried a single protocol, so a multi-stage vLLM-Omni server had to be configured as seven separate connections. This adds vllm-omni-local: a self-hosted preset whose default URL is http://127.0.0.1:8000/v1, with a per-kind stage table.

Kind Stage protocol → endpoints
chat openai → /chat/completions
image openai → /images/generations, /images/edits
vllm_omni_segmentation → /images/segmentations, /videos/segmentations
video vllm_omni → /videos
vllm_omni_segmentation → /images/segmentations, /videos/segmentations
vllm_omni_depth → /videos/depth
audio openai (TTS) → /audio/speech
vllm_omni_music → /audio/speech
vllm_omni_score → /audio/transcriptions
vision vllm_omni_segmentation → /images/segmentations, /videos/segmentations

The connection form shows a stage selector when a kind has more than one stage, so music, score, segmentation and depth remain reachable from the single preset. The selector also shows the stage's endpoint paths in both languages. Opening a new image/video/audio form on the preset now starts with its URL, self_hosted, no-auth mode and first-stage protocol.

Saved connections written by the retired presets keep their identity: vllm_omni, vllm_omni_music, vllm_omni_score, vllm_omni_segmentation and vllm_omni_depth resolve to the local preset, and presetProtocol() keeps the saved stage instead of resetting it. The retired image and speech presets used the shared openai protocol, so a self-hosted openai connection resolves to the local preset only at a loopback URL (localhost, 127.0.0.1, ::1); a generic OpenAI-compatible service on a LAN or remote URL still reads as custom.

ASR is deliberately not a stage: upstream vllm-omni does not serve /audio/transcriptions yet (S1 in vllm-project/vllm-omni#10007) and this application has no ASR adapter. SheetSage2 score uses the same path with its own protocol.

Linked issue

  • Refs #224 (V1; V3 docs and V4 validation remain tracked there)

Labels

  • type:feat, area:provider (applied by the reviewer-routing workflow)

Test plan

  • Local:
    • node --test tests/providerPresets.test.ts → 26 pass, 0 fail
    • npx tsc --noEmit → clean
    • node scripts/check-i18n.mjs → 3227 keys, 0 problems
    • Full frontend suite on Node 24.22.1 → 925 pass, 1 fail. The single failure is tests/browserHost.test.ts ("installed server env removes source imports, ambient Node PATH and agent credentials"), which reports the host's /usr/bin/node where the test expects none. It reproduces unchanged on unmodified main at 838f8d7a, so this PR does not introduce it.
    • Browser workflows updated for the single preset: tests/browser/provider_settings.mjs, tests/browser/kling_video.mjs and tests/browser/moss_provider.mjs.
  • CI on this revision: all checks pass, including checks / frontend, checks / python, checks / browser (isolated), checks / browser (voice), checks / browser (assistant), checks / static, CodeQL and tests.
  • Not run here: real vLLM-Omni servers.

Evidence that it works

  • Media attached, or N/A because:
    • No browser screenshot was captured in this environment. The preset selection, stage list, saved-stage retention and loopback identity rules are covered by tests/providerPresets.test.ts; the isolated browser workflow asserts the stage selector, its endpoint note, the default URL and saved-provider identity, and passes in CI. The merged feat(providers): discover capabilities and validate generation inputs #266 capability-check paths are untouched.

Provider verification

Bilingual UI

  • Both locales updated: models.service.stage and models.service.stageEndpoints in en-US and zh-CN; check:i18n passes.

Not in this PR

AI assistance

This contribution was made with assistance from Claude Code.

🤖 Generated with Claude Code

Xingkai98 and others added 2 commits November 19, 2026 15:31
The seven per-stage presets each carried a protocol of their own, so a
multi-stage vLLM-Omni server needed seven separate connections. One preset
now maps every stage to its endpoints -- /chat/completions,
/images/generations, /images/edits, /videos, /audio/speech,
/audio/transcriptions, /images/segmentations and /videos/segmentations --
with http://127.0.0.1:8000/v1 as the default connection.

Each stage keeps the protocol its adapter expects, so the form takes the
protocol from the stage instead of the preset as a whole.

Connections saved by the retired presets keep their provider identity: the
five vLLM-Omni protocols resolve to the new preset instead of reading as
"custom". The test that pinned vllm_omni to vllm-omni-video now pins the
single service, and new cases cover the stage table, the per-kind options
and the legacy protocols.
- Offer each per-kind stage through a stage selector instead of pinning
  the first protocol: image (generation, segmentation), video
  (generation, segmentation, depth), and audio (TTS, music, score).
- Keep the local service's URL, auth mode and protocol as the defaults
  when the form opens on a self-hosted preset.
- Resolve saved loopback `openai` stage connections to the local service;
  non-loopback generic OpenAI-compatible services still read as custom.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@Xingkai98
Xingkai98 force-pushed the feat/vllm-omni-local-preset branch from f369b3a to dcea95d Compare November 19, 2026 07:54
@Xingkai98
Xingkai98 marked this pull request as ready for review November 19, 2026 07:57
@github-actions github-actions Bot added area:provider Model providers and generation protocols type:feat New capability or behaviour labels Nov 19, 2026
Xingkai98 and others added 2 commits November 19, 2026 16:08
- Provider settings checks the per-kind stage selector and endpoint note
  instead of the retired per-stage preset names.
- Kling and MOSS flows expect the new default provider name.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Provider settings now removes the model configurations it creates, so the
shared fixture catalogue is left as it found it. The capability workflow
waits up to 30 seconds for the externally created canvas node before
clicking it; the suite can run well past the default 15-second timeout.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider Model providers and generation protocols type:feat New capability or behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant