Skip to content

[bugfix] canvas: show connected references on text-to-speech panels (#119) - #312

Open
Birfy wants to merge 3 commits into
ThinkFlowLab:mainfrom
Birfy:fix/119-tts-reference-visibility
Open

Birfy wants to merge 3 commits into
ThinkFlowLab:mainfrom
Birfy:fix/119-tts-reference-visibility

Conversation

@Birfy

@Birfy Birfy commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The audio generation panel hid the entire reference manager whenever the node was in text-to-speech (reading) mode. Canvas connections to a speech node were still saved, but the panel showed no thumbnails and no remove control, so the saved relationship could not be checked or edited there.

composerReferenceVisibility keeps the strip visible in reading mode when connections already exist, with a bilingual note that these references are not sent to the speech model. The empty “add reference” control is hidden in reading mode, so the panel does not invite a new connection while saying those references are unused. The existing @ mention control stays hidden in reading and music modes. Music models that accept an audio reference, and first-last video frame slots, are unchanged. Removing a thumbnail still updates the project references and canvas edges.

Reading-mode copy is used only when the reference kind itself is valid for the model or catalog path. A video or other unsupported kind on a TTS node reports the specific kind error (errors.rules.kindReference / generation.issue.unsupportedKind, or the existing video-reference rule) instead of “this mode does not use this reference”. An allowed kind in reading mode still gets the unused-in-reading message, and backendReferences / generationActiveReferences still exclude those inputs.

Review follow-up

Addresses the review on 2026-11-17 (Asia/Shanghai) by Probiotics5333:

  1. Merged current origin/main (58b6a96d) into this branch with a regular merge (no force-push) so CI can re-run against latest main. The earlier browser (voice) failure was in tests/browser/voice_library.mjs, which this PR does not touch.
  2. Hide the empty add control in reading mode; the strip stays only when references already exist.
  3. Do not let the unused-in-reading copy override a more specific kind error.

Linked issue

  • Fixes #119

Labels

  • Labels requested (type:bug, area:canvas, modality:audio, area:i18n); this account could not apply them when opening the PR

Test plan

Local (tip 3b0bc4d1, merged origin/main 58b6a96d, Asia/Shanghai)

node --test tests/composerReferenceVisibility.test.ts tests/backendRules.test.ts tests/generationCatalog.test.ts
# 27 passed, including:
# - reading + 1 incoming → showStrip and showReadingContextNote, no empty add
# - reading + 0 incoming → no strip, no empty add, no context note, no @ mention
# - music without musicReferences stays hidden; music with audio refs still shows the strip and no reading note
# - first-last video keeps the frame-slot path (no general strip)
# - image/video mentions unchanged
# - reading service with empty allowedKinds: image/video get kind or video-reference errors, not “不使用该参考”
# - reading service that allows image: image gets “不使用该参考”, video still gets the kind/video error; backendReferences returns []
# - catalog: audio in TTS still “不使用该参考”; video/image in TTS get “未核实支持该素材类型”
# - existing: reading needs text and voice; audio refs cannot replace reading text; generationActiveReferences is []

npm test
# 842 passed, 1 failed
# The failure is tests/browserHost.test.ts “installed server env removes source imports, ambient Node PATH and agent credentials”
# (asserts PATH’s node is null, this box has /usr/bin/node). Same failure is present on origin/main in this environment and is unrelated to this change.

npx tsc --noEmit
# clean

npm run check:i18n
# i18n: 3037 keys, 0 problem(s), 165 Chinese source lines (baseline 168)

Python / pre-commit were not re-run for this follow-up: the change is TypeScript UI and rule-message selection only. Full pytest was previously green on the original tip after fixing a stale editable install; this follow-up does not touch Python.

Browser verification, isolated fixture only (no user project, no paid generation):

# Isolated API on 8019; Vite on 5179 with OPENDIRECTOR_API_URL=http://127.0.0.1:8019
CREATION_WEB_URL=http://127.0.0.1:5179 CREATION_API_URL=http://127.0.0.1:8019 \
PLAYWRIGHT_MODULE=/tmp/od-pw/node_modules/playwright-core/index.mjs \
CHROMIUM_PATH=/home/box/.cache/ms-playwright/chromium-1248/chrome-linux64/chrome \
node tests/browser/tts_reading_references.mjs
# {"passed":true,"usedServiceModel":false,"jobs":0}  (~9.4s)

The isolated fixture had no preconfigured speech service, so this run used local TTS (audioMode: "tts"), which takes the same reading UI path as a configured service speech model. After removing the last reference, the strip and context note are gone and the empty “添加参考素材” control stays hidden.

Left to CI

  • Official Release checks on this updated branch, including browser (voice) after the merge of current main, and the isolated browser lane that runs tests/browser/tts_reading_references.mjs.

Evidence that it works

  • Media attached, or N/A because: the change is covered by unit tests plus a passing isolated Chromium journey (tts_reading_references.mjs, zh-CN, headless). No screenshots were attached. A reviewer can repeat that script against the isolated fixture, or open a TTS audio node that already has a canvas image connection and confirm the strip, note, and remove control, without submitting to a paid provider.

Provider verification

  • Real-provider calls are described honestly, or none were made

No live or paid provider calls were made. The browser run did not submit a generation job.

Bilingual UI

  • Both locales are updated, or no user-facing copy changed

No new strings in this follow-up. Existing generation.reference.readingContext and generation.issue.audioModeReference / kind-error strings are reused.

…hinkFlowLab#119)

Reading mode hid the whole reference strip, so saved canvas connections
could not be inspected or removed. Keep those references visible, say
clearly that they are not sent to the speech model, and leave music and
first-last behaviour unchanged.
@Probiotics5333

Copy link
Copy Markdown
Collaborator

The functional change looks right, and the current blocker is CI rather than this PR's source.

1. The failing check is in a file this PR does not touch. browser (voice) fails in tests/browser/voice_library.mjs, waiting 45s for Saved on this device — the same symptom class #250/#255 addressed, and #255 is merged while current main is green. Rebase onto the current main and rerun before treating it as a regression introduced here, and please report the new result in this thread.

2. Reading mode still offers an "add reference" entry. showEmptyAdd is not restricted for reading, so the panel tells the user these references are not sent to the model while still inviting them to add one. If that is deliberate (context kept for the agent), one sentence in the PR or in the note would explain it; otherwise I suggest showing the strip only when references already exist and hiding the add control in reading mode.

3. unusedInReading overrides the more specific kind error. A video reference attached to a TTS node used to report that the reference kind is unsupported; it now reports that it is not sent to the model, which hides a genuinely wrong connection. Using the reading-mode copy only when the kind itself is valid would keep the diagnosis.

The extraction into composerReferenceVisibility with unit tests is a clear improvement over the previous nested conditions, and keeping @ mentions hidden in reading and music modes preserves existing behaviour.

…hinkFlowLab#119)

Reading mode only shows the reference strip when connections already exist.
Unsupported kinds keep their kind-specific errors instead of the unused-in-reading copy.
@github-actions github-actions Bot added area:canvas Canvas, nodes and direct manipulation area:i18n Localisation and bilingual UI modality:audio Speech, music or audio processing type:bug Something is broken labels Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:canvas Canvas, nodes and direct manipulation area:i18n Localisation and bilingual UI modality:audio Speech, music or audio processing type:bug Something is broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants