Skip to content

[bugfix] canvas: show and edit fixed-asset first and last frames (#121) - #319

Open
Birfy wants to merge 3 commits into
ThinkFlowLab:mainfrom
Birfy:fix/121-pinned-frame-slots
Open

Birfy wants to merge 3 commits into
ThinkFlowLab:mainfrom
Birfy:fix/121-pinned-frame-slots

Conversation

@Birfy

@Birfy Birfy commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A video configuration that uses a fixed (pinned) asset as its first or last frame now shows and edits that frame in the slot, and slot edits change the stored reference role.

Before this change (reproduced on main 3e1fc08, see Test plan):

  • the frame slots reopened empty while the prompt still showed the image label (firstFrameNodeId is the only slot field, and the projection only restored follow_current frames);
  • clearing the (apparently empty) slot produced no reference change;
  • a 首尾帧 → 参考模式 → 首尾帧 round trip left the reference at role: reference, so the backend resolved it as context_only instead of a first-frame content input.

Changes:

  • GenerationSettings gains firstFrameReferenceId / lastFrameReferenceId: a slot holds either a canvas node (existing fields) or a pinned reference by ID, never both. Stored canvas documents accept and copy them (dataBoundary.ts).
  • CreationProjection.read fills those fields from active pinned references with a frame role; a follow_current frame in the same slot keeps precedence, as before.
  • canvasReferenceRole decides a pinned reference's role from the slot that names it rather than from the stored role, so clear / choose / swap persist as reference.update and the reference itself is kept.
  • Composer: fixed images in the prompt are offered as slot options (labelled 「固定素材」/ "fixed asset") with a thumbnail loaded through mediaUrl; swap and clear work for both kinds. Removing the fixed image from the prompt empties its slot (pruneFrameReferences).
  • Entering 首尾帧 fills still-empty slots with fixed images in prompt order, after connected images (same default as connected nodes). After a round trip through 参考模式 the frames come back in prompt order; a swapped order is not remembered across the mode switch.
  • backendBlocker rejects a fixed last frame on a model without last_frame, as it already does for a node; the local blocker counts a fixed first frame for "last needs first" / "prompt or first frame".
  • New i18n key generation.frame.pinnedTag (zh-CN + en-US).

Review follow-up

Addresses the review on 2026-11-17 (Asia/Shanghai) by Probiotics5333 (nothing blocking; these three smaller points):

  1. frameSlotPatch now writes an empty string for the field being cleared, which survives JSON.stringify. changeGenerationSettings then drops empty frame-slot keys so stored settings stay omit-empty (compatible with isGenerationSettings). A unit test applies a JSON.parse(JSON.stringify(patch)) partial merge and checks the previous node id is gone.
  2. When projectId is missing, the preview effect calls setPinnedPreviews({}) before returning, so switching projects cannot keep the previous project's thumbnails.
  3. The preview effect depends on a useMemo of the sorted unique asset ids instead of JSON.parse(JSON.stringify(...)).

Also merged current origin/main (58b6a96d) with a regular merge (no force-push).

Linked issue

Labels

  • Labels applied: I can't apply labels from a fork. Suggested type: bug, area: canvas.

Test plan

Original local results (tip 57913be5)

Local (Linux, Node v24.22.1, Python 3.12; isolated creation_server.py on :8121 and Vite on :5221, no real provider calls):

  • node --test tests/pinnedFrameSlots.test.ts → 14/14 pass on the original tip, including failing-on-main reproductions for empty slots, clear/choose/swap, and the mode round trip.
  • Related browser workflows passed on the original tip: first_last_frame_connections.mjs, minimax_h3.mjs, kling_video.mjs, generation_parameters.mjs, canvas_generation_reference.mjs.
  • npm test then: 810 pass / 1 fail (browserHost.test.ts ambient Node PATH; also fails on main here).

Follow-up (tip 47afebdd, merged origin/main 58b6a96d, Asia/Shanghai)

node --test tests/pinnedFrameSlots.test.ts tests/generationCatalog.test.ts
# 26 passed, including the new “JSON-serialized slot patch still clears the other field on merge”

npx tsc --noEmit
# clean

npm test
# 852 passed, 1 failed: tests/browserHost.test.ts “installed server env … ambient Node PATH”
# (asserts PATH’s node is null, this box has /usr/bin/node). Same failure is present on origin/main in this environment.

npm run check:i18n
# i18n: 3037 keys, 0 problem(s), 165 Chinese source lines (baseline 168)

Browser re-run of the official workflow, isolated fixture only (no user project, no paid generation):

# Isolated API on 8019; Vite on 5179 serving this worktree
CREATION_WEB_URL=http://127.0.0.1:5179 CREATION_API_URL=http://127.0.0.1:8019 \
PLAYWRIGHT_MODULE=/tmp/od-pw/node_modules/playwright-core/index.mjs \
CHROMIUM_PATH=/home/box/.cache/ms-playwright/chromium-1248/chrome-linux64/chrome \
BROWSER=chromium \
node tests/browser/pinned_frame_slots.mjs
# “pinned frame slots ok”

Python / pre-commit were not re-run for this follow-up: the change is TypeScript composer/settings-merge only.

Left to CI

Official Release checks on this updated branch, including browser lanes.

Not verified: Agent-created pinned frames were checked only through the same projection path (unit tests build the snapshot directly). No real paid generation was submitted.

Evidence that it works

  • Media N/A here: I can't upload images from the CLI, so the browser workflow above covers the evidence instead. It asserts the slot values, thumbnail naturalWidth > 0 and the stored roles.

Provider verification

  • No real-provider calls. MiniMax-H3 is the fixture config (https://fixture.invalid/v2) and nothing was submitted.

Bilingual UI

  • generation.frame.pinnedTag added to zh-CN and en-US. This follow-up adds no new strings.

…nkFlowLab#121)

A video configuration that used a pinned asset as its first or last frame
showed only a prompt label: the frame slot stayed empty, clearing it changed
nothing, and a reference-mode round trip left the image as a context-only
reference.

Frame slots can now hold a pinned reference by ID (firstFrameReferenceId /
lastFrameReferenceId) alongside the existing node IDs. The projection restores
them, the persisted role follows the slot, the composer lists fixed images as
slot choices with thumbnails, swap and clear cover both kinds, and entering
first-last fills empty slots with fixed images after connected ones.
@Probiotics5333

Copy link
Copy Markdown
Collaborator

Reviewed the change against main and the fixed-asset slot behaviour. Nothing blocking in the code — happy to see this merged once the two smaller points below are answered.

1. Clearing a slot relies on JSON dropping undefined. frameSlotPatch returns { [node]: undefined, [reference]: value }, so moving from a connected node to a fixed asset clears the node field only because the save submits the whole projected spec and a missing key reads as empty. An undefined property does not survive JSON. If a partial-patch writer ever appears (for example an agent command that sends only changed settings fields), the old firstFrameNodeId stays on the server, resolveFrameRole's !node condition fails, and the frame silently resolves to context_only while the UI still shows it in the slot. Expressing the clear explicitly, or documenting that settings are always replaced wholesale, would remove the trap.

2. pinnedPreviews keeps the previous project's map. The effect returns early when projectId is missing, so switching projects can keep thumbnails from the previous project until a new key replaces them. setPinnedPreviews({}) before the early return is enough.

3. JSON.parse(pinnedFrameKey) — serialising the asset-id array into a dependency key and parsing it back works, but a useMemo returning the sorted array reads better and avoids the round trip.

Two things I checked on purpose because they are easy to get wrong, and both are correct:

  • pruneFrameReferences runs unconditionally in applyCommand (not only for frame patches), so deleting a pinned reference from the prompt also clears the slot.
  • frameSlotsInitialized keeps a slot the user cleared from being refilled by fillPinnedFrames.

…hinkFlowLab#121)

Frame slot patches use explicit empty strings and changeGenerationSettings
drops those fields, so a serialized partial patch cannot leave a stale node
or reference id. Switching projects clears pinned thumbnails, and the preview
effect depends on a memoized asset-id list instead of JSON.parse.
@github-actions github-actions Bot added area:canvas Canvas, nodes and direct manipulation type:bug Something is broken labels Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:canvas Canvas, nodes and direct manipulation area:docs Documentation / 文档 type:bug Something is broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] 视频生成:固定素材首尾帧槽位为空且模式切换后丢失首帧角色

2 participants