Skip to content

feat: show pages in the conversation and add a built-in browser - #207

Merged
GoDiao merged 1 commit into
mainfrom
feat/browser-tab
Oct 29, 2026
Merged

GoDiao merged 1 commit into
mainfrom
feat/browser-tab

Conversation

@GoDiao

@GoDiao GoDiao commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Closes #201
Closes #202
Part of #189

What changed

Pages in the conversation (ShowUI)

  • A new ShowUI tool lets the agent build a page from ShowAI components (MIT, vendored in vendor/showai, unchanged from upstream commit 9be7012): charts and the G2 chart family, flowcharts, sequence and architecture diagrams, mind maps, step-by-step walkthroughs, comparison matrices, roadmaps, 3D scenes, PDFs and more. The media components that need the network (image, gallery, video, audio, bookmark) are left out.
  • For what no component can draw, the agent writes an html block, which runs in its own nested sandbox. Every html block must say which components were considered and why none fits.
  • The tool enforces its steps: list the components before the first page, and read a component's format before using it. A call that skips a step, uses an unknown component, or gives G2 records without the fields the chart draws is rejected and nothing is saved.
  • Each create or change is a read-only revision file in the session data folder. The conversation shows the latest write to a page per turn as a card; the card can be collapsed, enlarged, or opened as a side panel tab.
  • Pages are served by the sidecar under /ui-pages/ with a sandbox CSP (sandbox allow-scripts, connect-src 'none', no forms or popups) and render in a sandboxed iframe that cannot reach the app, its storage or the network.

Side panel as tabs

  • Each session has a strip of tabs: changes, the file tree, single files, web pages, terminals and pages, opened from a new tab page or from the chat. Tabs are saved per session and restored after a restart, except terminals.
  • The panel runs the full window height with its tabs in the title row. One button at the top right opens and closes it; the separate terminal button is gone. An expanded panel shows the session title as the way back to the chat.
  • The H5 page offers file and page tabs only.

Built-in browser

  • A web page tab opens a local dev server, an HTML file in the project (through a root-scoped dreamcoder-project:// protocol) or an https site, one <webview> per tab on a persistent partition. Other schemes are refused or offered to the system browser.
  • The Electron main process forces the guest sandbox, denies permissions, and checks navigation, redirects, subframes and popups.
  • The browsing session presents itself as plain Chrome (the app and Electron tokens are removed from the user agent); without this, Google refused to sign in.

Fixes

  • Tool calls that arrived after a reply fell back to a non-streaming request were missing from the conversation.
  • Read rejected text files when a model sent an empty pages value.
  • A tool group was marked as failed even when a failed call was followed by a successful call of the same tool. ShowUI rows now say what each call did.

README (both languages), CHANGELOG, third-party notices and the screenshots are updated.

Size

macOS arm64, electron:pack of main and this branch:

main this branch
.dmg 150.8 MB 156.5 MB (+5.7 MB)
.app 335.7 MiB 353.7 MiB (+18.0 MiB)

The page reader adds 15.6 MiB to the app (PDF engine, external diagrams and 3D load only when a page uses them) and the sidecar grows by 2.3 MiB.

How I tested

  • bun test: 1483 pass, 0 fail
  • cd desktop && bun run lint: no errors
  • cd desktop && bun run test -- --run: 1618 passed, 4 skipped, 159 files
  • Touched test files also pass in shuffled order (bun test --randomize, vitest --sequence.shuffle with several seeds).
  • Real-Chrome tests (show-ui-isolation.test.ts) check that a page cannot reach its parent, the app storage, the sidecar API or the network, that the bundled reader renders, and that an html block grows with its content and stays offline. Every newly enabled component's official example was also rendered in headless Chrome.
  • By hand in the macOS dev app: pages with charts, walkthroughs, 3D scenes and html blocks; tabs, restore after restart, expand and close; a dev server, a project HTML file and https sites in the browser, Google sign-in, menus over the page, session switching, panel resizing and clearing browsing data.

Not in this PR

The agent can answer with pages built from ShowAI components through a
new ShowUI tool. A page appears in a card in the conversation, each
change is a new revision, and the card can be enlarged or opened in the
side panel. Pages run in a sandboxed frame with no network access and
are stored as read-only revision files in the session data folder. The
tool lists the components and requires the agent to read a component's
format before using it; html blocks must say why no component fits.

The side panel becomes a strip of tabs per session: changes, the file
tree, files, web pages, terminals and pages, saved across restarts
except terminals. It runs the full window height with its tabs in the
title row. Web page tabs host a browser that opens local dev servers,
project HTML files and https sites, keeps sign-ins, and presents itself
as plain Chrome.

Also: tool calls that arrive after a non-streaming fallback are kept,
Read accepts an empty page range, a tool group that recovered is no
longer marked as failed, and the README and screenshots are updated.

Closes #201
Closes #202
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Interactive pages in the conversation with ShowUI (#189 phase 2) Browser tab in the workspace panel (#189 phase 1)

1 participant