Repository navigation
feat(safari): port the FSB extension to Safari via a native macOS wrapper - #145
Open
LakshmanTurlapati wants to merge 29 commits into
Open
LakshmanTurlapati wants to merge 29 commits into
LakshmanTurlapati wants to merge 29 commits into
Conversation
…pper Add a Safari Web Extension target (safari/FSB Xcode project with a Swift app, extension handler, and native MCP bridge) plus build/release/version scripts. The extension gains a platform adapter that degrades CDP-only features (pointer, keyboard, network capture, file upload) to content-script fallbacks, and a native-messaging MCP transport used in place of the WebSocket bridge. Font Awesome moves to assets/vendor, and the Chrome-only offscreen STT page is removed. New tests cover the adapter, native transport, manifest build, and source/version parity.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Refresh the generated date stamps in llms.txt, llms-full.txt, and sitemap.xml produced by build-crawler-files.mjs.
… roots - Give the content tab its own session key with a single writer so a page closing the workspace no longer overwrites the worker's tab id, and run tab/window listeners after hydration so an evicted worker doesn't write empty ids over the saved record. - Release the native read handle on every non-drained exit (throw, not-ok chunk, host that never marks the last chunk) instead of pinning it until the host TTL. - Store granted-root bookmarks in a file in the App Group container rather than a UserDefaults suite, whose per-process cache hid a fresh grant from a running extension until Safari quit.
Carry the Safari port onto main's hardened bridge. Main now classifies upgrades by Origin: only chrome-extension:// origins can be the extension, and an Origin-less socket must say relay:hello or be closed. That rejected both Safari transports, so: - bridge.ts accepts safari-web-extension:// origins alongside chrome-extension://, keeping main's host, pairing and pinning checks. - The native transport sends the extension origin in its open frame and the container app dials :7225 with it as the Origin header, instead of relying on the removed "no Origin means allowed" branch. - The bridge client keeps main's pairing subprotocols on the direct socket; the native socket connects unpaired. Other conflicts: - sidepanel.js: main's tab-surface sync and ownership refresh now resolve the target tab through getTargetTabs so Safari's workspace window is never mistaken for the content tab. - Safari build: nativeMessaging is in the Chrome manifest now, so it is classified as kept; capture_screenshot is declared as having no DOM fallback; the importScripts pins move to main's 333/329. - Rolled main's sidepanel and onboarding hash pins for the Safari edits and kept the native-free bridge-client check by rewording a comment.
…old-cache tabs
upload_file on Safari sent domSetFileInput straight to the tab, so a tab
whose content script was not loaded yet (freshly opened or reloaded) failed
with "Receiving end does not exist". It now runs ensureContentScriptInjected
first and targets frameId 0. It deliberately does not use
sendMessageWithRetry: a retry after a lost reply would set the file twice
and fire change twice, a double upload on auto-upload inputs.
resolveTargetTab fell back to an unscoped tabs.query({active:true}) when no
content tab was cached, which returns every window's active tab in no
particular order. It now asks windows.getLastFocused({windowTypes:['normal']})
first (the workspace is a popup window, so it is excluded), and keeps the old
query as the fallback when that API is missing, rejects, or returns the
workspace anyway.
… read uploads eagerly BridgeCoordinator: portDisconnected() armed the linger one queue turn after the watchdog's silence check, so an open or poll landing in between was ignored and the linger then closed a live socket without a closed frame, leaving the extension connected to nothing. It now arms synchronously on the queue. Silence is also counted from when a held poll is answered rather than when it arrived; with a 5s poll and a 5s silence window, every idle cycle used to arm a spurious linger. Worst-case close is now the ~13s the portSilenceSeconds comment already described. wheelScrollAt: _fsbScrollableAncestor walked parentElement, which stops at a shadow root, so wheel events on shadow content scrolled the document instead of the host or a light-DOM scroller above it, and slotted content skipped the component's own scroll box. The walk now follows the composed tree (assignedSlot, then parentElement, then shadow root host). FileReadService: read files with Data(contentsOf:) instead of .mappedIfSafe so a read handle holds its own copy and never depends on the file staying put after the security-scoped access is released.
… target GrantedRoots.rootContaining resolved the requested path before any granted root's security-scoped access had started. The sandbox hides a link inside the grant until then, and resolvingSymlinksInPath does not fail on a link it cannot read -- it returns the link's own path, which is contained. The read then resolved the path again with access live and followed the link without re-checking, so ~/Downloads/link -> ~/.ssh/id_rsa passed containment. The target is now resolved per root after its access starts, and beginRead reads exactly the url the check approved. That alone does not cover a broad grant such as ~, where the link's target really is inside the grant and the sensitive-path denylist only ever saw the link's harmless name. FileReadService now reports the resolved path, and the native reader re-runs executeUploadFile's denylist on it before fetching any chunk, releasing the handle on refusal. A host that reports no path, or a screen that throws, fails closed. The resolved path stays out of results, logs and audit records, and the managed-screenshot exception carries through to the second check.
…failure NativeFraming only checked its reassembly timeout inside ingest(), for the same message, so a lost tail chunk -- which never sends another chunk -- left the buffer parked indefinitely. The coordinator's 1s watchdog now calls expireStalled() ahead of its parked-poll guard, since a stalled reassembly looks like a perfectly healthy port. The timeout is now 5s since the last chunk rather than 30s since the first. The extension posts every chunk of a message in one loop, so a 5s gap means loss, and a 30s budget started at the first chunk could never fire before the server's own 30s sendAndWait, whose clock started before the tool ran. A failed reassembly also closes the MCP socket now. Delivering only an error frame let the extension's reconnect adopt the still-open socket, so the hub never saw a close and the request still sat out its 30s timeout instead of arming sw_evicted recovery. handleOpen clears leftover buffers before dialing, because a server-side close nils the socket without discardSocket() and the new socket's watchdog would otherwise expire the old connection's buffers.
Keep Safari's native upload and shorter keepalive, and route Chrome upload through the shared debugger attach. Click-and-hold stays on the long content-script timeout, and the Safari version tracks 0.9.92. Co-authored-by: Cursor <cursoragent@cursor.com>
On Safari the remote-control start handler reported 'ready', then every click, keystroke and scroll was refused before dispatch and only logged locally, so the dashboard showed an active session that did nothing. Each input now runs the content-script verb the AI tools already use there: pointerClickAt, domInsertTextAt, wheelScrollAt, and a new domKeyPress for editing keys. They are untrusted events, so sites that check isTrusted still ignore them. A tab without the content script gets it injected and the verb retried; a miss on the page (no element at the point, nothing focused) is logged and keeps the session; an unreachable tab ends it with retarget-required. Chrome keeps the CDP path. pointerClickAt now moves focus between mousedown and mouseup the way a real click does, unless the page cancels the mousedown, so typing after a remote click reaches the field. domKeyPress performs Backspace, Delete, Enter and Tab when the page leaves the keydown alone; arrow keys are not emulated.
…content tabs Safari has no side panel, so the FSB workspace is its own window with a safari-web-extension:// page. None of the restricted-URL checks knew that scheme, so the dashboard relay could stream it, start a task in it, or navigate it when that window was the last one focused. The stream filter in ws-client, isRestrictedURL / getPageTypeDescription in the background, and their copies in the MCP dispatcher now include it. On Safari the relay also asks FsbPlatform.resolveTargetTab() for the last real content tab before its own active-tab queries, for the stream candidate, remote navigate and history, dashboard task start, and DOM forwarding. Streaming still requires a streamable page; navigation and task start accept any tab but the dashboard, since they may begin from a blank one. On Chrome the helper returns null and the existing queries run as before.
…estarts The relay only connected from onInstalled, onStartup, the Sync-tab storage listener and the reconnect message. onStartup and onInstalled do not fire again after an idle eviction, and the backoff timer dies with the worker, so a relay that dropped while the worker went idle stayed down until the browser restarted, even when something else woke the worker. background.js now arms the relay on every worker start, right after ws-client loads; only an explicit serverSyncEnabled:false opts out, as in the storage listener. connect() already keeps a socket that is open or opening to the same room, so the overlap with the startup listeners is harmless. _scheduleReconnect also sets a backstop alarm that wakes the worker each minute until a socket opens; onopen and disconnect() clear it, and onAlarm re-arms the relay. This mirrors the MCP bridge's reconnect alarm and applies to Chrome and Safari alike.
resolveTargetTab() and the tabs.onActivated tracker accepted any tab whose URL was not an extension page. If Safari withholds a tab's URL, the workspace's own tab reads as a content page: activating it made it the "last content tab", and the dashboard relay, which now asks resolveTargetTab() first on Safari, could navigate or drive FSB's own window. A content tab now also has to sit outside the workspace window, the check windows.onFocusChanged already made.
The MCP section said the native path needed no server change because URLSessionWebSocketTask sends no Origin. Both halves are stale: the server now treats an Origin-less socket as a relay, so the host sends the extension's Origin, and published fsb-mcp-server (0.11.1) still refuses safari-web-extension:// on both transports. It also now says plainly that the server serves one browser at a time, and that Safari cannot pair yet: no native host can hand it the code, and its extension UUID changes on every launch. Known gaps: network_capture is unavailable rather than "degraded to metadata" (there is no metadata path and no webRequest permission), and dashboard remote control, telemetry and pairing get their own entries. The must-verify table gains relay recovery after eviction (V9) and workspace tab targeting (V10), and V5 now asks the question that matters now that the host sets Origin itself.
A stalled host-to-extension chunked message waited 30s, the same budget as
the server's sendAndWait, and the timer was never refreshed as later chunks
arrived. When it fired, _fail() dropped the native port without posting
{t:'close'}. Safari gives the host no teardown signal, so the bridge's
reconnect adopted the still-open socket, the hub never saw a disconnect,
and sendAndWait rejected with a timeout that does not arm sw_evicted
recovery.
The deadline is now 5s measured from the last accepted chunk, matching
NativeFraming.reassemblyTimeout on the host, and _fail() tells a still
attached port to close :7225 before tearing it down. The close goes through
a helper that swallows post errors: routing it through _post() would
re-enter _fail() while CLOSING, and in close() a throwing post fired
onclose twice. Teardown also clears pending reassembly timers.
If attributesOfItem failed, the size defaulted to 0 and passed the 32MB check, so Data(contentsOf:) read the file with no limit. The loaded bytes were never re-checked either, so a file that grew past the cap between the stat and the read was kept and sent to the service worker. The stat is now only an early refusal when a size is known. The read goes through readBounded(), which stops one byte past the cap, and the loaded length is checked against maxFileBytes before a handle is issued.
…onSuspend Safari 26.5 runs the extension's service worker on its WebContent process's main thread. A worker `new WebSocket()` hands its setup to the main thread and blocks on a semaphore, so the worker waits on itself and the process deadlocks. The MCP bridge's first ws:// probe froze the worker partway through its importScripts chain, every extension resource load and page in that process hung with it, and Safari dropped the background after about two minutes (WKWebExtensionContextErrorDomain Code=6). On device that showed up as no toolbar icon, a toolbar click that did nothing and a settings page that never loaded. Behind it, background.js registers on chrome.runtime.onSuspend at top level. Safari's worker has no onSuspend, so evaluation threw a TypeError. platform-adapter install() now, on Safari only: - in the service worker, replaces WebSocket with a constructor that throws SecurityError, which the bridge already treats as "pin transport=native"; - gives runtime.onSuspend a no-op stand-in when it is missing. safari/README.md records the deadlock, the on-device V1-V10 results and the gaps found while testing: capture_screenshot (CDP only), the dashboard relay, the press_key foreground guard, and a corrected get_memory_stats note. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…gates to A visual_reason/client tick broadcasts the overlay once, at tick time. open_tab and navigate record their tick before the destination page has a content script, so that broadcast was lost and the overlay stayed dark until the next action. Playing a YouTube video through open_tab + navigate never showed it. replayMcpVisualSessionForTab only replays explicit start_visual_session sessions. The lifecycle module now exports replayVisualSessionLifecycleForTab, which re-sends a live per-tab entry without extending its 60 s window, clears an elapsed one and leaves is_final alone. background.js calls it when the tab finishes loading and when the content script reports ready; an explicit session on the same tab still wins. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… square
Safari treats a grayscale toolbar icon as a template: it keeps only the alpha
channel and tints it, gray or the accent color while the extension is active.
icon*.png are opaque grayscale tiles, so the toolbar showed a solid square.
- extension/assets/safari-toolbar-{16,32,48,64}.png: the FSB lettering as black
plus alpha on transparent (hand-drawn at 16 and 32, the sizes the toolbar
shows), from scripts/generate-safari-toolbar-icons.py.
- The Safari manifest's action.default_icon points at them; icons keeps the tile.
- utils/action-icon.js: on a platform with caps.templateToolbarIcon (new in the
adapter) every frame is drawn as black-plus-alpha with no plate, so the orbit,
sweep, capability ring and breathe still read once Safari tints them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… app requireForegroundNativeInput guards CDP input, which needs the tab in front. Safari has no CDP: every caller's attach fails and it falls back to untrusted DOM events, which need no focus. The guard now lets a platform without trusted input through, so the fallback runs instead of TAB_NOT_FOREGROUND. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every capture went through chrome.debugger, so Safari always returned SCREENSHOT_CAPTURE_FAILED. Without CDP, screenshot-capture.js now captures the visible viewport of the active tab with tabs.captureVisibleTab and crops region and element captures out of it at device pixels, hiding FSB's overlays the same way. full_page, device emulation, off-screen targets and background tabs get typed errors the MCP server already maps. The adapter declares cdpCaptureScreenshot in a new CDP_PLATFORM_HANDLED list, so tool-executor.js sends it to the handler instead of failing it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Under require-trusted-types-for 'script' (YouTube, among others) new Function(code) is refused: "Refused to evaluate a string as JavaScript because this document requires a 'Trusted Type' assignment." The page function now passes the code through a Trusted Types policy it creates once per page, falls back to eval if the engine only accepts TrustedScript there, and explains the refusal on a page whose directive forbids new policies. Verified in WebKit with a YouTube-style header and with trusted-types 'none'. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The tool takes a ref or a CSS selector, and schema-bridge.ts passes either through as sourceRef/targetRef, but dragdrop only looked them up through FSB.getElementByRef (which does not exist) and [data-fsb-ref] (which nothing sets), so a selector was always "Source element not found". It now resolves refs through FSB.resolveRef and selectors through FSB.querySelectorWithShadow, like every other tool, and contains the lookup that threw on quoted selectors. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Safari's extension service worker deadlocks on new WebSocket(), so the
adapter makes it throw there and ws/ws-client.js never reached the
dashboard relay: no DOM streaming, no remote control. The relay now takes
the same route as the MCP bridge, over native messaging, on its own channel.
- ws/mcp-native-transport.js: FsbNativeBridgeSocket takes a channel ('mcp'
by default, or 'relay') and names it on every message.
- ws/ws-client.js: createFSBRelaySocket uses the native socket where the
worker WebSocket is guarded, and the real WebSocket everywhere else.
- BridgeCoordinator.swift: one coordinator per channel, each with its own
socket, poll, linger and watchdog; unknown channels are refused.
SafariWebExtensionHandler.swift routes by `ch`, defaulting to mcp.
The relay server authenticates by key and role only, so the app's socket
needs no server change. tests/safari-native-bridge-channels.test.js builds
the shipped Shared/*.swift into a command-line harness and drives both
channels against two local WebSocket servers (macOS only; skips elsewhere).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Records why the toolbar uses template icons, how the dashboard relay shares the native pipe on its own channel, what capture_screenshot can and cannot capture without CDP, and drops the press_key foreground gap that is fixed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Bring in main's Online-Mind2Web benchmark lead for the showcase and README, MCP 0.11.2 with the FSB server icon and automatic npm releases, glama.json, the showcase design refresh, and the portfolio frame-ancestors fix. Both conflicts were the root test chain, where each side had appended gates after no-orphan-descriptor: main's benchmark-assets-parity and this branch's execute-js-trusted-types and drag-drop-selector-resolution. Keep all three, main's first, in package.json and in the refinement list that delegation-phase-contract pins. Nothing else overlapped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Safari tints any toolbar icon it detects as grayscale, keeping only its
alpha. The letters-only template this branch shipped therefore showed as
small accent-blue lettering instead of FSB's icon. An icon with some real
color is shown as it is, so the Safari toolbar now gets the original tile
with a faint warm tint on every pixel (warm black plate, warm white
lettering): visually the same tile, but no pixel is gray.
- extension/assets/safari-toolbar-{16,19,32,38,48,64}.png from
scripts/generate-safari-toolbar-icons.py; 19/38 added because recent
macOS Safari asks for them.
- utils/action-icon.js: on caps.tintsGrayscaleToolbarIcon (renamed from
templateToolbarIcon) the frames draw from the tinted 16/32 with the normal
plate and colours, and dimming skips its grayscale filter. The template
drawing mode is gone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Safari decides whether to tint a toolbar icon with -[NSImage safari_isGrayscale] (SafariShared). Calling it on test images gives the rule: a pixel is colored when its premultiplied channel spread is at least 25/255, and the icon keeps its own colors only when at least 3% of its visible pixels are colored. The faintly warm tile from 541e7ac failed it, because only 1 to 2% of its pixels qualified, so Safari still drew a tinted square. The Safari toolbar now shows the same artwork in FSB orange with black lettering. action-icon.js repaints the orange plate under the breathe and draws the orbit, sweep and glow white so they stay visible on orange. It still cuts the glow from the black-and-white tile and dims without the grayscale filter. Chrome is unchanged. Safari's own classifier calls all six assets and every rendered toolbar frame (288 across all states, at 16 and 32) colored. tests/safari-toolbar-icons.test.js runs that classifier on macOS. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Safari Web Extension port: a new
safari/FSBXcode project (Swift app, extension handler, native MCP socket bridge, granted-roots file reader) plusbuild-safari,release-safari, andsync-safari-versionscripts, with esbuild andvalidate-extensionupdated to build and check the Safari bundle. The extension gets aplatform-adapterthat degrades CDP-only features (pointer, keyboard, network capture, file upload) to content-script fallbacks, and a native-messaging MCP transport used instead of the WebSocket bridge in Safari. Font Awesome moves toextension/assets/vendor(woff2 only), and the Chrome-only offscreen STT page is removed. New tests cover the adapter, native transport, CDP-degradation routing, pointer fallbacks, Safari manifest build, and source/version parity; the plan is in.planning/SAFARI-PORT-PLAN.md.