Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,13 @@ For 60 fps, select WebRTC video passthrough under Settings → Display & streami

The WebRTC interruption tolerance setting offers 4, 10 (default), or 30 seconds and applies on the next browser connection. Recoverable interruptions keep the last video frame visible while waiting for recovery; persistent problems fall back to JPEG after the grace period. Unsupported codecs and terminal connection failures still fall back immediately. This does not enlarge the video buffer or change control and audio connection rules. Refresh the webpage after updating the App. See the [stability and troubleshooting notes](docs/WEBRTC_RECOVERY.md) (Chinese).

If the video element renders incorrectly in your car's browser, enable “Use Canvas to display video (experimental)” on the webpage's connection screen before connecting. This is disabled by default. The selection is saved in the current browser and persists across automatic reconnects and page reloads; if browser storage is disabled, it applies only to the current page. WebRTC video is still decoded by an offscreen `<video>` element and drawn directly onto a visible Canvas with `drawImage()`. Video frame callbacks are preferred, with animation frames used for older browsers or when offscreen callbacks stall. Touch coordinates are mapped to the video's aspect ratio, and drawing failures trigger a fallback to JPEG. This mode may increase GPU / CPU usage. The video frame rate in the status bar still reflects the browser's video presentation statistics, not the actual Canvas drawing rate.
The connection page offers three **Video display mode** radio cards. **Native Video** remains the default; both Canvas options are experimental:

- **Native Video** displays the WebRTC stream in a video element.
- **Compatibility Canvas** decodes with an offscreen `<video>` and copies frames with `drawImage()`. Video frame callbacks are preferred, with animation frames recovering older browsers or stalled offscreen callbacks.
- **Direct Canvas** reads WebRTC `VideoFrame` objects through the main-thread `MediaStreamTrackProcessor` and draws them directly. It never creates a `<video>` or uses `VideoDecoder`. Worker-only Processor implementations are outside this first version. Missing support or runtime failure falls back directly to JPEG / WebP without switching to a video-element mode.

The browser remembers the choice across reloads and automatic reconnects; unavailable storage retains it only in the current page. Existing Canvas preferences migrate to Compatibility Canvas. Disconnect before choosing another mode; automatic retries retain the launch choice. Both Canvas modes wait for a successfully drawn frame before stopping the image stream, and touch mapping follows actual frame dimensions. Canvas can increase GPU / CPU usage and does not guarantee 60 fps. Direct Canvas reports successful draw submissions in the status bar; the other modes retain video presentation / decoding statistics. Neither measures physical display refresh. `window.wheelplayStats.presentation` exposes selected / active modes, drawing statistics and fallback reasons.

Canvas mode requires testing on car browsers such as NIO's. Changing the rendering path may help if an overlay covers only the native video element; it cannot guarantee success if the system stops decoding or overlays the entire page. Test while the vehicle is parked. Drivers should use only essential navigation interfaces while driving.

Expand Down
8 changes: 7 additions & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,13 @@ WheelPlay 是一款 Android 局域网 CarPlay 服务端应用,可将 iPhone

「WebRTC 卡顿容忍时间」可选 4 秒、10 秒(默认)或 30 秒,下次浏览器连接生效。短暂断流时保留最后一帧并等待恢复,持续异常超过阈值才回退 JPEG;不支持的编码或彻底失败的连接仍立即回退。该设置不会扩大视频缓冲,控制与音频保持各自的连接规则。更新 App 后请刷新网页。详见 [稳定性分析与排查](docs/WEBRTC_RECOVERY.md)。

车机浏览器的视频元素显示异常时,可在网页连接界面勾选「使用 Canvas 显示视频(实验性)」,再连接设备。默认关闭;选择保存在当前浏览器,自动重连及刷新后保留(浏览器禁用存储时仅当前页面有效)。WebRTC 视频仍由 `<video>` 解码,元素放在屏幕外,使用 `drawImage()` 直接绘制到可见 Canvas;优先使用视频帧回调,旧浏览器或离屏回调停顿时使用动画帧刷新。触控区域按画面比例映射,绘制失败自动回退 JPEG。该模式可能增加 GPU / CPU 消耗;状态栏中的视频帧率仍为浏览器的视频呈现统计,不能代表 Canvas 的实际绘制帧率。
网页连接界面的「视频显示方式」提供三张单选卡片,默认「原生 Video」。两个 Canvas 模式均为实验性:

- **原生 Video**:由视频元素直接显示 WebRTC 画面。
- **兼容 Canvas**:保留离屏 `<video>` 解码并使用 `drawImage()` 绘制,优先视频帧回调,旧浏览器或离屏回调停顿时使用动画帧补偿。
- **直接 Canvas**:使用主线程 `MediaStreamTrackProcessor` 读取 WebRTC `VideoFrame` 并绘制,全程不创建 `<video>`,也不使用 `VideoDecoder`。仅在 Worker 中提供 Processor 的浏览器不在首版支持范围;不支持或运行失败时直接使用 JPEG / WebP,不切换到旧的视频元素模式。

浏览器保存所选方式,刷新和自动重连后保留;禁用存储时仅当前页面有效。原有 Canvas 偏好自动迁移为「兼容 Canvas」。需要断开后重新选择,自动重连不会改变本次启动的方式。两种 Canvas 都在首帧绘制成功后才停止图像串流;尺寸与触控按实际画面映射。Canvas 可能增加 GPU / CPU 消耗,不保证达到 60 fps。直接 Canvas 状态栏的 fps 是成功绘制次数,其他模式保留原有视频呈现 / 解码统计,均不等于屏幕实际刷新率。`window.wheelplayStats.presentation` 可查看所选 / 实际方式、绘制统计和回退原因。

Canvas 模式需要蔚来等车机实测:若遮罩只覆盖原生视频元素,改变显示路径可能有效;若系统停止解码或在整个页面上叠加遮罩,则无法保证有效。请在车辆停稳后验证,驾驶员行驶中只应使用必要的导航界面。

Expand Down
Loading
Loading