Skip to content

feat(sdk): adaptive quality and rate for topic subscriptions - #5053

Open
paul-nechifor wants to merge 8 commits into
paul/feat/topic-video-sdkfrom
paul/feat/topic-adaptation
Open

paul-nechifor wants to merge 8 commits into
paul/feat/topic-video-sdkfrom
paul/feat/topic-adaptation

Conversation

@paul-nechifor

@paul-nechifor paul-nechifor commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Why do we need this?

On slow connections, the browser SDK keeps requesting full camera and point cloud streams. This leads to stale or missing frames.

What is this feature?

Added automatic adjustments that lower quality or frame rate when the connection is congested and restore the requested settings when it recovers. Lower priority streams are reduced first, and a single video viewer can change quality without reconnecting. Some slow-link benchmark targets still fail.

How to manually run this

Which issue(s) does this PR close?

Fixes

Checklist

  • I have read and approved the CLA.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 62177a6 Previous: 7312553 Ratio
peak memory 1719.859 MB 1744.168 MB 0.99
peak threads 370 threads 371 threads 1.00
network (transport) 2728.311 MB 2733.714 MB 1.00
disk write 1.891 MB 1.625 MB 1.16
instructions 138.787 G 137.257 G 1.01

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.34694% with 15 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/e2e_tests/test_topic_video_browser.py 0.00% 12 Missing ⚠️
dimos/web/relay_bridge/topic_video.py 93.75% 1 Missing and 1 partial ⚠️
dimos/web/relay_bridge/test_topic_relay_e2e.py 98.11% 0 Missing and 1 partial ⚠️
@@                     Coverage Diff                     @@
##           paul/feat/topic-video-sdk    #5053    +/-   ##
===========================================================
  Coverage                      83.92%   83.93%            
===========================================================
  Files                           1870     1870            
  Lines                         183842   184030   +188     
  Branches                       15147    15160    +13     
===========================================================
+ Hits                          154297   154471   +174     
- Misses                         25620    25630    +10     
- Partials                        3925     3929     +4     
Components Coverage Δ
Tests 96.57% <91.39%> (-0.01%) ⬇️
Flag Coverage Δ
OS-macos-latest 78.00% <67.85%> (-0.01%) ⬇️
OS-ubuntu-24.04-arm 77.77% <67.85%> (-0.01%) ⬇️
OS-ubuntu-latest 78.13% <67.85%> (-0.01%) ⬇️
Py-3.10 78.13% <67.85%> (-0.01%) ⬇️
Py-3.11 78.11% <67.85%> (-0.01%) ⬇️
Py-3.12 78.12% <67.85%> (+<0.01%) ⬆️
SelfHosted-Linux 40.92% <53.57%> (+0.01%) ⬆️
SelfHosted-macOS 40.65% <53.57%> (+0.22%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/web/relay_bridge/protocol.py 99.26% <100.00%> (-0.37%) ⬇️
dimos/web/relay_bridge/test_topic_media.py 100.00% <100.00%> (ø)
dimos/web/relay_bridge/test_topic_variants.py 99.55% <100.00%> (+0.03%) ⬆️
dimos/web/relay_bridge/test_topic_video.py 99.74% <100.00%> (+0.04%) ⬆️
dimos/web/relay_bridge/topic_profiles.py 88.76% <100.00%> (+4.05%) ⬆️
dimos/web/relay_bridge/topic_protocol.py 100.00% <100.00%> (ø)
dimos/web/relay_bridge/topic_service.py 83.75% <100.00%> (+0.56%) ⬆️
dimos/web/relay_bridge/topic_test_support.py 98.76% <ø> (ø)
dimos/web/relay_bridge/test_topic_relay_e2e.py 97.62% <98.11%> (+0.15%) ⬆️
dimos/web/relay_bridge/topic_video.py 90.83% <93.75%> (+0.24%) ⬆️
... and 1 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@paul-nechifor
paul-nechifor added this pull request to stack #4992 November 5, 2026 07:56
@paul-nechifor
paul-nechifor marked this pull request as ready for review November 5, 2026 07:58
@paul-nechifor
paul-nechifor force-pushed the paul/feat/topic-adaptation branch from e73e49a to 035d7af Compare November 5, 2026 07:59
@greptile-apps

greptile-apps Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 2/5

[Medium impact] The changes reach browser topic subscriptions, relay delivery feedback, and direct-video providers.

Fix the video freeze race, lost receiver callback, and bitrate increases before merging.

Findings

  1. P1 Quality changes can freeze video ▶
  2. P1 Updates stop video measurements ▶
  3. P1 Congestion can raise video bitrate ▶
  4. P2 Caps hide invalid page settings ▶

Summary

Adds automatic topic quality and rate changes, delivery statistics, congestion feedback in pongs, and in-place video reconfiguration.

  • The SDK lowers topic detail or rate when delivery slows.
  • Topic viewers receive a count of congested latest-stream sends.
  • A sole video viewer can change quality without replacing its track.
  • Sessions report the lowest recent ping round-trip time.

Reviews (1) · Last reviewed commit: "test(cockpit): implement RTT on the fake..." · Reviewed by Greptile

Comment thread dimos/web/relay_bridge/topic_video.py Resolved
Comment thread web/sdk/src/topics.ts Resolved
Comment thread web/sdk/src/topicAdapt.ts Resolved
Comment thread web/sdk/src/topics.ts Resolved
- provider: an encode that already applied a reconfigure keeps its IDR;
  only one encoded before the settings reached the worker is dropped
- sdk: the video receiver callback lives on the handle, so a group change
  (priority, keep, source) keeps polling receiver statistics
- sdk: a resolution rung never raises the bitrate above the rungs before it
- sdk: a page update is checked against the ranges before the cap applies
@paul-nechifor
paul-nechifor force-pushed the paul/feat/topic-adaptation branch from 635e74e to 62177a6 Compare November 11, 2026 09:17
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Nov 11, 2026

This branch was successfully deployed

No deployments
cachix — 7e8ab2f9 Deployed Nov 10, 2026 by paul-nechifor via cachix-build-macos #16891
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants