Skip to content

Sync bobleesj main: WebGPU display, Show2D/Show3D streaming, Find drift, ScanControl; merges upstream DiffractionSim - #251

Merged
bobleesj merged 174 commits into
electronmicroscopy:mainfrom
bobleesj:main
Nov 12, 2026
Merged

bobleesj merged 174 commits into
electronmicroscopy:mainfrom
bobleesj:main

Conversation

@bobleesj

Copy link
Copy Markdown
Collaborator

Brings bobleesj/quantem.widget main into this repository's main. It includes current upstream main (DiffractionSim, quantem.diffsim, the polyhedra fix), merged in bed91fbf, so this merges without conflicts. CHANGELOG.md under Unreleased lists every change. The main ones:

Display: WebGPU at display rate

  • Show1D, Show2D, Show3D and Show3DSlices draw scrub, playback, zoom, pan, contrast and histogram drags on WebGPU in the frame the input arrives. Show3D slider moves answer in about 1 ms with every move drawn, also with the CPU slowed 4x. Before, it took about 25 ms and dropped up to half the moves.
  • Show2D and Show3D draw true-color (RGB) frames on WebGPU. The title badge names the path that drew the frame, CPU only on a page without WebGPU. Both paths give the same bytes.
  • Show2D galleries draw each panel straight into a WebGPU canvas, binned galleries and detail tiles included.

New

  • Show3D.stream() / append() and Show2D.stream() / update(): viewers that grow or change while the notebook stays interactive. They send only the new frames and keep native pixels.
  • Show3D thumbnail strip (thumbnails=True, mode="stream") and Find drift: pick frames in the strip, register them on WebGPU, chart the drift and its fitted velocity, and add a drift-corrected panel. viewer.drift holds the result.
  • quantem-widget show2d / show3d: write image files, a stack or a folder as one HTML page. Lossless by default.
  • ScanControl and ShowGPU widgets.

Renamed and removed (no aliases)

  • quantem.gpu's direct-ptychography names (upsample, phase_damp, method="phase_of_mean", trials, ...). Display names follow the vocabulary: columns=, mode=, --save-dir, and the console script quantem-widget. Saved records get a schema bump plus a rewrite script (scripts/rewrite_widget_state.py, scripts/rewrite_ssb_calibration.py).
  • Removed: ShowCIF, ShowDiffraction and ChooseLattice; the ase and scikit-image dependencies and the crystal extra; folder watching; and the Show1D trial review.

How the upstream merge was resolved

  • ShowCIF, ShowDiffraction and ChooseLattice stay removed, and upstream's abTEM edits to ShowCIF go with them. DiffractionSim uses none of them.
  • No ase dependency line: ASE arrives with quantem core, which DiffractionSim already requires. The dependency test treats ase as optional, since it is imported inside functions, and skips if TYPE_CHECKING: imports.
  • DiffractionSim takes voltage_kV (was energy_keV), the name the repository's vocabulary test requires. The value is the same, and the trait and saved key follow. The published quantem.diffsim 0.1.0 keeps its own energy_keV.
  • DiffractionSim's sliders use js/shared/Slider (no position easing), which Slider.test.ts requires of every widget.

Tests run on the merge commit (Apple M-series GPU; WebGPU adapter apple / metal-3, subgroup 32, plus the CPU path)

  • pytest with the browser tier: 943 passed, 35 skipped. 3 browser tests are intermittent, and they fail the same way on bobleesj main before this merge (same commands, same machine): the Show3D export pixel check test_export_pixels_match_with_and_without_webgpu[show3d, show3d_panels] (one channel value off by 255, about one run in two) and test_show3d_held_arrow_key_steps_one_frame_per_repeat[cpu]. The merge touches no Show3D code. They are tracked separately.
  • vitest 413 passed. tsc clean. npm run build builds every bundle, DiffractionSim included. The public-repository test, check_large_files.py and check_notebook_sizes.py pass.

bobleesj and others added 30 commits October 21, 2026 09:15
Show3D drew RGB stacks on the CPU canvas while its badge said WebGPU, and
Show2D packed color panels in a JavaScript loop on every page.

- Show3D: RGB frames are WebGPU-resident slots (uploadRgbData, 3 floats per
  pixel) drawn by a direct true-color pass (DIRECT_SLOT_RGB_SHADER) for scrub,
  playback, multi-panel grids with zoom/pan and the moving average. The status
  line reads "true-color display · WebGPU resident"; the JavaScript pack runs
  only without a WebGPU engine.
- Show2D: color panels pack through the colormap engine
  (renderRgbToImageBitmapAsync: WebGPU, or the Canvas2D engine without it) and
  commit together with the grayscale panels.
- The title badge takes the path that drew the frame (drawnBy), so it never
  says WebGPU for CPU pixels.
- Both paths quantize round(clamp(v, 0, 1) * 255), NaN to 0; the WebGPU
  passthrough truncated before. Colormap passes refuse rgb slots.

Browser tests: tests/browser/test_show3d_rgb.py (1 and 2 panels, export and
live notebook, readback == given bytes, 60 WebGPU presents/s and no CPU pack)
and tests/browser/test_show2d_rgb.py (color + gray and color + color, live).
quantem-widget show2d/show3d pages, Show3D thumbnail strip (mode="stream"), true color on WebGPU
- Each gallery panel gets a WebGPU canvas over its 2D canvas. Contrast
  drags, zoom, pan and the settled frame draw the panel's resident slot
  through one shader (renderSlotsDirectToCanvases: one submission, each
  panel with its own range, LUT and view), in the input's animation frame,
  instead of 48 colormap passes into pooled OffscreenCanvases, a queue
  wait, 48 ImageBitmaps and two drawImage per panel.
- The 2D canvas gets the frame once a panel has not changed for 150 ms:
  the current state is drawn again and copied in the same task, before the
  canvas texture is presented. Copy and SVG export flush that copy first.
- Rotated or flipped kernel-less exports, packed uint8 folder frames,
  binned previews with detail tiles, true color and panels whose slot lags
  the data keep the 2D path, which hides the GPU canvas.
- A histogram drag writes auto_contrast and contrast_preset once on
  release; the first move used to write them and re-render the widget.
A display_bin gallery (what Show2D suggests for 48 x 4096^2, over the
browser's message limit) kept the old bitmap path, and once zoomed in
far enough to stream full-resolution detail tiles every visible panel
did: a histogram drag drew about half the moves at ~70 ms. Binned
previews now draw on the panel's WebGPU canvas, and a detail tile is
uploaded to its own slot and drawn over the preview in the same pass,
clipped to the tile, with the same range, LUT and view.
…on WebGPU to 1/100 px, chart the drift and its fitted velocity, sync the result to Python as viewer.drift, and add a drift-corrected panel
…4K stream

- Show3D linked-contrast colorbar showed 0 to 1; it reads the stack range
- thumbnail strip: no seam between cells (cover a fractional 1fr column), no black empty slots, More > Thumbnails brings a closed strip back
- frame counter keeps the live total while frames stream in and the view is parked
- binned HTML export names the original stack and the mean bin in its header
- an empty Show3D.stream shows only its waiting panel until the first frame
- io.download(verbose=False) hides the Hugging Face progress bars
- Show2D scale, denoise, bin, sigma, ROI and FFT controls carry accessible names
- bench_interactivity takes the window position from QUANTEM_WIDGET_WINDOW_POSITION
show2d: gallery panels draw straight into WebGPU canvases
Show2D/Show3D: fixes from a full drive on 2K/4K gold data and a live 4K stream
…vice and on the CPU path recovers a known subpixel drift, sends viewer.drift with the picked frames and nm/s to Python, and adds the drift-corrected panel
…k there is a right click) and passes on the Apple GPU; browser tests say to run on a Mac first, headless
A headed run against a bundle built before a pull or merge loads and draws through the
frontend code it was built from, so it tests old frontend code against new tests.

The chrome fixture now fails every browser test at once, naming what is stale, when a widget
bundle is older than the js/ sources or quantem.widget is imported from outside this checkout.
QUANTEM_WIDGET_ALLOW_STALE_BUNDLE=1 skips the check for an installed-wheel run.
…the stale-bundle check

The browser tier now refuses a quantem.widget imported from outside the checkout. The CUDA and MPS
release gate runs tests/browser/test_showptycho_release.py against the installed candidate wheel on
purpose, so that step sets QUANTEM_WIDGET_ALLOW_STALE_BUNDLE=1.
Browser tests refuse a stale frontend bundle
…heir own times (times_s, velocity_px_per_s), the chart plots against time and the drift-corrected panel places unpicked frames by time; tested in a notebook on an Apple GPU and the CPU path
The JupyterLab kernel starts in the notebook root, so a relative
PYTHONPATH=src resolved to nothing there and the kernel imported another
quantem.widget (an editable install with bundles older than true color on
WebGPU). The stale-bundle guard checked only the pytest process, so the
Show2D and Show3D color tests failed against old frontend code. The
fixture now makes PYTHONPATH absolute for the kernel and fails when the
kernel would import quantem.widget from outside this checkout.
…ns the histogram

A per-panel histogram binned on WebGPU left panelHistogramData empty, so
the Auto snap returned early and the clip handles stayed at 0-100 % while
the image was drawn with the percentile clip. Whether a page got the snap
depended on which histogram path ran first, so the handle shading differed
between the WebGPU and CPU paths now and then (show3d_panels parity
[0, 0, 0, 13, 13]). The handles now take the range the image is drawn
with: the stack-wide Auto range when it applies, else the panel's
percentile clip of the frame shown.
Browser tier: kernel imports this checkout; Show3D per-panel handles show the Auto range on WebGPU
Brings in DiffractionSim, quantem.diffsim and the polyhedra fix from
electronmicroscopy/quantem.widget main.

Resolutions:
- ShowCIF, ShowDiffraction and ChooseLattice stay removed (this branch
  deleted them); upstream's abTEM edits to ShowCIF go with them.
  DiffractionSim does not use them.
- No `crystal` extra and no `ase` dependency line: ASE arrives with
  quantem core, which DiffractionSim already needs. The dependency test
  lists ase as optional (imported inside functions) and skips
  `if TYPE_CHECKING:` imports.
- DiffractionSim takes `voltage_kV` (was `energy_keV`), the vocabulary's
  name; same number, the trait and saved key follow. quantem.diffsim
  0.1.0 on PyPI keeps its own `energy_keV`.
- DiffractionSim sliders come from js/shared/Slider (no position easing).
- Tests: the hub progress-bar test and the Show4DSTEM stream test no
  longer depend on test order (an earlier test set
  HF_HUB_DISABLE_PROGRESS_BARS or left a Playwright event loop running).
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
  • Delete bobleesj/quantem.widget:main from its remote if you own it, it is not long-lived, and no other PR uses it.
  • Remove the dedicated worktree and local main branch after confirming the worktree is clean.
  • Fetch electronmicroscopy/quantem.widget and update local main before starting new work.

Show3D keeps its WebGPU presentation canvas at opacity 0 once an idle frame
is copied to the 2D canvas. Whether Chrome still holds pixels in that hidden
buffer varies run to run, so the export parity compared a hidden layer and
failed with [0, 255, 0, ...] about one run in two while the screen matched.
A headed Playwright window on a Mac becomes the key window when it opens, so
typing meant for another app reaches the test page. A space started Show3D
playback during the held-arrow test (31/60 at key-up, 46/60 a second later),
and '. ' plus Return sent another test page to a web search. Headless Chrome
on a Mac still uses the Apple GPU through Metal. QUANTEM_WIDGET_HEADLESS=0
opens windows; Linux stays headed (headless there is the CPU renderer).
A CUDA ShowPtycho export reads source/bf_columns.qem by byte range. The
serve fixture used SimpleHTTPRequestHandler, which ignores Range, so
test_tilt_auto_save_and_exact_export_reopen failed on CUDA while the
export's own launcher (.viewer/serve_range.py) opens it fine. An MPS
export loads its counts whole, which is why the Mac passed.
Show4DSTEM.compute_ssb() is direct_phase() and its ssb_compute_* traits are
direct_*; ShowPtycho.ssb_session and ssb_parameters are ptycho_session and
ptycho_parameters; adapters.gpu.is_ssb is is_direct_ptycho. UI text, docs,
env vars (QUANTEM_DIRECT_RELEASE*), the rewrite script, the release-check page
and the workflow tutorial follow. Widget-owned records move one version on
(direct-stars/6, direct-snapshots/3, direct-browser-folder/4,
show4dstem-direct-calibration/5); scripts/rewrite_direct_calibration.py carries
them over, plus calibration/8 and browser-calibration/3 to their direct ids.
Names shared with quantem.gpu stay for the joint merge.
…tem.direct-calibration/9

quantem.gpu renamed its SSB names (WebGPUDirectBackend, the Direct*
folder helpers, direct_auto_upsample, direct_upsample_requirement,
benchmark-direct). ShowPtycho's calibration.json is
quantem.direct-calibration/9 and the exported browser calibration
quantem.direct-browser-calibration/4; an earlier quantem.ssb-calibration
file is pointed at scripts/rewrite_direct_calibration.py.
@bobleesj
bobleesj merged commit d983c24 into electronmicroscopy:main Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant