Skip to content

Bound virtual rows across Web, MyGo, SwiftUI and Kotlin - #274

Merged
tiensonqin merged 13 commits into
mainfrom
fix/mygo-virtual-list-memory
Oct 25, 2026
Merged

tiensonqin merged 13 commits into
mainfrom
fix/mygo-virtual-list-memory

Conversation

@tiensonqin

@tiensonqin tiensonqin commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Loading 100,000 Gallery rows previously created 100,871 retained LUI nodes before the host virtualized their views. Clearing them left a large native process footprint. This follow-up to #273 adds virtual_rows so Web, MyGo, SwiftUI and Kotlin request only the viewport, overscan and active interaction rows; application data and its key index remain complete.

virtual_list ~height:400 ~overscan:5 ~item_extent:32. [
  virtual_rows ~source:rows ~key:(fun row -> row.id) ~cmp:Int.compare
    ~scroll_key:(fun row -> Printf.sprintf "row-%d" row.id)
    ~mount:row_view ()
]

Overlapping windows retain identity. Evicted rows dispose their scopes and handlers; persistent editor values belong in controlled application data. Focus, pointer holds, composition and owned popup interactions pin the required rows. Reorders, failed constructors, rollback, hot reload and stale generations preserve ownership. Internal window feedback does not call the public visible-range handler.

Literal and keyed lists keep eager LUI ownership. Updated native clients probe the optional lui_ocaml_virtual_rows_version() export and negotiate host codes 258 (SwiftUI) or 260 (Kotlin), preserving their semantic platform and extension profiles. Old clients/libraries retain the complete eager fallback. Custom OCaml initializers pass the negotiated ~virtual_rows capability before mounting; custom native event switches forward the maintenance window event. Sparse metadata validates atomically with rollback on every supported host.

SwiftUI uses native scrolling with bounded stable row views, retaining the actual editor during far reorder and composition. Kotlin keeps native LazyColumn geometry with stable bounded row composition and native pin ownership. Empty or zero-height roots retain positive logical geometry so layout cannot progressively materialize the entire source.

The migration guide records constructor/runtime capability arguments, lifecycle, host negotiation, wire/event changes and canonical native property admission. The Gallery README gives build, usage and reproducible native test commands.

The focused MyGo fork commit exposes actual requested windows, separate interaction pins and authoritative key lookup. The current immutable union pin also preserves the input-capture, committed-geometry and live native-input snapshot APIs merged into main. The related MyGo PR #3 is merged. Its upstream module identity/imports/links remain unchanged. Embedding applications copy the immutable replacement documented in the README. No dependency source, caches, binaries or recordings are committed.

The earlier allocation improvements remain: single-buffer OCaml encoding, streaming MyGo decoding and transactional application, owned-string receiving, deletion journals without discarded property clones, and abandoned handle binding release. Wire bytes and exact integers remain preserved. There is no active GC policy, forced collection, heap-limit change or allocation trick.

Actual retained counts

Current real OCaml Components integration, including the new Gallery controls, a distant bottom jump, resync and two load/Clear cycles:

State LUI/backend nodes Host states Mounted row roots
Idle 874 839 0
100,000 rows loaded 894 859 20
Bottom, visible 99,985–99,999 894 859 20
Clear, both cycles 874 839 0

Verification

  • OCaml: 235 core tests, including 60 bounded-row/native-host/startup lifecycle cases, plus all 10 schema/tooling tests and PPX ownership checks.
  • Web: 26 sparse regressions per engine in Chromium, Firefox and WebKit; production build and 156 unit checks. Final CI reruns all three engines.
  • MyGo: complete race suite, production allocation budgets, canonical OCaml oracle, real Components/Todos processes and fresh-start qualification. New real-layout zero-height/empty-extension cases preserve compact positive-height rows.
  • Apple: 211 Swift Testing cases plus one XCTest; actual NSWindow pointer, menu, sheet, variable geometry, scroll tokens and ownership. The real OCaml 100k fixture retained 52 nodes/14 row roots/14 native editors; bottom had 46/12/12; Clear returned exactly to 10/0/0, twice.
  • Actual iOS simulator: adjusted-inset geometry, a native marked-text editor reordered from index 2 to 99,999 without losing identity/composition, true bottom/top scrolling and Clear. The ordinary iOS and Android Galleries each passed two complete cycles using public controls: Load 100k, Jump to last, native row99999 Press, Back to top, row0 Press and Clear. iOS backend/tests compile for ARM64 and x86_64.
  • Android: 69 complete store/Compose tests in one process, with native input-connection identity/composition, far reorder, pointer/selection/popup pins, reading anchors, geometry and target lifecycle. Each layoutlib composition owns and releases its public frame clock/recomposer. Actual Android accessibility scrolling advances the sparse viewport; compact native row glyphs remain readable. The ordinary Android app kept the same process throughout both complete cycles.
  • Production JNI qualified through a real JVM and OCaml: optional marker present/absent loading, bounded 100k windows, far pin/reorder, stale generations, malformed payloads, resync and Clear. The ARM64 JNI bridge also compiles with warnings as errors and loads against real current/legacy component libraries.
  • Native bridge GC stress roots 2,048 pins and preserves exact integers above 2^53. Existing ordinary event ordinals remain stable.
  • Native Gallery builders now compile the maintained OCaml sources; Android uses the matching target Unix/Threads runtime, SDK and correct JNI source path. Intrinsic ordinary lists keep their outer native scroll owner. The Apple C bridge closes its ARC-owned window safely. Todos, Markdown and Split Swift clients build from fresh scratch directories.

CI now runs full Android coverage and real JVM/JNI/OCaml, SwiftUI/OCaml and UIKit lifecycle gates. The Native OCaml renderer remains outside this task.

Release physical memory

Ordinary macOS release Gallery at 04486a5d, 1,100 × 820 content, same baseline interaction/time windows. These are median OS physical footprints including compressed memory. No race instrumentation, debug endpoints, heap instrumentation, forced collection or active GC policy.

Checkpoint Original release Bounded rows
Initial idle 65.86 MiB 68.05 MiB
Representative controls, settled 78.41 MiB 79.08 MiB
100,000 rows loaded and scrolled 894.80 MiB 86.83 MiB
Clear + 50–60 seconds 716.91 MiB 85.95 MiB
Lifetime peak, first cycle 1,153.33 MiB 182.22 MiB

Loaded footprint decreased 90.3%; the matching Clear checkpoint decreased 88.0%. A second ordinary load/Clear cycle measured 93.45/94.03 MiB and a 188.81 MiB lifetime peak. The second Clear was about 8 MiB above the first; this proves neither zero allocator retention nor an indefinitely flat process footprint. The two actual node/state cycles returned exactly to idle.

The allocation-only streaming control still measured 661.30 MiB loaded and 719.27 MiB after Clear. Bounding the mounted representation addresses the dominant live-node cost.

The frozen artifacts, manifests, raw 100 ms samples, AX evidence and reports remain outside Git at /tmp/lui-mygo-release-bounded-20261026/. Both measurement processes closed normally. These footprint measurements are frozen at 04486a5d; later native-host and zero-height edge-case fixes are qualified separately and are not a new release-footprint measurement.

Final merge head 6f6912af814bc1d5eaaa573272a1c6aa7828bbd2 passed all 15 CI jobs and WIP: Core, Web, SwiftUI, Kotlin and Native CI and MyGo desktop and actual OCaml integration CI. The hosted UIKit lifecycle check completed every native phase successfully.

An earlier hosted UIKit launch returned no completion output. Fresh local iOS 26 cold-start and stdin-EOF checks passed; the runner now records durable phase checkpoints and preserves launch logs and crash reports on failure. Assertions and timeouts remain unchanged. The final hosted run completed every native phase successfully.

Merge qualification at 6f6912af814bc1d5eaaa573272a1c6aa7828bbd2 preserves the new main APIs and the bounded-row fork changes in one immutable dependency. The live-input and bounded-row fork tests pass with the race detector. A full local macOS fork run found the same two font-thickening pixel failures (internal/text.TestThick, plugins/terminal.TestFont) reproduced on the earlier 2949164d baseline; no tests were disabled. The complete LUI CI passed on this exact merge head. Local full LUI race tests, production allocation budgets and the real OCaml/MyGo integration suite also passed.

Reproduce the main checks:

make test-schema
make test-mygo
make test-mygo-integration
opam exec -- dune exec test/test_lui.exe
bash tooling/test/build_bounded_virtual_fixture.sh /tmp/lui-bounded-native-fixture
LUI_APPLE_BOUNDED_LIBRARY=/tmp/lui-bounded-native-fixture/liblui_bounded_virtual.dylib \
  swift test --package-path platform/apple --filter LUISparseVirtualNativeTests
swift test --package-path platform/apple
python3 platform/apple/Tests/RuntimeChecks/run-ios-check.py bounded-virtual-rows --simulator booted
python3 platform/android/test_native_virtual_rows.py /tmp/lui-bounded-native-fixture
platform/android/gradlew --no-daemon -p platform/android :lui:testDebugUnitTest
for browser in chromium firefox webkit; do
  LUI_WEB_BROWSER="$browser" node --test platform/web/test/virtual-rows.e2e.mjs
done

@tiensonqin tiensonqin changed the title Reduce large virtual-list memory allocation Bound virtual-list nodes to the host viewport Oct 25, 2026
@tiensonqin
tiensonqin marked this pull request as ready for review October 25, 2026 19:03
@tiensonqin tiensonqin changed the title Bound virtual-list nodes to the host viewport Bound virtual rows across Web, MyGo, SwiftUI and Kotlin Oct 25, 2026
@tiensonqin
tiensonqin merged commit 7717c03 into main Oct 25, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant