Skip to content

feat(core): expose foldable display features - #121

Merged
houko merged 6 commits into
nsgo-dev:mainfrom
houko:feat/foldable-geometry
Nov 15, 2026
Merged

houko merged 6 commits into
nsgo-dev:mainfrom
houko:feat/foldable-geometry

Conversation

@houko

@houko houko commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Implements the foldable design in docs/architecture/foldable.md for #49, with the answers to its decisions the user confirmed on 2026-11-15. This replaces the branch's earlier model of a fold, hinge, cutout and bar Kind with a flat, half-folded or folded posture sent through SetEnvironment, which the design rejected.

What it does

  • internal/driver gains Geometry: usable Regions, each with a frame, a safe area and a reserved rectangle, and Obstructions, each with a frame, an orientation and separate Occluding, Separating and State (HingeUnknown, HingeClosed, HingeHalfOpen, HingeOpen), plus an angle where the platform reports one. driver.Divide derives the regions from the obstructions at a size: it clips frames to the surface, drops frames outside it, and orders regions and obstructions by position.
  • Geometry travels with the surface size, not with SetEnvironment. Engine.Resize, Engine.ResizeAndWait and Tree.Resize take it with the size, and engine.Options.Geometry gives it to the first commit with Options.Size, so an app started on a half-open foldable builds its first tree with both regions. Merged resizes keep the newest size and geometry together, and Tree.Resize sets the size and the geometry in one batch, so a new geometry at the same size still applies.
  • core.Env.Geometry exposes it as a signal that changes only when the geometry, or the safe area inside its regions, changes. It returns a copy. Core lays nothing out by it. System bars and cutouts stay in SafeArea.
  • Android reaches the device's androidx.window.extensions by reflection (Folds.java), at vendor API levels 1 and 2+, from API 30. The library takes one listener per context, so all NsgoViews of an activity share one listener that fans each report out to them. A report is sent at the view's next layout, so it goes with the size the view gets for the window it was measured at, and a view that attaches again sends the latest report itself. Frames are clipped to the view in pixels, so an embedded view offset in its window keeps a fold that crosses it. The runtime sends the features with START and RESIZE, and sends none while the window's bounds differ from the ones the latest report was measured at. The extension sends its first report only after the window has drawn its first frame, so on a device that folds (a fold in the extension's getSupportedWindowFeatures from vendor level 6, else a hinge angle sensor) the view holds the app's start until that report: at most 300 ms from the first frame, or 3 s from the view's first size for a window that never draws, after which it starts without the folds and a RESIZE brings them. Devices without the extension, and devices that do not fold, start at once. The Go side derives the occluding and separating properties as Jetpack's HardwareFoldingFeature does. classes.dex and classes.jar are rebuilt.
  • nsgo build declares <uses-library android:name="androidx.window.extensions" android:required="false"/> in the APK, App Bundle or -embed AAR only for an app that reads core.Env.Geometry: its executable must keep the method's name, and one of its packages outside NSGo's internal ones must select a Geometry that is not an imported package's. The second check is there because reflection by non-constant names, as tsx uses, keeps every exported method, so every TSX app's executable names the method. No other app declares the library.
  • iOS 27.1 reads the surface view's active reservedRegionsOfKind: regions in layoutSubviews and when the engine starts, so they go with the size it reports and with the first commit. It adds a UIHingeInteraction whose updates set the state and the angle and lay the view out again. It also reports the trait collection's verticalBarEdge as Environment.VerticalBarEdge / Env.VerticalBarEdge. The backend checks classes and selectors first, so older systems report one region.
  • Every other target reports the one-region form.
  • Docs: the design's open decisions become confirmed decisions dated 2026-11-15. The entry is removed from ARCHITECTURE.md's open questions, and the roadmap row now lists what is left (device checks and a gallery page). CHANGELOG.md is untouched, as main freezes it until 0.1.0.

Review findings addressed

  • The Windows and Linux builds broke because Environment was no longer comparable. Main's Environment.Equal already covers this, and the geometry is no longer in Environment.
  • The public API exposed an internal type. Every new type (Geometry, Region, Obstruction, HingeState, VerticalBarEdge) and its constants are now aliased in core, and TestPublicAPINamesNoInternalType passes.
  • Conflicts with main's Environment.Displays and Equal: the branch is rebuilt on main and keeps main's Equal.
  • The signals notified on every change. The geometry signal and memo now compare with Geometry.Equal, the stored value is cloned on the way in and out, and tests check that unrelated environment changes do not rerun readers.
  • The CHANGELOG clause is dropped: main freezes CHANGELOG.md until 0.1.0.
  • A reattached Android view lost its folds: the extension's first report arrived while the view's Folds was still unset. Registration no longer calls the views, and the view sends its size with the latest report once it is listening.
  • A second NsgoView in an activity got no reports, because the extension ignores a second listener for the same context. There is now one listener per activity, shared by its views.
  • A report could go out with the view's old size, since the extension reports when a configuration is applied, before the view is measured. Reports now wait for the view's next layout.
  • The first Android commit was laid out without the geometry START carried. The engine now starts with it, on iOS too.
  • Float rounding dropped folds crossing an offset embedded view. Frames are now clipped to the view in pixels.
  • Every TSX app declared the window extensions library. Detection now also checks the sources (see above).
  • An app cold-launched on a half-open foldable sent START before the extension's first report, which arrives after the window's first frame, so it built its first tree with one region and split it in a RESIZE 54-315 ms later. The view now holds the start for that report on devices that fold (see above).
  • Rebased onto main after feat(android): add system floating windows #130 (floating windows): aarManifest takes both the overlay flag and the libraries, the binary manifest keeps both the overlay service and uses-library, and the overlay surfaces' Resize calls (Android runtime and nsgotest) pass an empty geometry, since an overlay window over other apps has no fold.
  • Rebased onto feat(android): implement long-running microphone recording #122 (which includes main's fix(android): give the Go process the device's time zone #249, the Android time zone): the APK manifest keeps both feat(android): implement long-running microphone recording #122's Recording service and the uses-library for the window extensions, docs/architecture/tool.md keeps both paragraphs' sentences, and ARCHITECTURE.md drops both resolved open questions. feat(android): implement long-running microphone recording #122's TestFrameworkAttributeIDs checks every attribute ID the manifest encoder writes, so it now lists android:required (0x0101028e, android.R.attr.required). The Java in Folds.java and NsgoView.java merged without changes to this branch's hunks. The half-open cold-launch checks on the pixel_fold emulator below therefore still describe this code. classes.dex and classes.jar are rebuilt at each commit that changes Java and rebuild byte for byte from that commit's sources.

Validation

After the rebase onto #122, at each of the six commits: go tool nsgo check -target all (vet for every target, GOOS=android included), go tool nsgo test of internal/driver, core, internal/engine, internal/platform/..., nsgotest and cmd/nsgo/..., and go tool nsgo test -run 'Layer|Skills|Doc' .. On a Pixel 3 XL (API 31, no window extensions), a probe app with a log of the view's first size and START: 9-11 ms from the first size to START over 6 cold launches (one outlier at 28 ms under host load), against 8-13 ms for the same probe built on #122's head; rotation reports one region at 411x846 and 846x411; time.Now reads the device's clock and Asia/Tokyo zone.

Before the rebase:

  • go test of every touched package (cmd/nsgo, cmd/nsgo/apk, core, internal/driver, internal/engine, internal/platform/android, internal/platform/ios, internal/platform/macos, internal/platform/miniprogram, internal/platform/watchos, nsgotest) and go test -run 'Layer|Skills|Doc' .
  • go vet ./... for darwin, GOOS=ios, GOOS=android GOARCH=arm64, GOOS=linux, GOOS=windows and GOOS=js GOARCH=wasm
  • classes.dex and classes.jar rebuild byte for byte from the committed Java
  • New tests: TestStartLaysOutWithItsGeometry (engine), TestAndroidLibraries and TestFileSelectsGeometry (tool)

Device checks, with a probe app that draws each region and logs every geometry it reads:

  • pixel_fold emulator, API 35: 6 cold launches half open with the final build (16 counting earlier iterations of the fix), each with START carrying the fold and the first tree built with two regions; screen recordings of 4 of them show the first frame presented after the splash already split. The same probe built from the previous head sent START without the fold and split 155-229 ms later. Flat reports one region with a non-separating open fold; half open, rotation (horizontal fold), back, fold (outer display, one region) and unfold all report as before. A cold launch on the folded outer display starts as soon as the first report (no folds) arrives. With the extension's reports ignored, the app starts 300 ms after the first frame with one region.
  • Pixel 3 XL (API 31, no window extensions): the time from the view's first size to START is unchanged, 7-14 ms before and 7-18 ms after over 6 launches each, the process spawn either way.

Device checks still needed:

  • A physical Android foldable (Pixel Fold or Galaxy Z Fold): the first frame at launch half open, and split-screen and freeform resizes across the fold, including from the left edge, never pairing a geometry with the old size.
  • Android -embed host: two NsgoViews in one activity both report the hinge, detaching one leaves the other reporting, removing a view and adding it back at the same size keeps two regions, and a view placed below a toolbar keeps a vertical fold.
  • A non-foldable Android device with the window extensions (a Pixel phone on Android 13 or later) starts without waiting.
  • iOS 27.1 fold simulator or device: open, half-open and closed hinge, rotation, and the vertical bar on the leading and trailing edges. Confirm that a hinge comes as one occlusion region and one division region at the same frame, that the first commit at launch already has the regions, and that the vertical bar comes only through the safe area. Check that an earlier iOS reports one region.

@github-actions github-actions Bot added the conflict Conflicts with the base branch; resolve it first label Nov 14, 2026
@houko
houko force-pushed the feat/foldable-geometry branch from e20040e to e95afb3 Compare November 15, 2026 02:26
@github-actions github-actions Bot added enhancement New feature or request size/XL documentation Improvements or additions to documentation macOS iOS watchOS Windows Linux Web Android Mini-program core cli and removed conflict Conflicts with the base branch; resolve it first labels Nov 15, 2026
@houko
houko force-pushed the feat/foldable-geometry branch from e95afb3 to e190ebf Compare November 15, 2026 03:44
@github-actions github-actions Bot added the conflict Conflicts with the base branch; resolve it first label Nov 15, 2026
@houko
houko force-pushed the feat/foldable-geometry branch from e190ebf to 5437746 Compare November 15, 2026 12:20
@github-actions github-actions Bot added conflict Conflicts with the base branch; resolve it first and removed conflict Conflicts with the base branch; resolve it first labels Nov 15, 2026
@houko
houko force-pushed the feat/foldable-geometry branch from 5437746 to 37b4b15 Compare November 15, 2026 14:32
@github-actions github-actions Bot removed the conflict Conflicts with the base branch; resolve it first label Nov 15, 2026
The device's window extensions send their first report of the folds only after the window has drawn its first frame, so an app cold-launched half open sent START without them, built and presented its first tree with one region and split it in a RESIZE a frame or more later. On a device that folds, by the folds the extensions list from vendor level 6 or by a hinge angle sensor before it, the view now holds the app's start until that first report, 300 ms at most from the first frame or 3 s from its first size for a window that does not draw. Devices without the extensions, and those that do not fold, start at once as before.
@houko
houko force-pushed the feat/foldable-geometry branch from 37b4b15 to e74054b Compare November 15, 2026 14:34
@houko

houko commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

Reviewed. Rebased onto main after #122 and #249. The Java in Folds.java and NsgoView.java is unchanged from the version checked on the foldable emulator, and the Android runtime's classes.dex and classes.jar rebuild byte for byte from each commit's Java. TestFrameworkAttributeIDs from #122 now lists android:required (0x0101028e).

On a Pixel 3 XL, START still follows the view's first size within about 10 ms on cold launches, rotation reports one region with the safe area following the notch, and the clock and zone match the device's.

One related gap that exists on main and is not introduced here: Android sends a rotation's safe area before its new size, so for a few milliseconds the engine sees the new safe area with the old window size.

@houko
houko marked this pull request as ready for review November 15, 2026 14:34
@houko
houko merged commit 330763d into nsgo-dev:main Nov 15, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant