Skip to content

feat(android): implement long-running microphone recording - #122

Merged
houko merged 6 commits into
nsgo-dev:mainfrom
houko:feat/android-recording
Nov 15, 2026
Merged

houko merged 6 commits into
nsgo-dev:mainfrom
houko:feat/android-recording

Conversation

@houko

@houko houko commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

Implements the Android recording design for #38, with the answers to its open decisions the user confirmed on 2026-11-15: a file-only first API (microphone.Capture stays unsupported on Android), WAV only, no service restarting a stopped process, and recording kept in microphone behind a channel of internal/driver, with callbacks off every engine. The design proposed no default for an interruption; it ends the recording unless the app asks to pause.

Commits:

  1. docs: confirm Android recording's decisions: the design's open decisions become its decisions, and ARCHITECTURE.md no longer asks.
  2. feat(nsgo): encode every android:foregroundServiceType a service may declare: main's overlay service (feat(android): add system floating windows #130) brought android:foregroundServiceType with only specialUse among its flags; foregroundServiceTypes is now the one map of every ServiceInfo.FOREGROUND_SERVICE_TYPE_* name in attrs_manifest.xml, used by both services. TestFrameworkAttributeIDs holds every attribute ID the encoders know, android:value included, to the SDK's public-final.xml, and TestForegroundServiceTypes holds the flags; main's TestEncodeManifestOverlay is kept.
  3. feat(microphone): record WAV files on Android in the runtime's foreground service:
    • microphone.RecordWAV(name, RecordOptions) returns a *Recording with Stop (which waits for the file to be complete), Ended, File (path, rate and channels the device gave, duration), Err and Interrupted. RecordOptions takes Rate, Channels, Level with LevelInterval (a tenth of a second by default) and PauseOnInterruption. Earlier reports the recordings of earlier runs: ErrAbandoned when the Go process ended, ErrPartial for a killed app's file repaired at the next start.
    • New errors: ErrUnavailable, ErrBusy, ErrInterrupted, ErrFull, ErrNoSpace, ErrAbandoned and ErrPartial. A taken name is fs.ErrExist.
    • internal/driver gains a recording channel (SendRecording, ConnectRecordings, HearRecordings, ReportRecording), carried as JSON in msgRecording (Go to Java, 18) and msgRecordingReport (Java to Go, 18, after main's msgTimeZone, 17, from fix(android): give the Go process the device's time zone #249), which TestJavaMessagesMatchTheBackend holds to Protocol.java. microphone may now import internal/driver.
    • The Java runtime's RecordingService runs one recording per app, owned by the NsgoEngine whose process started it:
      • it starts only while an activity of the app is visible and after RECORD_AUDIO is granted, asking through that activity from the UI thread;
      • it is a microphone foreground service returning START_NOT_STICKY;
      • the reader writes each AudioRecord read to .<name>.part itself, rewrites the header about once a second, and renames the file when it ends;
      • from Android 10, an AudioRecordingCallback reports a silenced capture;
      • it tries the asked rate, then 48, 44.1 and 16 kHz, stereo then mono, catching the UnsupportedOperationException AudioRecord.Builder.build() throws for a format the device cannot open;
      • a per-recording sender thread writes its reports to the pipe, keeping only the latest level, and without levels reports the frames written about once a second, so File().Duration grows while a recording runs;
      • the engine's process ending finishes the file and hands its record to the next process: at once if one connected while the file was finishing, else when one connects. The owner's end and the record's destination are decided under one lock.
    • The first view, not an overlay's, tells the app as its window regains the focus whether RECORD_AUDIO changed in Settings, beside main's overlay permission check.
    • nsgo.yaml's android.recording: true declares the service (apk.RecordingService, beside main's apk.OverlayService, both built by one foregroundService helper, each only when its feature is on) and RECORD_AUDIO, FOREGROUND_SERVICE and FOREGROUND_SERVICE_MICROPHONE (plus POST_NOTIFICATIONS from commit 5). No other APK declares them, and an -embed host declares them in its own manifest. With the overlay on too, the permissions they share are asked for once (TestEncodeManifestServices).
    • classes.dex and classes.jar are rebuilt.
  4. feat(nsgotest): record what an app records with a fake microphone: MicrophoneInput connects a fake recorder to the same channel on every host. It writes Microphone.Emit's samples into real WAV files and lists them in Recordings, one app at a time, and reports the duration written to an app recording without levels.
  5. fix(microphone): ask for POST_NOTIFICATIONS so the recording's notification shows: from Android 13 a foreground service's notification is hidden without POST_NOTIFICATIONS. android.recording now declares it too, and a start asks for it in the same prompt as RECORD_AUDIO, or alone the first time where RECORD_AUDIO is already granted, once, as Notifications.java does. A refusal of notifications still records, without a visible notification. RECORD_AUDIO's request code moves to 0x6e7b, as main's web media prompt took 0x6e7a. classes.dex and classes.jar are rebuilt.
  6. fix(microphone): keep an earlier recording's record until the app takes it: RecordingService keeps the records of repaired files and of abandoned recordings in .earlier in the recordings directory (RecordingJournal.java, written whole through a temporary file) rather than in memory. Every Go process that connects hears the records its directory keeps. Earlier sends a taken command for each file it returns, which lets that record go, and a record whose file the app deleted is dropped. Go replaces a record it has not returned yet with a newer one of the same file. TestRecordingJournal runs RecordingJournal on the JVM, one process per step; TestEarlierTellsTheRecorderWhatItTook checks the Go side. classes.dex and classes.jar are rebuilt.

Refs #38

The branch is rebased onto main at c12742b, after #84, #120, #130, #134, #136, #137, #249 and #271. #249's TZ in the process's environment and its ACTION_TIMEZONE_CHANGED receiver in NsgoEngine sit beside this branch's recordings directory and RecordingService.ownerEnded calls, and RECORDING_REPORT sits after TIME_ZONE in Protocol.java's Java-to-Go block. #271's service gate covers core.Scope methods; microphone has none, as microphone.Capture on main has none. CHANGELOG.md is left as main has it, frozen until 0.1.0, so commit 3 no longer edits it. Request codes, notification IDs and channels do not clash with main's: RECORD_AUDIO is 0x6e7b (main's are 0x6e73 to 0x6e7a), the recording's notification 0x6e7372 on nsgo.recording and the overlay's 0x6e6f76 on nsgo.overlay.

Review findings

  • foregroundServiceType's attribute ID and flags: fixed in commit 2.
  • Conflicts with main's NsgoEngine/NsgoView split: rebuilt on main. Recordings belong to the engine, a view detaching leaves them running, and the engine's end finishes them.
  • startForegroundService from the background: a start runs on the UI thread only while an activity is visible, and a refusal ends the recording. A stop never goes through startForegroundService.
  • Permission prompt answered late: there is no start timeout. A process that ends meanwhile leaves nothing to start.
  • Zeroed WAV header: Java writes the real header at once and rewrites it about once a second. The next start repairs a killed app's file. 4 GiB and low space are terminal states.
  • Callbacks blocking the serve loop: the backend's report handler only updates state, and levels are coalesced on microphone's goroutine. Java's reader never writes to the pipe.
  • START_STICKY: now START_NOT_STICKY, and a start with no recording stops the service.
  • Static callback across views: each recording is owned by one engine, and a second start is busy.
  • RECORD_AUDIO denial not ErrDenied: Authorize, Allowed and RecordWAV report ErrDenied.
  • Stop's guarantee: Stop waits for the end report, and no level starts after it.
  • Ineffective nsgotest fake: it now works on macOS hosts too and refuses a second app.
  • Permissions declared in every APK and -embed host: they are now only in apps with android.recording.
  • Dead rate/channel fallback (build() throws rather than returning an uninitialized record): each attempt now catches UnsupportedOperationException and tries the next format.
  • File().Duration stuck at 0 without Level: the runtime and the fake report the frames written (written), and TestDurationWithoutLevels and TestMicrophoneInputInterrupts check it while recording.
  • orphaned read twice in report(): it is now set and read under RecordingService.class, read once.
  • An abandoned recording's record missing the process that connected while its file was finishing: RecordingService.connected remembers that engine and hands it the record when the file finishes.
  • Device check, Android 15 emulator: the microphone service's notification was invisible (no record in dumpsys notification) until POST_NOTIFICATIONS was declared and granted. Fixed in commit 5.
  • Device check: a repaired ErrPartial record was lost when the process that repaired it was killed before calling Earlier. The record lived only in that Go process's memory, and an abandoned recording's only in the app process's. Fixed in commit 6: the records stay on disk until Earlier takes them.
  • Device check, Pixel: the first RecordWAV after the grant took about 5.1 s to started, and later ones took 63 ms. RecordingService.begin has no synchronous wait on another thread or on Go. It runs on the main thread from onStartCommand, and the start path repairs any .part file (with an fsync) before the service starts. The first AudioRecord build and startRecording in a process, a repair, or a busy main thread right after the permission dialog closes could each explain it, so it is left unchanged until logcat timestamps around startForegroundService, onStartCommand, build() and startRecording show which one it is.
  • Missing Android protocol tests: still missing, as the Java runtime has no harness; the design's status and the roadmap's row now list them beside the fake-AudioRecord runtime tests.

Validation

After the rebase onto main at c12742b:

  • every commit builds and its binaries match its sources: at each commit, go build ./..., GOOS=android GOARCH=arm64 builds of internal/platform/android, microphone and nsgotest and vet of internal/platform/android and microphone, and build.sh reproducing the committed classes.dex and classes.jar (rebuilt in commits 3, 5 and 6, as each changes Java)
  • at the head: go test of microphone, nsgotest, internal/driver, internal/platform/android/... (with fix(android): give the Go process the device's time zone #249's TestTimeZoneMessageMatchesJava and this branch's TestJavaMessagesMatchTheBackend), cmd/nsgo/... and core, and go test -run 'Layer|Skills|Doc' .; go vet of the same; builds of microphone, internal/driver and nsgotest for linux/amd64, windows/amd64, js/wasm, android/arm64, darwin/arm64 and ios/arm64
  • a probe with android.recording: true and SYSTEM_ALERT_WINDOW: aapt2 dump xmltree shows RecordingService with foregroundServiceType=0x80 and OverlayService with 0x40000000
  • Android 15 emulator: Record started a microphone foreground service (types=0x80) with its nsgo.recording notification; an overlay opened over the launcher beside it (types=0x40000000), and its Close removed the window and its service while the recording went on; Stop ended the recording with a nil Err at 48 kHz mono, no service or notification left, no crash. time.Now in the probe read EST -0500 America/New_York, the zone set before launch, and IST +0530 Asia/Kolkata within 5 s of the zone changing while it ran, matching date on the device
  • Pixel 3 XL, Android 12: the same sequence, both services in the foreground, the overlay closed with its service, Stop ended the recording with a nil Err at 48 kHz mono and nothing left behind; time.Now read JST +0900 Asia/Tokyo, as the device's clock

After the rebase onto main at ec83a11:

  • every commit builds: go build ./..., go vet of microphone, cmd/nsgo/..., internal/driver, internal/platform/android/... and nsgotest, cross-builds of microphone, internal/driver and nsgotest for android/arm64, linux/amd64, windows/amd64 and js/wasm, and GOOS=android GOARCH=arm64 builds and vet of internal/platform/android/... and cmd/...
  • go test of microphone, cmd/nsgo/apk, cmd/nsgo, internal/driver, internal/platform/android/..., nsgotest, and go test -run 'Layer|Skills|Doc' .; cmd/nsgo's TestInitTemplates fails on this machine as on main, as its go mod tidy runs with GOPROXY=off and the module cache lacks goja's test dependencies
  • build.sh rebuilt classes.dex and classes.jar in commits 3, 5 and 6; a rebuild at the head reproduces them
  • a probe with android.recording: true and SYSTEM_ALERT_WINDOW, built with this branch's nsgo build: aapt2 dump xmltree shows RecordingService with foregroundServiceType=0x80, OverlayService with 0x40000000 and its PROPERTY_SPECIAL_USE_FGS_SUBTYPE, and each permission once
  • Android 15 emulator: the first Record asked for the microphone, then notifications; the recording ran with its notification; an overlay opened while it ran, both services in the foreground (types=0x80 and types=0x40000000) and the overlay over the launcher; the overlay's Close removed its window and stopped its service while the recording went on; Stop ended it with a nil Err at 48 kHz mono, no service or notification left, no crash
  • Pixel 3 XL, Android 12: the same sequence, both services in the foreground, the overlay closed with its service, and Stop ended the recording with a nil Err

Before the rebase:

  • go tool nsgo check -target all
  • go tool nsgo test: 1920 passed and 2 failed: the known flake TestASwipeTurnsTheRing, and TestWebStrictCSP, whose build ran out of disk space and which passes on its own. The run includes TestLayering, TestPublicAPINamesNoInternalType, microphone's recorder_test.go (with TestDurationWithoutLevels), nsgotest's microphone_test.go, the apk manifest tests and the Android protocol constant tests.
  • go tool nsgo test -target macos in microphone/
  • every commit after main builds with go build ./...
  • go vet with GOOS=windows, GOOS=linux and GOOS=android GOARCH=arm64 on the touched packages
  • internal/platform/android/runtime/build.sh rebuilt the dex and jar
  • after the rebase onto main and commits 5 and 6: go build ./...; go test of microphone, nsgotest, internal/platform/android (with TestRecordingJournal on the JVM), internal/driver, cmd/nsgo, cmd/nsgo/apk and the root package; go vet on them, and with GOOS=android GOARCH=arm64 on microphone and internal/platform/android; the dex and jar of commits 3, 5 and 6 were each built from that commit's sources, and a rebuild at the head reproduces the committed ones

Not yet covered: apart from RecordingJournal, which runs on the JVM, the Java runtime has no test harness, so neither the design's Android protocol tests (start and stop, busy, writer failure, repair, a dead Go process, level back-pressure) nor its runtime tests with a fake AudioRecord exist; the roadmap lists both.

Screenshots

None: no view of the framework changes.

Device checks still needed

An app with android.recording: true, on an Android 9 or 10 device, one of 11 to 13 and one of 14 or later:

  • the manifest carries foregroundServiceType=0x80 on RecordingService and asks for POST_NOTIFICATIONS. On Android 13 or later, a first RecordWAV asks for RECORD_AUDIO and notifications in one prompt and records with the notification shown (dumpsys notification lists it). Refusing notifications still records. On an install where RECORD_AUDIO was granted before this change, the first RecordWAV asks for notifications alone;
  • "Don't allow" gives ErrDenied, and granting it later in Settings records without a restart;
  • a 30-minute recording through screen-off, backgrounding and a rotation plays with audio.Open at the reported rate, with levels at 10/s or more;
  • killing the app mid-recording leaves a playable .part file, which the next RecordWAV repairs and Earlier reports as ErrPartial, with no stuck notification. If the app is killed again after that repair and before Earlier runs, a later process's Earlier still reports the file. Once one Earlier has returned it, no later process reports it again;
  • playback and recording run together;
  • a call ends the recording with ErrInterrupted, or pauses it with PauseOnInterruption;
  • a start while backgrounded gives ErrUnavailable without a crash;
  • stopping leaves no open descriptor;
  • Channels: 2 on a device without a stereo microphone records mono rather than failing;
  • destroying an -embed host's recording engine and attaching a new one at once: the new app's microphone.Earlier() reports the abandoned recording.

@github-actions github-actions Bot added the conflict Conflicts with the base branch; resolve it first label Nov 14, 2026
@houko
houko force-pushed the feat/android-recording branch from 2fd40a8 to 7471b2d Compare November 15, 2026 02:42
@github-actions github-actions Bot added enhancement New feature or request size/XL documentation Improvements or additions to documentation Android core cli and removed conflict Conflicts with the base branch; resolve it first labels Nov 15, 2026
@houko
houko force-pushed the feat/android-recording branch from 7471b2d to 3517d8a Compare November 15, 2026 04:02
@github-actions github-actions Bot added the conflict Conflicts with the base branch; resolve it first label Nov 15, 2026
@houko
houko force-pushed the feat/android-recording branch from 3517d8a to 23796f2 Compare November 15, 2026 10:27
@github-actions github-actions Bot added conflict Conflicts with the base branch; resolve it first and removed conflict Conflicts with the base branch; resolve it first labels Nov 15, 2026
@houko
houko force-pushed the feat/android-recording branch from 23796f2 to 34e146c Compare November 15, 2026 12:11
@github-actions github-actions Bot added conflict Conflicts with the base branch; resolve it first and removed conflict Conflicts with the base branch; resolve it first labels Nov 15, 2026
The user confirmed on 2026-11-15 the answers the Android recording design (nsgo-dev#38) proposes for its open decisions: a file-only first API, with microphone.Capture unsupported on Android; WAV alone, as a killed recording in it stays playable; no service restarting a stopped process to deliver completion, the durable file and the next connection's terminal record being enough; and recording kept in microphone behind a channel in internal/driver, with callbacks off every engine and nsgotest recording one app at a time. The design proposes no default for an interruption, only that it is an option, so ending the recording is the default, as on iOS. The design records them as decisions and ARCHITECTURE.md no longer asks.
…declare

The manifest encoders knew android:foregroundServiceType for the overlay's service alone, with only specialUse among its flags, so a service of any other type, such as the microphone's a recording takes, failed the build. foregroundServiceTypes now holds every ServiceInfo.FOREGROUND_SERVICE_TYPE_* name of attrs_manifest.xml, one map for the overlay's service and any other. A test holds every attribute ID the encoders know to the SDK's, as the system reads a binary manifest's attributes by ID and ignores one under another's, and another holds the flags to attrs_manifest.xml's.
…ound service

microphone.RecordWAV records 16-bit PCM WAV to a file in the app's recordings directory and returns a Recording, with Stop, which waits for the file to be complete, Ended, File, which reports the rate and channels the device gave and the duration written, Err and Interrupted. Levels of peak and RMS arrive ten times a second by default on a goroutine of microphone's, a waiting one replaced by the next, and Earlier returns the records of earlier runs' recordings: one whose Go process ended (ErrAbandoned) and one a killed app left, repaired at the next start (ErrPartial). New errors tell denied, unavailable, busy, interrupted, full and out of space apart, and an interruption ends the recording unless PauseOnInterruption keeps it.

The package reaches Android's runtime through a channel of internal/driver outside any surface, as widget's Live Updates do, carried as JSON in msgRecording and msgRecordingReport. The runtime's RecordingService runs one recording per app, owned by the NsgoEngine whose process started it: it starts only while an activity of the app is visible and after RECORD_AUDIO is granted, asking for it through that activity, from the UI thread; it starts as a microphone foreground service that is never sticky; its reader writes each AudioRecord read to the file itself and rewrites the header about once a second; a recording callback reports a silenced capture from Android 10; a sender thread writes its reports to the engine's pipe so the reader never waits for Go; and the engine's end finishes the file for the next process to hear of. nsgo.yaml's android.recording declares the service and its permissions in the APK's and the bundle's manifest, so only an app that records declares them, and an -embed host declares them itself.

classes.dex and classes.jar are rebuilt. Capture stays unsupported on Android, as the design decided.
MicrophoneInput gives an app a fake recorder for microphone.RecordWAV, Authorize and Allowed on every host, macOS's included: it connects to the recording channel of internal/driver as Android's runtime does and writes the samples Microphone.Emit sends into real WAV files under the test's temporary directory, reporting levels, the rate it records at, busy, denied, a taken name and interruptions, and Recordings lists what the app recorded and for how long. The channel is one per process, so Start fails a test whose app would take the fake another running test's app holds rather than letting it hear the other's recordings.
…cation shows

From Android 13 a foreground service's notification is not shown unless the app holds POST_NOTIFICATIONS, so an app set up with android.recording recorded with no notification at all. android.recording now declares POST_NOTIFICATIONS, and a start asks for it in the same prompt as RECORD_AUDIO, or alone the first time where RECORD_AUDIO is granted, once, as Notifications.java asks for it. The microphone's grant is read back from the system rather than from the prompt's first result, and a refusal of notifications still records, without a visible notification.

RECORD_AUDIO's request code moves to 0x6e7b, as main's web media prompt took 0x6e7a.
…es it

The record of a file the next start repaired went only to the process that started, and an abandoned recording's waited in the app process's memory, so a process that ended before calling Earlier, or a killed app, lost it although the file was there. RecordingService now keeps both in .earlier in the recordings directory (RecordingJournal.java), written whole through a temporary file: every Go process that connects hears the records its directory keeps, Earlier tells the runtime which files it returned with a taken command, which lets their records go, and a record whose file the app deleted is dropped. Go replaces a record it has not returned yet with a newer one of the same file.

RecordingJournal uses java.* alone, so TestRecordingJournal runs it on the JVM, one process per step, as the Animator test does.
@houko
houko force-pushed the feat/android-recording branch from 34e146c to bee8aa8 Compare November 15, 2026 13:55
@github-actions github-actions Bot removed the conflict Conflicts with the base branch; resolve it first label Nov 15, 2026
@houko

houko commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

Reviewed. Rebased onto main after #249: RECORDING_REPORT moved to Java-to-Go 18, since #249 took 17 for TIME_ZONE, and both protocol tests pass. The Android runtime's classes.dex and classes.jar rebuild byte for byte from each commit's Java.

Checked again on devices:

  • Android 15 emulator: recording ran in a microphone foreground service with its notification, an overlay opened and closed beside it without stopping it, and stopping left no service or notification behind. A time zone change while the app ran reached Go within 5 seconds.
  • Pixel 3 XL (Android 12): the same recording and overlay sequence passed (48 kHz mono, no crash), and the zone matched the device's.

@houko
houko marked this pull request as ready for review November 15, 2026 14:33
@houko
houko merged commit a6e0e60 into nsgo-dev:main Nov 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Android cli core documentation Improvements or additions to documentation enhancement New feature or request size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant