Repository navigation
feat(android): implement long-running microphone recording - #122
Merged
Merged
Conversation
houko
force-pushed
the
feat/android-recording
branch
from
November 15, 2026 02:42
2fd40a8 to
7471b2d
Compare
houko
force-pushed
the
feat/android-recording
branch
from
November 15, 2026 04:02
7471b2d to
3517d8a
Compare
houko
force-pushed
the
feat/android-recording
branch
from
November 15, 2026 10:27
3517d8a to
23796f2
Compare
houko
force-pushed
the
feat/android-recording
branch
from
November 15, 2026 12:11
23796f2 to
34e146c
Compare
The user confirmed on 2026-11-15 the answers the Android recording design (nsgo-dev#38) proposes for its open decisions: a file-only first API, with microphone.Capture unsupported on Android; WAV alone, as a killed recording in it stays playable; no service restarting a stopped process to deliver completion, the durable file and the next connection's terminal record being enough; and recording kept in microphone behind a channel in internal/driver, with callbacks off every engine and nsgotest recording one app at a time. The design proposes no default for an interruption, only that it is an option, so ending the recording is the default, as on iOS. The design records them as decisions and ARCHITECTURE.md no longer asks.
…declare The manifest encoders knew android:foregroundServiceType for the overlay's service alone, with only specialUse among its flags, so a service of any other type, such as the microphone's a recording takes, failed the build. foregroundServiceTypes now holds every ServiceInfo.FOREGROUND_SERVICE_TYPE_* name of attrs_manifest.xml, one map for the overlay's service and any other. A test holds every attribute ID the encoders know to the SDK's, as the system reads a binary manifest's attributes by ID and ignores one under another's, and another holds the flags to attrs_manifest.xml's.
…ound service microphone.RecordWAV records 16-bit PCM WAV to a file in the app's recordings directory and returns a Recording, with Stop, which waits for the file to be complete, Ended, File, which reports the rate and channels the device gave and the duration written, Err and Interrupted. Levels of peak and RMS arrive ten times a second by default on a goroutine of microphone's, a waiting one replaced by the next, and Earlier returns the records of earlier runs' recordings: one whose Go process ended (ErrAbandoned) and one a killed app left, repaired at the next start (ErrPartial). New errors tell denied, unavailable, busy, interrupted, full and out of space apart, and an interruption ends the recording unless PauseOnInterruption keeps it. The package reaches Android's runtime through a channel of internal/driver outside any surface, as widget's Live Updates do, carried as JSON in msgRecording and msgRecordingReport. The runtime's RecordingService runs one recording per app, owned by the NsgoEngine whose process started it: it starts only while an activity of the app is visible and after RECORD_AUDIO is granted, asking for it through that activity, from the UI thread; it starts as a microphone foreground service that is never sticky; its reader writes each AudioRecord read to the file itself and rewrites the header about once a second; a recording callback reports a silenced capture from Android 10; a sender thread writes its reports to the engine's pipe so the reader never waits for Go; and the engine's end finishes the file for the next process to hear of. nsgo.yaml's android.recording declares the service and its permissions in the APK's and the bundle's manifest, so only an app that records declares them, and an -embed host declares them itself. classes.dex and classes.jar are rebuilt. Capture stays unsupported on Android, as the design decided.
MicrophoneInput gives an app a fake recorder for microphone.RecordWAV, Authorize and Allowed on every host, macOS's included: it connects to the recording channel of internal/driver as Android's runtime does and writes the samples Microphone.Emit sends into real WAV files under the test's temporary directory, reporting levels, the rate it records at, busy, denied, a taken name and interruptions, and Recordings lists what the app recorded and for how long. The channel is one per process, so Start fails a test whose app would take the fake another running test's app holds rather than letting it hear the other's recordings.
…cation shows From Android 13 a foreground service's notification is not shown unless the app holds POST_NOTIFICATIONS, so an app set up with android.recording recorded with no notification at all. android.recording now declares POST_NOTIFICATIONS, and a start asks for it in the same prompt as RECORD_AUDIO, or alone the first time where RECORD_AUDIO is granted, once, as Notifications.java asks for it. The microphone's grant is read back from the system rather than from the prompt's first result, and a refusal of notifications still records, without a visible notification. RECORD_AUDIO's request code moves to 0x6e7b, as main's web media prompt took 0x6e7a.
…es it The record of a file the next start repaired went only to the process that started, and an abandoned recording's waited in the app process's memory, so a process that ended before calling Earlier, or a killed app, lost it although the file was there. RecordingService now keeps both in .earlier in the recordings directory (RecordingJournal.java), written whole through a temporary file: every Go process that connects hears the records its directory keeps, Earlier tells the runtime which files it returned with a taken command, which lets their records go, and a record whose file the app deleted is dropped. Go replaces a record it has not returned yet with a newer one of the same file. RecordingJournal uses java.* alone, so TestRecordingJournal runs it on the JVM, one process per step, as the Animator test does.
houko
force-pushed
the
feat/android-recording
branch
from
November 15, 2026 13:55
34e146c to
bee8aa8
Compare
Collaborator
Author
|
Reviewed. Rebased onto main after #249: Checked again on devices:
|
houko
marked this pull request as ready for review
November 15, 2026 14:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Implements the Android recording design for #38, with the answers to its open decisions the user confirmed on 2026-11-15: a file-only first API (
microphone.Capturestays unsupported on Android), WAV only, no service restarting a stopped process, and recording kept inmicrophonebehind a channel ofinternal/driver, with callbacks off every engine. The design proposed no default for an interruption; it ends the recording unless the app asks to pause.Commits:
docs: confirm Android recording's decisions: the design's open decisions become its decisions, and ARCHITECTURE.md no longer asks.feat(nsgo): encode every android:foregroundServiceType a service may declare: main's overlay service (feat(android): add system floating windows #130) broughtandroid:foregroundServiceTypewith onlyspecialUseamong its flags;foregroundServiceTypesis now the one map of everyServiceInfo.FOREGROUND_SERVICE_TYPE_*name in attrs_manifest.xml, used by both services.TestFrameworkAttributeIDsholds every attribute ID the encoders know,android:valueincluded, to the SDK's public-final.xml, andTestForegroundServiceTypesholds the flags; main'sTestEncodeManifestOverlayis kept.feat(microphone): record WAV files on Android in the runtime's foreground service:microphone.RecordWAV(name, RecordOptions)returns a*RecordingwithStop(which waits for the file to be complete),Ended,File(path, rate and channels the device gave, duration),ErrandInterrupted.RecordOptionstakesRate,Channels,LevelwithLevelInterval(a tenth of a second by default) andPauseOnInterruption.Earlierreports the recordings of earlier runs:ErrAbandonedwhen the Go process ended,ErrPartialfor a killed app's file repaired at the next start.ErrUnavailable,ErrBusy,ErrInterrupted,ErrFull,ErrNoSpace,ErrAbandonedandErrPartial. A taken name isfs.ErrExist.internal/drivergains a recording channel (SendRecording,ConnectRecordings,HearRecordings,ReportRecording), carried as JSON inmsgRecording(Go to Java, 18) andmsgRecordingReport(Java to Go, 18, after main'smsgTimeZone, 17, from fix(android): give the Go process the device's time zone #249), whichTestJavaMessagesMatchTheBackendholds toProtocol.java.microphonemay now importinternal/driver.RecordingServiceruns one recording per app, owned by theNsgoEnginewhose process started it:RECORD_AUDIOis granted, asking through that activity from the UI thread;START_NOT_STICKY;AudioRecordread to.<name>.partitself, rewrites the header about once a second, and renames the file when it ends;AudioRecordingCallbackreports a silenced capture;UnsupportedOperationExceptionAudioRecord.Builder.build()throws for a format the device cannot open;File().Durationgrows while a recording runs;RECORD_AUDIOchanged in Settings, beside main's overlay permission check.nsgo.yaml'sandroid.recording: truedeclares the service (apk.RecordingService, beside main'sapk.OverlayService, both built by oneforegroundServicehelper, each only when its feature is on) andRECORD_AUDIO,FOREGROUND_SERVICEandFOREGROUND_SERVICE_MICROPHONE(plusPOST_NOTIFICATIONSfrom commit 5). No other APK declares them, and an-embedhost declares them in its own manifest. With the overlay on too, the permissions they share are asked for once (TestEncodeManifestServices).classes.dexandclasses.jarare rebuilt.feat(nsgotest): record what an app records with a fake microphone:MicrophoneInputconnects a fake recorder to the same channel on every host. It writesMicrophone.Emit's samples into real WAV files and lists them inRecordings, one app at a time, and reports the duration written to an app recording without levels.fix(microphone): ask for POST_NOTIFICATIONS so the recording's notification shows: from Android 13 a foreground service's notification is hidden withoutPOST_NOTIFICATIONS.android.recordingnow declares it too, and a start asks for it in the same prompt asRECORD_AUDIO, or alone the first time whereRECORD_AUDIOis already granted, once, asNotifications.javadoes. A refusal of notifications still records, without a visible notification.RECORD_AUDIO's request code moves to0x6e7b, as main's web media prompt took0x6e7a.classes.dexandclasses.jarare rebuilt.fix(microphone): keep an earlier recording's record until the app takes it:RecordingServicekeeps the records of repaired files and of abandoned recordings in.earlierin the recordings directory (RecordingJournal.java, written whole through a temporary file) rather than in memory. Every Go process that connects hears the records its directory keeps.Earliersends atakencommand for each file it returns, which lets that record go, and a record whose file the app deleted is dropped. Go replaces a record it has not returned yet with a newer one of the same file.TestRecordingJournalrunsRecordingJournalon the JVM, one process per step;TestEarlierTellsTheRecorderWhatItTookchecks the Go side.classes.dexandclasses.jarare rebuilt.Refs #38
The branch is rebased onto main at c12742b, after #84, #120, #130, #134, #136, #137, #249 and #271. #249's
TZin the process's environment and itsACTION_TIMEZONE_CHANGEDreceiver inNsgoEnginesit beside this branch's recordings directory andRecordingService.ownerEndedcalls, andRECORDING_REPORTsits afterTIME_ZONEinProtocol.java's Java-to-Go block. #271's service gate coverscore.Scopemethods;microphonehas none, asmicrophone.Captureon main has none.CHANGELOG.mdis left as main has it, frozen until 0.1.0, so commit 3 no longer edits it. Request codes, notification IDs and channels do not clash with main's:RECORD_AUDIOis0x6e7b(main's are0x6e73to0x6e7a), the recording's notification0x6e7372onnsgo.recordingand the overlay's0x6e6f76onnsgo.overlay.Review findings
startForegroundServicefrom the background: a start runs on the UI thread only while an activity is visible, and a refusal ends the recording. A stop never goes throughstartForegroundService.START_STICKY: nowSTART_NOT_STICKY, and a start with no recording stops the service.RECORD_AUDIOdenial notErrDenied:Authorize,AllowedandRecordWAVreportErrDenied.Stopwaits for the end report, and no level starts after it.-embedhost: they are now only in apps withandroid.recording.build()throws rather than returning an uninitialized record): each attempt now catchesUnsupportedOperationExceptionand tries the next format.File().Durationstuck at 0 withoutLevel: the runtime and the fake report the frames written (written), andTestDurationWithoutLevelsandTestMicrophoneInputInterruptscheck it while recording.orphanedread twice inreport(): it is now set and read underRecordingService.class, read once.RecordingService.connectedremembers that engine and hands it the record when the file finishes.dumpsys notification) untilPOST_NOTIFICATIONSwas declared and granted. Fixed in commit 5.ErrPartialrecord was lost when the process that repaired it was killed before callingEarlier. The record lived only in that Go process's memory, and an abandoned recording's only in the app process's. Fixed in commit 6: the records stay on disk untilEarliertakes them.RecordWAVafter the grant took about 5.1 s tostarted, and later ones took 63 ms.RecordingService.beginhas no synchronous wait on another thread or on Go. It runs on the main thread fromonStartCommand, and the start path repairs any.partfile (with an fsync) before the service starts. The firstAudioRecordbuild andstartRecordingin a process, a repair, or a busy main thread right after the permission dialog closes could each explain it, so it is left unchanged until logcat timestamps aroundstartForegroundService,onStartCommand,build()andstartRecordingshow which one it is.AudioRecordruntime tests.Validation
After the rebase onto main at c12742b:
go build ./...,GOOS=android GOARCH=arm64builds ofinternal/platform/android,microphoneandnsgotestand vet ofinternal/platform/androidandmicrophone, andbuild.shreproducing the committedclasses.dexandclasses.jar(rebuilt in commits 3, 5 and 6, as each changes Java)go testofmicrophone,nsgotest,internal/driver,internal/platform/android/...(with fix(android): give the Go process the device's time zone #249'sTestTimeZoneMessageMatchesJavaand this branch'sTestJavaMessagesMatchTheBackend),cmd/nsgo/...andcore, andgo test -run 'Layer|Skills|Doc' .;go vetof the same; builds ofmicrophone,internal/driverandnsgotestfor linux/amd64, windows/amd64, js/wasm, android/arm64, darwin/arm64 and ios/arm64android.recording: trueandSYSTEM_ALERT_WINDOW:aapt2 dump xmltreeshowsRecordingServicewithforegroundServiceType=0x80andOverlayServicewith0x40000000types=0x80) with itsnsgo.recordingnotification; an overlay opened over the launcher beside it (types=0x40000000), and its Close removed the window and its service while the recording went on; Stop ended the recording with a nilErrat 48 kHz mono, no service or notification left, no crash.time.Nowin the probe readEST -0500 America/New_York, the zone set before launch, andIST +0530 Asia/Kolkatawithin 5 s of the zone changing while it ran, matchingdateon the deviceErrat 48 kHz mono and nothing left behind;time.NowreadJST +0900 Asia/Tokyo, as the device's clockAfter the rebase onto main at ec83a11:
go build ./...,go vetofmicrophone,cmd/nsgo/...,internal/driver,internal/platform/android/...andnsgotest, cross-builds ofmicrophone,internal/driverandnsgotestfor android/arm64, linux/amd64, windows/amd64 and js/wasm, andGOOS=android GOARCH=arm64builds and vet ofinternal/platform/android/...andcmd/...go testofmicrophone,cmd/nsgo/apk,cmd/nsgo,internal/driver,internal/platform/android/...,nsgotest, andgo test -run 'Layer|Skills|Doc' .;cmd/nsgo'sTestInitTemplatesfails on this machine as on main, as itsgo mod tidyruns withGOPROXY=offand the module cache lacks goja's test dependenciesbuild.shrebuiltclasses.dexandclasses.jarin commits 3, 5 and 6; a rebuild at the head reproduces themandroid.recording: trueandSYSTEM_ALERT_WINDOW, built with this branch'snsgo build:aapt2 dump xmltreeshowsRecordingServicewithforegroundServiceType=0x80,OverlayServicewith0x40000000and itsPROPERTY_SPECIAL_USE_FGS_SUBTYPE, and each permission oncetypes=0x80andtypes=0x40000000) and the overlay over the launcher; the overlay's Close removed its window and stopped its service while the recording went on; Stop ended it with a nilErrat 48 kHz mono, no service or notification left, no crashErrBefore the rebase:
go tool nsgo check -target allgo tool nsgo test: 1920 passed and 2 failed: the known flakeTestASwipeTurnsTheRing, andTestWebStrictCSP, whose build ran out of disk space and which passes on its own. The run includesTestLayering,TestPublicAPINamesNoInternalType, microphone'srecorder_test.go(withTestDurationWithoutLevels), nsgotest'smicrophone_test.go, the apk manifest tests and the Android protocol constant tests.go tool nsgo test -target macosinmicrophone/go build ./...go vetwithGOOS=windows,GOOS=linuxandGOOS=android GOARCH=arm64on the touched packagesinternal/platform/android/runtime/build.shrebuilt the dex and jargo build ./...;go testofmicrophone,nsgotest,internal/platform/android(withTestRecordingJournalon the JVM),internal/driver,cmd/nsgo,cmd/nsgo/apkand the root package;go veton them, and withGOOS=android GOARCH=arm64onmicrophoneandinternal/platform/android; the dex and jar of commits 3, 5 and 6 were each built from that commit's sources, and a rebuild at the head reproduces the committed onesNot yet covered: apart from
RecordingJournal, which runs on the JVM, the Java runtime has no test harness, so neither the design's Android protocol tests (start and stop, busy, writer failure, repair, a dead Go process, level back-pressure) nor its runtime tests with a fakeAudioRecordexist; the roadmap lists both.Screenshots
None: no view of the framework changes.
Device checks still needed
An app with
android.recording: true, on an Android 9 or 10 device, one of 11 to 13 and one of 14 or later:foregroundServiceType=0x80onRecordingServiceand asks forPOST_NOTIFICATIONS. On Android 13 or later, a firstRecordWAVasks forRECORD_AUDIOand notifications in one prompt and records with the notification shown (dumpsys notificationlists it). Refusing notifications still records. On an install whereRECORD_AUDIOwas granted before this change, the firstRecordWAVasks for notifications alone;ErrDenied, and granting it later in Settings records without a restart;audio.Openat the reported rate, with levels at 10/s or more;.partfile, which the nextRecordWAVrepairs andEarlierreports asErrPartial, with no stuck notification. If the app is killed again after that repair and beforeEarlierruns, a later process'sEarlierstill reports the file. Once oneEarlierhas returned it, no later process reports it again;ErrInterrupted, or pauses it withPauseOnInterruption;ErrUnavailablewithout a crash;Channels: 2on a device without a stereo microphone records mono rather than failing;-embedhost's recording engine and attaching a new one at once: the new app'smicrophone.Earlier()reports the abandoned recording.