[houko]

What changed

Stage 3 of audio playback for macOS and iOS: the Apple backend behind the audio package of #84, which is merged. The branch is rebased onto main (with #271 and #106) and holds only this PR's seven commits.

• internal/platform/apple/audio.go is one Audio per engine, on the main thread, modeled on NSH's audio.rs. A surface makes it with its first ItemAudio and closes it as it closes.
• Sounds. They are read through imageload, held to 8 MiB before decoding, written to a temporary file and decoded into an AVAudioPCMBuffer by AVAudioFile on a goroutine of their own inside an autorelease pool. A sound longer than 5 seconds is reported cut without decoding. Each playback is a voice, up to 32 of them, on one AVAudioEngine per engine. A voice is an AVAudioPlayerNode into an AVAudioUnitVarispeed for the rate, then a mixer of its own for the volume, because an effect node keeps the player's own volume from the main mixer. A varispeed that is already connected refuses another format (-10868), so a sound of another format gets a new voice. The engine starts at a playback, starts again after the system stopped it, and pauses two seconds after the last playback ends.
• Players. They are layerless AVPlayers, as Video uses, made at the first preload or play. Bytes and data: URLs play from a temporary file that is removed as the player closes.
  • The load is polled every 100 ms for at most a minute.
  • The position comes from a periodic time observer, with the seeks and laps counted for the core.
  • The end comes from AVPlayerItemDidPlayToEndTimeNotification. A looping player seeks to the start there and counts a lap.
  • A stop that no command asked for (an interruption, a lost route) is reported as a system pause as soon as AVPlayerRateDidChangeNotification (macOS 12, iOS 15) says the rate went to 0, and also when the time observer sees it, which covers macOS 11.
  • On iOS, an audio session interruption (AVAudioSessionInterruptionNotification) pauses the players as it begins: they report AudioPaused, stop reporting progress and let their session level go. They stay paused when the interruption ends, even with ShouldResume, because the core has them paused by the system and Play resumes them, as after the app goes to the background. The design now says this.
  • A seek made before the load is applied at the load. Seeks are in nanoseconds, so a load lands where it was asked.
  • A failed player loads again at the next Play or Preload, as the design says, and plays on from the position it last reported, which the core keeps, or from the start after an end.
• Errors. A path that is not UTF-8 or contains a NUL fails with NotSupported before it reaches NSURL, and so does a local source that cannot be read. http(s) sources fail with Network, and sources that cannot be decoded fail with Decode.
• Sessions. On iOS, sounds hold the shared session (internal/audiosession) as ambient while they play, and players hold the level their AudioSession asks for while they play. iOS pauses players as the app enters the background.
  • No deactivation on the main thread. On the iPhone XR, every Pause and every background pause blocked the main thread for 593 to 634 ms, and iOS showed its hang banner: the last holder letting go called setActive:NO with NotifyOthersOnDeactivation at once, on the main thread, while the AVPlayer's output was still running down, and AudioPaused waited for it because the release comes before the report. The last release now only arranges the deactivation. It runs one second later on a goroutine of its own, inside an autorelease pool. A hold that comes first cancels it: with the same level it makes no session call at all (a play right after a pause), and with another level it moves the session from the level it still has. A hold that comes while the deactivation runs waits for it under the holders' mutex and then activates the session again, so a deactivation and an activation never overlap, and a new level is still held before the old one goes.
  • A playing video keeps the session. A playing Video whose AudioSessionFunc switched to AudioSessionDefault stalled about 0.8 s later with a ~620 ms hang, also on main: the default session held nothing, so releasing the old level deactivated the session under the playing AVPlayer. A new lowest level, Kept, holds the session active in the category from before the first holder (the app's own) for a video that plays with the default session, so Playback -> Default -> Playback moves the session across without deactivating it.
  • A video's system stop. A video's session followed only its Paused, so an interruption, a lost route or a pause from the controls kept it held at its level until the next command, which is the gap fixed for players. The video now observes AVPlayerRateDidChangeNotification and lets the session go when the rate goes to 0. It holds the session again if the controls play the video.
• Availability. macOS and iOS report Env.CanPlayAudio. tvOS does not, because the design leaves it out of the first stages.
• Docs. The design (now including Kept and the deferred deactivation), native-views.md, the roadmap row, ARCHITECTURE.md and the services subject now say that macOS and iOS play audio. CHANGELOG.md is not changed, as it is frozen until 0.1.0.

This replaces the earlier AVAudioPlayer branch, which was written against an API that #84 no longer has. The earlier review findings either no longer apply to this design or are fixed here:

• The end now comes from an AVPlayerItem notification, not a missing AVAudioPlayer one.
• Positions go back only with a seek or a loop, through the seek and lap counters.
• Non-UTF-8 and NUL paths are refused before NSURL.
• Decoding runs inside autorelease pools.
• Sessions are held through internal/audiosession.
• Errors are reported under the design's codes.
• Opening and decoding run off the main thread and the engine goroutine.
• New tests drive AVFoundation directly.

Validation

• go test in internal/platform/apple on macOS: 14 audio tests through real AVFoundation, with the main run loop running. They cover decoding, cuts, quotas, the voice limit across two formats, a player's end and replay, loops, seeks before and after the load, pauses, failures, and a preload and a play that load a failed player again from where it played. M4A (AAC) and FLAC files that afconvert encodes at test time are decoded as sounds and played to their end; these are skipped where afconvert is missing. Undecodable M4A, FLAC and MP3 files fail with Decode, and temporary files are removed. Two of the tests are new:
  • An interruption posted as iOS posts it pauses a playing player at once, its AVPlayer too, with no session held and no more progress. The interruption's end with ShouldResume leaves it paused, and Play resumes it.
  • A player stopped with its time observer removed is still reported paused at once, from the rate change.
  • Both new tests fail without the fix.
• go test in internal/platform/macos: an app that loads and plays a sound and plays a player to its end through the window's engine.
• go test in audio, core and internal/audiosession, and go test -run 'Layer|Skills|Doc' . from the root. internal/audiosession's tests (also run with -race -count=3) record the session's moves. They check that the last release moves nothing on its caller's thread, that a hold of the same level before the deactivation moves nothing, that a hold of another level moves across from the level the session still has, and that a hold that comes while the deactivation runs waits for it and then activates the session again.
• go vet on macOS (arm64 and amd64). An iOS (arm64, cgo) vet and build of internal/audiosession, microphone, internal/platform/apple and internal/platform/ios. Builds of internal/audiosession and microphone for Linux, Windows and Android.

Device checks

On the Mac (output muted, volume left as it was), the current head passed a probe app: Pause reported AudioPaused at once, with the main thread's longest stall at 0.4 ms. MP3, M4A and FLAC each played to their end once. A loop kept playing with no end. A file that appeared late failed with NotSupported, then loaded with Preload and played after a seek to its end. A Video played across Default, Playback and MixWithOthers.

Earlier heads passed these on the iPhone XR: a sound while Music plays (Music keeps playing, and the silent switch silences the sound); AudioSessionPlayback pausing other apps' audio and AudioSessionMixWithOthers mixing with it, both heard in silent mode; a sound during a Video, microphone.Capture and StartSpeech stopping none of them; going to the background pausing players; MP3, M4A and FLAC through a player; and on the Mac, open and close cycles leaving no audio device open.

Needed on the iPhone XR for this head, which has not run there yet:

• Pause and the background pause: no hang banner, AudioPaused at once, and the session back to its earlier category about a second later.
• Music started from Control Center pauses a Playback player at once and releases its level. When Music pauses again, the player stays paused, and Play resumes it.
• A playing Video switched Playback -> Default -> Playback keeps playing with no stall and no hang.
• A Video interrupted by Music lets its session go.
• The earlier list again: end once for each format, seek, loop, a broken file, a late file with Preload, and MixWithOthers vs Playback.