Sharp Media Foundation Interop

Simple Windows Media Foundation interop written in C#. No native dependencies, only managed code.

Platforms

The library is one plain net8.0/net10.0 DLL. Its codecs and devices are reached through interfaces that belong to no platform - IMediaVideoTransform, IMediaAudioTransform, IMediaVideoSource, IAudioOutput, IAudioInput - made by two factories, MediaCodecs and MediaDevices, which pick what the system has:

• Windows 10 1809 or later: Media Foundation's codecs, waveOut/waveIn, Media Foundation's camera capture and DXGI's desktop duplication. Everything Windows-specific is marked [SupportedOSPlatform("windows10.0.17763.0")] and only called there.
• macOS 11 or later: VideoToolbox's and AudioToolbox's codecs, AudioQueue for sound out and in, AVFoundation for the camera, and ScreenCaptureKit (macOS 12.3 or later) for the screen. These are reached through P/Invoke as well - the system's C frameworks directly, AVFoundation and ScreenCaptureKit through the Objective-C runtime - so there is still no native dependency. Everything macOS-specific is marked [SupportedOSPlatform("macos11.0")] ("macos12.3" for screen capture).
• iOS 14 or later: the same VideoToolbox, AudioToolbox, AudioQueue and AVFoundation code as macOS, the app's AVAudioSession moved to Playback, or to PlayAndRecord to record, as it plays and records; ReplayKit (iOS 16 or later) for the app's own screen, which is all iOS lets an app record.
• Android 8 or later: the NDK's C libraries, as the WPF fork's Android head uses them, with no JNI - MediaCodec (libmediandk) for the codecs, AAudio for sound out and in, and Camera2 (libcamera2ndk) for the camera. Android has no screen capture but Java's MediaProjection, whose consent only the app's activity can ask for, so GetScreens() is empty there and CreateScreenCapture throws PlatformNotSupportedException.
• Linux: GStreamer for the codecs, sound out and in, and the camera - the codecs and devices of whatever plugins are installed, a GPU's VA-API or V4L2 ones before the software ones - and, under Wayland, xdg-desktop-portal's ScreenCast for the screen, over libdbus-1, read through PipeWire. These are the libraries the WPF fork's Linux head uses, reached the same way: P/Invoke, no variadic calls. Everything Linux-specific is marked [SupportedOSPlatform("linux")].

The source follows the same split: Codecs, Input, Output and Utils each hold the interfaces and shared code, with the implementations of each platform in their Windows, MacOS, iOS, Android and Linux folders - and what macOS and iOS share in Apple. The namespaces are those four - SharpMediaFoundationInterop.Codecs, .Input, .Output, .Utils - whatever the platform, and MediaDevices is in SharpMediaFoundationInterop itself.

Elsewhere the factories report nothing: MediaCodecs.CanDecode/CanEncode return false, the device lists are empty, and creating one throws PlatformNotSupportedException. Code written against the factories needs no change from one platform to another.

The Windows classes behind the factories (H265Decoder, AACEncoder, WaveOut, ScreenCapture, ...) stay public for what only Media Foundation has: decoding on the GPU through IMediaFoundationVideoTransform, or ICodecAPI properties through CodecProperties. So do the macOS ones: VideoToolboxDecoder, VideoToolboxEncoder, AudioToolboxDecoder, AudioToolboxEncoder, AudioQueueOutput, AudioQueueInput, AVFoundationCapture and ScreenCaptureKitCapture; the iOS one: ReplayKitScreenCapture; the Android ones: AndroidVideoDecoder, AndroidVideoEncoder, AndroidAudioDecoder, AndroidAudioEncoder, AAudioOutput, AAudioInput and AndroidCameraCapture; and the Linux ones: GStreamerVideoDecoder, GStreamerVideoEncoder (whose Element names the encoder it chose), GStreamerAudioDecoder, GStreamerAudioEncoder, GStreamerAudioOutput, GStreamerAudioInput, GStreamerCameraCapture and PortalScreenCapture.

On macOS the user is asked, the first time, whether the app may use the microphone, the camera or the screen; the permission belongs to the app the process runs in (for a console app, the terminal). Until it is given, the camera capture throws UnauthorizedAccessException, as the screen capture does - which asks macOS to show the request - and the microphone records silence. macOS applies a new screen recording permission only once the app is started again.

On iOS, the app's Info.plist must have NSCameraUsageDescription and NSMicrophoneUsageDescription, or iOS ends the app as the camera or microphone is asked for; the user is asked the first time, and Initialize throws UnauthorizedAccessException where they say no. ReplayKit asks the user each time a screen capture starts. On Android, the app must hold the CAMERA and RECORD_AUDIO permissions, which only its Java side can ask the user for; Android lists its sound devices to Java alone, so the lists have the default output and input.

On Linux, install GStreamer and its plugins - on Ubuntu or Debian libgstreamer1.0-0, gstreamer1.0-plugins-base and -good for the devices, and -bad, -ugly and gstreamer1.0-libav for the codecs - and, for the screen, gstreamer1.0-pipewire and an xdg-desktop-portal backend (-gnome, -kde or -wlr). What is not installed shows in CanDecode/CanEncode and the device lists. Under Wayland no app may list or read the screens itself: GetScreens() returns one device, and the user picks a screen in the desktop's dialog each time a capture is initialized, which throws UnauthorizedAccessException where they cancel. A codec of GStreamer's works on threads of its own, so a frame or packet comes out a little after its input went in: ProcessOutput returns what is ready, and draining returns the rest.

Codecs

Supported video codecs (VideoCodec) are:

• H264 (built-in Windows)
• H265 (requires paid HEVC Video Extensions from the Microsoft Store)
• AV1 (requires free AV1 Video Extensions from the Microsoft Store)
• VP9 (requires free VP9 Video Extensions from the Microsoft Store)
• MPEG-2 and MPEG-1, decoding only (requires MPEG-2 Video Extension from the Microsoft Store)
• H263, decoding only (built-in Windows)
• MPEG-4 Part 2, decoding only (built-in Windows; B-frames are skipped, Windows decodes them blank)
• Motion JPEG, decoding only (built-in Windows): QuickTime's jpeg and mjpa, MP4's mp4v of JPEG. NV12 out, or YUY2 of a 4:2:2 picture, as cameras' are; of JPEG's full range and BT.601's matrix, as JFIF has them, which MediaFileSource says. Windows' decoder upsamples 4:2:2 chroma otherwise than libjpeg does, its colours within a few levels of ffmpeg's.
• ProRes - 422, 422 HQ, LT, Proxy, 4444 and 4444 XQ - on Apple devices (every Mac; the iPhones and iPads of ProRes) and on Linux of gstreamer1.0-libav; encoded of VideoEncoderOptions.ProResProfile (Proxy, LT, Standard, HQ - the default -, 4444 or 4444 XQ), whose fixed quality sets the data rate, so a rate control asked for is reported in UnappliedSettings. Windows and Android have no ProRes codec.

Windows has no usable encoder for MPEG-1/2, H263 or MPEG-4 Part 2, and no H261 decoder. Nor has it AC-3 or E-AC-3 decoders: those it once had are gone, and what is left passes the stream on to a receiver over S/PDIF. Whether an extension is installed shows only when the codec is initialized, which then throws NotSupportedException.

macOS decodes every one of these video codecs with VideoToolbox, needing no extensions, with these limits:

• VP9 and AV1 are decoded on the GPU alone: VP9 on Apple silicon and some Intel Macs, AV1 on the M3 and later. MediaCodecs.CanDecode says which.
• MPEG-4 Part 2 of the simple profile alone. A stream of the advanced simple profile - B-VOPs, as XviD and DivX often have - throws NotSupportedException as its headers come in.

macOS encodes H264 and H265 (H265 on Apple silicon, and Intel Macs whose GPU encodes it); it has no VP9 or AV1 encoder, which MediaCodecs.CanEncode reports.

iOS decodes what its VideoToolbox has: H264, H265 and MPEG-4 Part 2's simple profile on every device, and AV1 in hardware on the A17 Pro and later; it has no VP9, MPEG-2 or H263 decoder. It encodes H264 and H265, as macOS does.

Android decodes and encodes what the device's MediaCodec has - H264, H265, VP9 and AV1 on most, often in hardware; MPEG-2 on some - each in whole access units, the NAL units and OBUs gathered as for VideoToolbox. Its software MPEG-4 Part 2 decoder takes B-frames, which are put in order here.

Linux decodes all of them, MPEG-4 Part 2's advanced simple profile included, with GStreamer's decoders, and encodes H264, H265, VP9 and AV1 with the first installed of VA-API's, V4L2's, x264, x265, libvpx, SVT-AV1 and libaom - each of its own properties for the rate control, what it has none of reported in UnappliedSettings.

VideoToolbox is told a stream's configuration before it decodes, so the decoder is made as that configuration comes in band: the parameter sets of H264 and H265, the headers before the first MPEG-4 picture, the first VP9 key frame, the AV1 sequence header. Whatever comes before it is dropped.

Supported audio codecs (AudioCodec) are:

• AAC (built-in Windows). Up to 6 channels; of 4.0 - channel configuration 4 - the channels are front left, right, centre and back centre, which AudioInfo.ChannelMask and CreateAudioOutput's channelMask (0x107) carry to the speakers they are for.
• MP3 (built-in Windows; the encoder at 32000, 44100 or 48000 Hz, or MPEG-2's lower rates, and its own list of bit rates)
• MP2, MPEG-1 and MPEG-2 audio of Layers I and II, decoding only (built-in Windows: Mp2Decoder). MediaFileSource tells it from MP3 by the first frame's header, as an mp4a of MPEG audio does not say its layer.
• FLAC, 16 and 24 bits (built-in Windows)
• ALAC, 16 and 24 bits (built-in Windows; the encoder pads the last frame to 4096 samples with silence)
• Opus (decode only on Windows: built-in, might require a newer version - works in Windows 11 25H2; Windows has no Opus encoder). Windows' decoder takes mono and stereo alone; Opus of more channels - of channel mapping family 1 (5.1, 7.1), 2 (ambisonics) or 255 - is split into its streams by OpusMultistreamDecoder, each decoded as mono or stereo, given the dOps box's payload as Config. Family 1 comes out in WAVE's order, ambisonics in ACN order.
• PCM, uncompressed (decode only, managed: PcmDecoder, the same on every platform). It takes the samples as AudioDecoderOptions.Pcm describes them - 8, 16, 24 or 32 bit integers, 32 or 64 bit floats, either byte order, 8 bit signed or offset binary, as QuickTime's lpcm, twos, sowt, raw , in24, in32, fl32, fl64 and ISO's ipcm and fpcm hold them - and gives little-endian 16 bit PCM of 16 bits or fewer, else 32 bit. MediaFileSource plays such a track, reading a tenth of a second of its frames at a time. G.711 - QuickTime's alaw and ulaw - is decoded by it too, of PcmFormat.Companding, expanded to 16 bit as ITU-T G.711 has it, sample for sample as ffmpeg does.

Windows has decoders of AMR-NB and MPEG-H 3D Audio that cannot be used here: its AMR-NB decoder is of Media Foundation's own pipeline alone - made by its class factory, or asked of a source reader that has made it, it gives no IMFTransform - and its MPEG-H decoder hands out IEC 61937, for a receiver to decode, not PCM. AC-3 and E-AC-3 it has no decoder of either, only their IEC 61937 converters. SharpMP4 reads all of them: their samples pass through.

On macOS, AudioToolbox decodes all five compressed codecs, to the same PCM formats as on Windows; more than two channels come out in WAVE's order, as Windows' decoders give them. It encodes all but MP3, of which macOS has no encoder, taking the same PCM as Windows' encoders. iOS decodes and encodes as macOS does. Android decodes AAC, MP3, Opus and FLAC with MediaCodec - ALAC where the device has a decoder - and encodes AAC, Opus and FLAC; Opus decodes to 48 kHz there, whatever was asked. Linux decodes and encodes all five with GStreamer, of the same formats. PCM is decoded by PcmDecoder everywhere.

Video decoding

Create and initialize the decoder; every codec has the same API. The size is the coded one, as the stream's parameter sets give it:

var options = new VideoDecoderOptions
{
   Width = width,
   Height = height,
   FpsNom = timescale,               // a hint; 0 over 0 where the stream does not say
   FpsDenom = defaultSampleDuration,
};
using (var videoDecoder = MediaCodecs.CreateVideoDecoder(VideoCodec.AV1, options))
{
   videoDecoder.Initialize();
   ...
}

Frames come out as NV12, padded to the size the decoder works in: videoDecoder.Width by videoDecoder.Height, or MediaCodecs.DecoderAlignment(codec) before there is a decoder. On macOS that alignment is 2 for every codec; the picture is in the top left of each frame. Create a buffer to hold the decoded frame:

var nv12Buffer = new byte[videoDecoder.OutputSize];

Pass the frame to the decoder and get the output. A decoder hands frames out in display order, each with the timestamp of the input it came from. On macOS, where VideoToolbox hands them out in decode order, they are put in order by those timestamps, so give each frame its own:

IEnumerable<byte[]> units = ...; // get a list of NALU/OBU from your video source
foreach (var unit in units)
{
   if (videoDecoder.ProcessInput(unit, timestamp))
   {
      while (videoDecoder.ProcessOutput(ref nv12Buffer, out _, out long frameTime))
      {
         // nv12Buffer holds the decoded nv12 frame
         ...
      }
   }
}

At the end of the stream, drain the frames the decoder still holds, and read them out before taking input again - audio decoders and encoders alike:

videoDecoder.BeginDrain();
while (videoDecoder.ProcessOutput(ref nv12Buffer, out _, out long frameTime))
{
   ...
}
videoDecoder.EndDrain();

On Windows, convert NV12 to RGB for display with Media Foundation's color converter:

using(var nv12Decoder = new ColorConverter(PInvoke.MFVideoFormat_NV12, PInvoke.MFVideoFormat_RGB24, width, height))
{
    nv12Decoder.Initialize();
    
    byte[] rgbBuffer = new byte[nv12Decoder.OutputSize];
    if(nv12Decoder.ProcessInput(nv12Buffer, 0))
    {
       if (nv12Decoder.ProcessOutput(ref rgbBuffer, out _))
       {
          // rgbBuffer holds the decoded RGB frame
          ...
       }
    }
}

Video encoding

Create and initialize the encoder; every codec has the same API:

var options = new VideoEncoderOptions
{
   Width = width,
   Height = height,
   FpsNom = timescale,
   FpsDenom = defaultSampleDuration,
   Bitrate = 8000000,
   RateControl = RateControlMode.ConstantQp, // or Default, ConstantBitrate, VariableBitrate, Quality
   Qp = 24,
   KeyFrameInterval = 60,                    // 0 leaves it to the encoder
};
using (var videoEncoder = MediaCodecs.CreateVideoEncoder(VideoCodec.H265, options))
{
   videoEncoder.Initialize();

   // support for each setting varies by codec and vendor: a setting the encoder did not take is reported, not thrown
   foreach (var setting in videoEncoder.UnappliedSettings)
      Console.WriteLine(setting);
   ...
}

Optionally, on Windows, convert RGB to nv12:

using(var nv12Encoder = new ColorConverter(PInvoke.MFVideoFormat_RGB24, PInvoke.MFVideoFormat_NV12, width, height))
{
    nv12Encoder.Initialize();
    
    byte[] nv12Buffer = new byte[nv12Encoder.OutputSize];
    if(nv12Encoder.ProcessInput(rgbBuffer, 0))
    {
       if (nv12Encoder.ProcessOutput(ref nv12Buffer, out _))
       {
          // nv12Buffer holds the encoded nv12 frame
          ...
       }
    }
}

Each access unit comes out in Annex B, with its start codes; a key frame has the parameter sets in front of it. On macOS the encoder takes NV12 of exactly Width by Height, encodes no B-frames, and reports a setting it has no equivalent of - Threads - in UnappliedSettings.

Create a buffer to hold the encoded NALU/OBU:

var au = new byte[videoEncoder.OutputSize];

Read encoded NALU/OBU:

if (videoEncoder.ProcessInput(nv12Buffer, timestamp))
{
   while (videoEncoder.ProcessOutput(ref au, out var length))
   {
      // au buffer holds the encoded access unit
      var parsedAU = AnnexBUtils.ParseNalu(au, length);
      foreach (var unit in parsedAU)
      {
         // here you can access the individual NALU/OBU
      }
   }
}

Audio decoding

Create and initialize the decoder. AAC needs its AudioSpecificConfig, which the MP4 file's esds box holds; FLAC its metadata blocks, each with its header, as the dfLa box holds them; ALAC its ALACSpecificConfig, the 24 byte magic cookie. The channels, rate and bits of FLAC and ALAC are read of those. MP3 needs no config, and Opus of mono or stereo none - more channels need its dOps payload - but give Opus its pre-skip, the samples the decoder needs to warm up, so that they are dropped:

var options = new AudioDecoderOptions
{
   Channels = channelCount,
   SampleRate = samplingRate,
   Config = aacTrack.AudioSpecificConfig.ToBytes(), // AAC
   SkipSamples = opusTrack.PreSkip,                 // Opus
};
using (var audioDecoder = MediaCodecs.CreateAudioDecoder(AudioCodec.AAC, options))
{
   audioDecoder.Initialize();
   ...
}

AAC and MP3 decode to 16 bit PCM, Opus to 32 bit float, FLAC and ALAC to PCM of the stream's own bits. Create a buffer to hold the decoded frame:

var pcmBuffer = new byte[audioDecoder.OutputSize];

Pass the frame to the decoder and get the output:

IEnumerable<byte[]> audioFrames = ...; // get a list of frames from your audio source
foreach (var audioFrame in audioFrames)
{
   if (audioDecoder.ProcessInput(audioFrame, 0))
   {
      while (audioDecoder.ProcessOutput(ref pcmBuffer, out var length))
      {
         // pcmBuffer holds the decoded audio
         ...
      }
   }
}

Audio encoding

Create and initialize the encoder. Windows' AAC encoder takes 44100 or 48000 Hz, 16 bit PCM of 1, 2 or 6 (5.1) channels, 1024 samples a frame, encoded at the rate it offers nearest 128 kbit/s a channel - not 3, 4 or 5 channels, which it does not encode, nor 8, whose 7.1 it encodes as front-wide, not as players play it; FLAC and ALAC 16 or 24 bit PCM (BitsPerSample), a frame of 4096 samples; MP3 16 bit PCM, at the Bitrate nearest its list. The config for the MP4 file is Config: AAC's AudioSpecificConfig, FLAC's metadata blocks - complete, with the total samples and MD5, once the encoder is drained - or ALAC's magic cookie:

var options = new AudioEncoderOptions { Channels = channelCount, SampleRate = samplingRate };
using (var audioEncoder = MediaCodecs.CreateAudioEncoder(AudioCodec.AAC, options))
{
   audioEncoder.Initialize();

   // the AudioSpecificConfig, for the MP4 file's esds box
   byte[] audioSpecificConfig = audioEncoder.Config;
   ...
}

Create a buffer to hold the encoded frame:

var aacBuffer = new byte[audioEncoder.OutputSize];

Pass the frame to the encoder and get the output:

IEnumerable<byte[]> pcmFrames = ...; // get a list of frames from your audio source
foreach (var pcmFrame in pcmFrames)
{
   if (audioEncoder.ProcessInput(pcmFrame, 0))
   {
      while (audioEncoder.ProcessOutput(ref aacBuffer, out var length))
      {
         // aacBuffer holds the encoded AAC audio
         ...
      }
   }
}

Devices

MediaDevices lists the system's devices with GetScreens(), GetCameras(), GetAudioOutputs() and GetAudioInputs(), each a MediaDevice with an Id and a Name, and creates them. Leave the device null for the system's default.

Screen capture

Captures a screen as 32 bit BGRA (on Windows, with the DuplicateOutput1 API; on macOS, with ScreenCaptureKit, at the display's size in pixels). Rows are bottom-up by default, as Media Foundation's RGB formats are, or top-down, as a bitmap is:

var screens = MediaDevices.GetScreens(); // the primary first
using (var screenCapture = MediaDevices.CreateScreenCapture(screens.First(), topDown: false))
{
   screenCapture.Initialize();
   ...
}

Create a buffer to hold the sample:

var buffer = new byte[screenCapture.OutputSize];

Read the sample. A frame is returned only when the screen has changed:

if (screenCapture.ReadSample(buffer, out var timestamp))
{
   // captured screen is in the buffer
}

Camera capture

Captures a camera (on Windows, any Media Foundation video capture device such as a webcam; on macOS, with AVFoundation, as NV12) in the best format it offers, which OutputFormat reports:

var cameras = MediaDevices.GetCameras();
using (var camera = MediaDevices.CreateCameraCapture(cameras.First()))
{
   camera.Initialize();
   ...
}

Create a buffer to hold the sample:

var buffer = new byte[camera.OutputSize];

Read the sample:

if (camera.ReadSample(buffer, out var timestamp))
{
   // captured image is in the buffer
}

Audio input

Records PCM (on Windows, with the waveIn API; on macOS, with AudioQueue):

using (var audioInput = MediaDevices.CreateAudioInput(sampleRate, channelCount, bitsPerSample))
{
    ...
}

Subscribe the FrameReceived callback:

audioInput.FrameReceived += (sender, e) => 
{
    byte[] pcmData = e.Data;
    ...
};

Recording starts right after you call Initialize, and stops with Reset or Dispose:

audioInput.Initialize();

Audio output

Plays PCM (on Windows, with the waveOut API; on macOS, with AudioQueue):

using (var audioOutput = MediaDevices.CreateAudioOutput(sampleRate, channelCount, bitsPerSample))
{
    audioOutput.Initialize();
    ...
}

To play audio, enqueue it. QueuedFrames is the number of buffers still to play, GetPosition() the bytes played so far, and Pause, Resume and Reset control playback:

byte[] pcmSample = ...
audioOutput.Enqueue(pcmSample, (uint)pcmSample.Length);

More than two channels are played on the speakers they are for, in WAVE's order (SpeakerLayout.ChannelMask); Windows mixes them down to fewer speakers. MediaDevices.GetSpeakerChannels() is how many speakers the default output has, as Windows' speaker configuration says.

AmbisonicRenderer plays a sound field - first-order ambisonics or higher, ACN and SN3D, with or without two channels of head-locked stereo after it - on so many speakers, turned as the listener looks (Turn(yaw, pitch)). The controls do this for a track that is a sound field (AudioInfo.IsAmbisonic: Opus of channel mapping family 2, or a track with Google's SA3D box), turning the sound as a 360° video's view is turned in GpuMediaControl. Of a higher order, its first order is played on speakers.

AmbisonicRenderer.Binaural(channels, sampleRate, bitsPerSample) hears the sound field on headphones instead: turned as the listener looks, up to its third order, then convolved with Google VR's HRIRs of the SADIE measurements (Apache 2.0, as Omnitone carries them; see Output/Resources/NOTICE.txt) into the left and right ears. The controls do this where Binaural is true, or, where it is null - the default - where Windows says the default output is headphones or a headset (MediaDevices.AreHeadphones()); changed while playing, the sound goes on through the other. The overlay shows a Headphones button (or H) for a sound field, lit while it is heard binaurally: Windows cannot always tell headphones - in a virtual machine, or headphones in a jack the driver does not report - so the listener can switch.

Video control

MediaControl (drawn into a bitmap) and GpuMediaControl (drawn with Direct3D) play a video source, with a bar of controls over the bottom of the video. The bar shows as the mouse moves over the video and fades out after ControlsHideDelay, or as the mouse leaves; it stays while the video is paused. Set AutoHideControls="False" to always show it, or ShowControls="False" to never show it.

Set IsOverlayEnabled="False" to remove the overlay completely: no bar, and none of its keyboard shortcuts, for an application that draws its own controls. Everything the overlay does is also a public method on the control: Play(), Pause(), TogglePlayPause(), Rewind(), FastForward(), SetRate(rate), Seek(position), SeekBy(offset), StepBackward(), StepForward(), TogglePositionFormat(), ToggleRepeat(), ToggleBinaural(), NextEyeView(), NextSubtitleTrack() and ResetView(), with PositionFormat, Looping, Mute, EyeView and SubtitleTrack as properties. The bar's Repeat button (or R) turns Looping on and off; it is off by default, and a video then stops at its end.

The position is shown as a time or, with PositionFormat="Frames", as the frame shown of how many; click it (or press T) to switch. The bar's step buttons (or Left and Right) go back or forward by a second in time mode, and by exactly one frame in frame mode, pausing there. A source gives its frames' times through ISeekableVideoSource.FrameCount, FrameAt and TimeOfFrame, as MediaFileSource does from the file's sample table; for one that does not, a frame is taken from the frame rate. Subtitles still show without the overlay; set SubtitleTrack="-1" to hide them.

The bar is part of the control's template, and can be restyled or replaced from XAML. Its buttons send WPF's MediaCommands (TogglePlayPause, Play, Pause, Rewind, FastForward, MuteVolume), and the control's read-only properties say what to show: IsPlaying, CanSeek, PlaybackRate, Position, Duration, TimeText and AreControlsVisible. The template is also put in the visual state ControlsVisible or ControlsHidden. Template parts, all optional: PART_Image (the frames are drawn on it), PART_Seek (a Slider that seeks) and PART_ControlsBar (kept shown while the mouse is over it).

<ctrl:MediaControl Source="{Binding Source}">
    <ctrl:MediaControl.Template>
        <ControlTemplate TargetType="ctrl:MediaControl">
            <Grid Background="Transparent">
                <Image x:Name="PART_Image" />
                <StackPanel x:Name="Bar" Orientation="Horizontal" VerticalAlignment="Bottom" Background="#80000000">
                    <Button x:Name="PlayPause" Command="MediaCommands.TogglePlayPause" Content="Play" Focusable="False" />
                    <TextBlock Foreground="White" Margin="8,0" Text="{TemplateBinding TimeText}" />
                </StackPanel>
            </Grid>
            <ControlTemplate.Triggers>
                <Trigger Property="IsPlaying" Value="True">
                    <Setter TargetName="PlayPause" Property="Content" Value="Pause" />
                </Trigger>
                <Trigger Property="AreControlsVisible" Value="False">
                    <Setter TargetName="Bar" Property="Visibility" Value="Hidden" />
                </Trigger>
            </ControlTemplate.Triggers>
        </ControlTemplate>
    </ctrl:MediaControl.Template>
</ctrl:MediaControl>

To restyle only the default template's buttons, base a style on {StaticResource {x:Static ctrl:MediaControlBase.ButtonStyleKey}}. The seek bar - a thin track and a round thumb with a dot, as Windows 11's Media Player draws it - is MediaControlBase.SeekSliderStyleKey; its Foreground is the accent color of the part played and the dot, and its Background the rest of the track.

Subtitles

Both controls show subtitles: of a file, its subtitle tracks (3GPP timed text, WebVTT, TTML), and SubRip or WebVTT files beside it named after it (movie.srt, movie.en.srt, movie.de.forced.vtt). SubtitleTracks lists them, and SubtitleTrack chooses one, -1 (the default) for none; the bar's CC button (or C) goes through them. A forced track is shown even with subtitles off. Where both eyes of a stereo video are shown side by side, the subtitle is shown over each. A source of its own gives subtitles by implementing ISubtitleSource, and a data template of the key MediaControlBase.SubtitleTemplateKey draws them otherwise.

Stereo and VR video

A stereo video's frame holds both eyes, side by side or top and bottom. EyeView shows both as the frame has them, or the left or right eye alone; the bar gets an eye button for it (or press E). MediaFileSource reads the layout from the file's st3d box, and StereoLayout overrides it for a file that does not say.

Two separate videos, one per eye, play as one stereo video side by side through StereoMediaSource, which decodes each eye on its own thread and pairs the frames by their times:

var source = new StereoMediaSource(new MediaFileSource("left.mp4"), new MediaFileSource("right.mp4"));

A 180 or 360 degree video (equirectangular, as a VR180 camera writes it, read from the file's sv3d box, or forced with Projection) is shown by GpuMediaControl as a view into the sphere, drawn on the GPU: drag to look around, use the wheel to zoom, and double-click to look straight ahead again. Yaw, Pitch and FieldOfView hold the view and can be bound to. Of a stereo VR video, EyeView shows one eye's view, or both side by side. MediaControl shows such a video flat.

Samples

The console samples run on Windows, macOS and Linux alike; the WPF player is Windows' alone. The recorders take how long to record, in seconds, as their first argument - dotnet run -- 10 - or record until a key is pressed.

SharpMediaPlayer

Sample WPF video player that supports RTSP real-time video, MP4 files, AVIF/HEIC/HEIF images, screen capture and Media Foundation devices such as a webcam.

SharpMediaTranscoder

Demonstrates how to transcode H264 fragmented MP4 into H265 fragmented MP4.

SharpScreenCapture

Demonstrates how to capture the screen and encode it into H265 MP4.

SharpWavePlayer

Shows how to play AAC audio from a MP4 file.

SharpWebcamRecorder

Shows how to capture video from a webcam and record it into H265 MP4.