Skip to content

Commit

Permalink
Disable subtitle rendering
Browse files Browse the repository at this point in the history 
Currently causes crashes
  • Loading branch information
@dotMorten
dotMorten committed Jul 10, 2022
1 parent 56f35a8 commit 4ad8524
Showing 1 changed file with 16 additions and 10 deletions.
26 changes: 16 additions & 10 deletions src/WinUIEx/MediaPlayerPresenter.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,22 +83,28 @@ private unsafe void MediaPlayer_VideoFrameAvailable(MediaPlayer sender, object a
Windows.Win32.PInvoke.CreateDirect3D11SurfaceFromDXGISurface(surface, out var isurface);
var d3dSurface = WinRT.MarshalInterface<Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface>.FromAbi(Marshal.GetIUnknownForObject(isurface));
MediaPlayer.CopyFrameToVideoSurface(d3dSurface);
if (hassubtitles) // TODO: This'll only render on one frame until next text. Crash if repeating without this flag getting set.
{
hassubtitles = false;
MediaPlayer.RenderSubtitlesToSurface(d3dSurface);
}
Windows.Win32.Graphics.Dxgi.DXGI_PRESENT_PARAMETERS presentParam = new Windows.Win32.Graphics.Dxgi.DXGI_PRESENT_PARAMETERS(); // { 0, nullptr, nullptr, nullptr };
presentParam.DirtyRectsCount = 0;
m_swapchain.Present1(1, 0, &presentParam);
});
}

private bool hassubtitles = false;

private unsafe void MediaPlayer_SubtitleFrameChanged(MediaPlayer sender, object args)
{
hassubtitles = true;
if (m_swapchain is null)
return;
swapchainPanel?.DispatcherQueue?.TryEnqueue(Microsoft.UI.Dispatching.DispatcherQueuePriority.Normal, () =>
{
Guid g = IID_IDXGISurface;
m_swapchain.GetBuffer(0, &g, out var surfaceobj);
var surface = surfaceobj.As<Windows.Win32.Graphics.Dxgi.IDXGISurface>();
Windows.Win32.PInvoke.CreateDirect3D11SurfaceFromDXGISurface(surface, out var isurface);
var d3dSurface = WinRT.MarshalInterface<Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface>.FromAbi(Marshal.GetIUnknownForObject(isurface));
MediaPlayer.RenderSubtitlesToSurface(d3dSurface);
Windows.Win32.Graphics.Dxgi.DXGI_PRESENT_PARAMETERS presentParam = new Windows.Win32.Graphics.Dxgi.DXGI_PRESENT_PARAMETERS(); // { 0, nullptr, nullptr, nullptr };
presentParam.DirtyRectsCount = 0;
m_swapchain.Present1(1, 0, &presentParam);
});
}

private unsafe void CreateSwapChain()
Expand Down Expand Up @@ -199,7 +205,7 @@ private void OnMediaPlayerPropertyChanged(DependencyPropertyChangedEventArgs e)
{
if(e.OldValue is MediaPlayer oldPlayer)
{
oldPlayer.SubtitleFrameChanged -= MediaPlayer_SubtitleFrameChanged;
//oldPlayer.SubtitleFrameChanged -= MediaPlayer_SubtitleFrameChanged;
oldPlayer.VideoFrameAvailable -= MediaPlayer_VideoFrameAvailable;
oldPlayer.Dispose();
}
Expand All @@ -209,7 +215,7 @@ private void OnMediaPlayerPropertyChanged(DependencyPropertyChangedEventArgs e)
newPlayer.SetSurfaceSize(new Windows.Foundation.Size(ActualWidth, ActualHeight));
newPlayer.IsVideoFrameServerEnabled = true;
newPlayer.VideoFrameAvailable += MediaPlayer_VideoFrameAvailable;
newPlayer.SubtitleFrameChanged += MediaPlayer_SubtitleFrameChanged;
//newPlayer.SubtitleFrameChanged += MediaPlayer_SubtitleFrameChanged;
}
}

Expand Down

1 comment on commit 4ad8524

@softworkz
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dotMorten,

we at FFmpegInteropX are wondering about the intended use of the RenderSubtitlesToSurface() method as well, also having seen "crashes",

I haven't worked on it myself, just heard the reports saying it's crashing. Recently I re-read the docs and I had a certain suspicion. Yet it didn't match with the symptoms of "always crashing" and I abandoned the idea.
Now I found this commit of yours which says:

this'll only render on one frame until next text. Crash if repeating without this flag getting set.

That's quite different from "always crashing" and actually aligns with what I had thought earlier. There is a pragraph in the docs which caught my attention:

For the overload with a target rectagle:

image

reference: https://learn.microsoft.com/en-us/uwp/api/windows.media.playback.mediaplayer.rendersubtitlestosurface?view=winrt-22621

=> So why should this method be "less efficient" than the other overload? This method renders to a constrained area only and the other one renders to the full-size frame. How can it be "less efficient"?

There are two more hints:

  • Full-Frame Overload has this text: if the method returns false, then no subtitles were rendered. In this case you may decide to hide the subtitle render surface in your UI.
  • Rectangle Overload also says: but it allows you to use the same surface for rendering video and subtitles rather than requiring a separate surface for subtitles.

Rendering onto the same surface as the video frames comes with the specific implication that you need to re-draw the subtitle again and again for every single video frame that is shown, because each video frame overwrites everything from before and so you need to redraw the subtitle text for each frame. To make this more economic (just think of 4k video at 60fps), you can/must constrain the rendering to a certain area (no idea how to determine it actually).
And - if I'm right - that would be the explanation why they say that it's less efficient: even though the area is constrained, you need to do it for every video frame.

I suspect that the other overload is meant to be used in a very different way: you create an additional (transparent) surface on top of the video frame and at the same size. When there's a subtitle graphic to display, you call RenderSubtitlesToSurface() to paint it onto your subtitle surface. As this is a separate surface, it doesn't get overwritten by the video frames.

=> In turn, you need to render it just once. And that would be the explanation why you were seeing subsequent calls failing

Conclusion would be: thiis method is not meant to be called "per-frame" but only once on each change and meant to be painted on a separate surface, while the other overload is meant to be painted on each video frame but constrained to a certain region.

Please sign in to comment.