[WPF] How to overlay WPF elements on the video renderer with FlyleafME?

[umlx5h]

Hi, thank you for all your support.

What I want to achieve

I am currently extending FlyleafPlayer (WPF Control) (WPF) to create a video player.

FlyleafPlayer uses FlyleafME with MainWindow as standAlone, making it a borderless video player.

Flyleaf/Samples/FlyleafPlayer (WPF Control) (WPF)/MainWindow.xaml.cs

Line 56 in 5de9b79

FlyleafME = new FlyleafME(this)

Subtitles and other WPF elements are displayed on top of the video as overlay using data binding.

Flyleaf/Samples/FlyleafPlayer (WPF Control) (WPF)/MainWindow.xaml

Line 235 in 5de9b79

Text="{Binding Player.Subtitles.SubsText}">

I would like to add a sidebar to this sample app like PotPlayer below, at the same time overlay WPF elements like TextBlock on top of the video.

I would like this sidebar to appear outside of the video like above, not overlaid on it. However, I want it to appear in the same window, not in a separate window.

Question

To display the sidebar outside the video as above, instead of initializing FlyleafME with standAlone=true,
I assumed that I need to use FlyleafME as a WPF element.

Sample code is provided below with reference to FlyleafMultiPlayer.

MainWindow.xaml

<Window x:Class="SamplePlayer.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:SamplePlayer"
        xmlns:fl="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib"
        xmlns:flwpf="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib.Controls.WPF" 
        mc:Ignorable="d"
        Title="SamplePlayer" Height="450" Width="800"
        d:DataContext="{d:DesignInstance local:MainWindow}">

    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="*" />
            <ColumnDefinition Width="Auto" />
        </Grid.ColumnDefinitions>

        <flwpf:FlyleafME
            Grid.Column="0"
            x:Name="FlyleafME"
            Player="{Binding Player, Mode=TwoWay}">

            <!-- NOTE: I want to display this element on top of the video. But it is not displayed. -->
            <TextBlock 
                Margin="50 100 0 0"
                FontSize="30" 
                Text="Hello Flyleaf Overlay! (e.g. Subtitle)"
                Foreground="Aquamarine"/>
        </flwpf:FlyleafME>

        <!-- Sidebar -->
        <StackPanel Grid.Column="1">
            <TextBlock Text="Sidebar" FontSize="20" Background="DarkGray" />
            <ListView Width="200"
                      FontSize="20">
                <ListViewItem>
                    aaaaaaaaaaaaaa
                </ListViewItem>
                <ListViewItem>
                    bbbbbbbbbbbbbb
                </ListViewItem>
                <ListViewItem>
                    ccccccccccccccc
                </ListViewItem>
            </ListView>
        </StackPanel>
    </Grid>
</Window>

MainWindow.xaml.cs (code-behind)

using FlyleafLib;
using FlyleafLib.Controls.WPF;
using FlyleafLib.MediaPlayer;
using System.ComponentModel;
using System.IO;
using System.Windows;

namespace SamplePlayer;

public partial class MainWindow : Window
{
    public Player Player { get; set; }

    public MainWindow()
    {
        Engine.Start(new EngineConfig()
        {
#if DEBUG
            LogOutput = ":debug",
            LogLevel = LogLevel.Debug,
            FFmpegLogLevel = FFmpegLogLevel.Warning,
#endif
            PluginsPath = ":Plugins",
            FFmpegPath = ":FFmpeg",
        });

        Config config = new();

        Player = new Player(config);
        DataContext = this;

        InitializeComponent();

        Closing += (o, e) =>
        {
            while (Engine.Players.Count > 0)
            {
                Engine.Players[0].Dispose();
            }
        };
    }
}

I want to put a WPF element on top of a video, but the TextBlock is not shown.

It showed up if I used FlyleafHost instead of FlyleafME like below, but I would prefer to use FlyleafME if possible.

        <!-- NOTE: Changed from FlyleafMe -->
        <fl:FlyleafHost
            Grid.Column="0"
            x:Name="FlyleafME"
            Player="{Binding Player, Mode=TwoWay}">
			
           <!-- NOTE: This element is displayed -->
            <TextBlock 
                Margin="50 100 0 0"
                FontSize="30" 
                Text="Hello Flyleaf Overlay! (e.g. Subtitle)"
                Foreground="Aquamarine"/>
        </fl:FlyleafHost>

The overlay is displayed as follows.

So my question is
Can I use FlyleafME as a WPF element (no StandAlone), but overlay a WPF element on top of the video?

Also, could you please let me know if there are any good ways to achieve the above requirements?

[SuRGeoNix]

Hi @umlx5h, this is tricky indeed.
Did you check the Tests/WpfFlyleafHost? There is the FlyleafSharedOverlay wpf control that I think you can achieve this.

[umlx5h]

@SuRGeoNix
Thanks for your reply!
I didn't check the sample yet, so I'm checking now, but it doesn't work as I don't know how to use it much yet.

I tried the following refer to this sample, but the video does not appear. Is this the correct way to use it?

<Window x:Class="SamplePlayer.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:SubPlayer"
        xmlns:fl="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib"
        xmlns:flwpf="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib.Controls.WPF" 
        mc:Ignorable="d"
        Title="SamplePlayer" Height="450" Width="800"
        d:DataContext="{d:DesignInstance local:MainWindow}">

    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="*" />
            <ColumnDefinition Width="Auto" />
        </Grid.ColumnDefinitions>

        <fl:FlyleafSharedOverlay Grid.Column="0">
            <fl:FlyleafSharedOverlay.DetachedContent>
                <flwpf:FlyleafME
                        x:Name="FlyleafME"
                        Player="{Binding Player, Mode=TwoWay}" />
            </fl:FlyleafSharedOverlay.DetachedContent>
            <Grid Background="#A0000000"  Margin="100">
                <TextBlock 
                    Margin="50 100 0 0"
                    FontSize="30" 
                    Text="Hello Flyleaf Overlay! (e.g. Subtitle)"
                    Foreground="Aquamarine"/>
            </Grid>
        </fl:FlyleafSharedOverlay>

        <!-- Sidebar -->
        <StackPanel Grid.Column="1">
            <TextBlock Text="Sidebar" FontSize="20" Background="DarkGray" />
            <ListView Width="300"
                      FontSize="20">
                <ListViewItem>
                    aaaaaaaaaaaaaa
                </ListViewItem>
                <ListViewItem>
                    bbbbbbbbbbbbbb
                </ListViewItem>
                <ListViewItem>
                    ccccccccccccccc
                </ListViewItem>
            </ListView>
        </StackPanel>
    </Grid>
</Window>

I can achieve the requirements with FlyleafHost, but can I make it work by combining FlyleafME and FlyleafSharedOverlay ?
I'll read the code more, but If possible, it would be very helpful if you could provide sample code with FlyleafME.

[SuRGeoNix]

Hey @umlx5h, check the FlyleafMEwithSideBar sample that I've included there (just change App.xaml startupuri to FlyleafMEwithSideBar.xaml)

I misunderstood your requirement, hope now it's what you want.
(I'm afraid that you might have issues with the keybindings)

[umlx5h]

@SuRGeoNix
Thank you very much for the sample code.
I checked it and it seems to behave differently than the PotPlayer one above because the sidebar is displayed above the video.
This is also good because it is transparent, but it looks more like a transparent window than a sidebar.

I would like to place it on the side, independent of the video viewport, but is this possible?

[SuRGeoNix]

I'm really confused, that can be simple achieved with this then?

<Window x:Class="WpfFlyleafHost.FlyleafMEwithSideBar"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:WpfFlyleafHost"
        xmlns:fl="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib" xmlns:fl1="clr-namespace:FlyleafLib.MediaPlayer;assembly=FlyleafLib"
        xmlns:flwpf="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib.Controls.WPF" 
        xmlns:materialDesign="http://materialdesigninxaml.net/winfx/xaml/themes"
        mc:Ignorable="d"
        Title="FlyleafMEwithSideBar" Height="450" Width="800">

    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition/>
            <ColumnDefinition/>
        </Grid.ColumnDefinitions>

        <flwpf:FlyleafME Player="{Binding Player}"/>
        <!--Playlist SideBar-->
        <StackPanel Grid.Column="1" >
            <TextBlock Text="Sidebar" FontSize="20" Background="DarkGray" />
            <ListView Width="200" FontSize="20">
                <ListViewItem>
                    aaaaaaaaaaaaaa
                </ListViewItem>
                <ListViewItem>
                    bbbbbbbbbbbbbb
                </ListViewItem>
                <ListViewItem>
                    ccccccccccccccc
                </ListViewItem>
            </ListView>
        </StackPanel>
    </Grid>
</Window>

[umlx5h]

I don't know how to display WPF elements on the video viewport.
The following code will not display the TextBlock.
If I change FlyleafME to FlyleafHost, it will be displayed.

I'm trying to display subtitles, etc.

    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="*" />
            <ColumnDefinition Width="Auto" />
        </Grid.ColumnDefinitions>

        <flwpf:FlyleafME Player="{Binding Player}">
            <!-- NOTE: This WPF element is not shown on the video. -->
            <TextBlock 
                Margin="50 100 0 0"
                FontSize="30" 
                Text="Hello Flyleaf Overlay! (e.g. Subtitle)"
                Foreground="Aquamarine"/>
        </flwpf:FlyleafME>
        <!--Playlist SideBar-->
        <StackPanel Grid.Column="1" >
            <TextBlock Text="Sidebar" FontSize="20" Background="DarkGray" />
            <ListView Width="200" FontSize="20">
                <ListViewItem>
                    aaaaaaaaaaaaaa
                </ListViewItem>
                <ListViewItem>
                    bbbbbbbbbbbbbb
                </ListViewItem>
                <ListViewItem>
                    ccccccccccccccc
                </ListViewItem>
            </ListView>
        </StackPanel>
    </Grid>

[SuRGeoNix]

I just saw you an example how you do this with FlyleafME... as it is not the same as FlyleafHost that you just add the content. You need to override the style.

[umlx5h]

Sorry, I didn't read the sample carefully.

I changed the sample to the following and was able to achieve what I wanted to do.
Thanks for the support!

❯ git --no-pager diff  Tests/WpfFlyleafHost/FlyleafMEwithSideBar.xaml
diff --git a/Tests/WpfFlyleafHost/FlyleafMEwithSideBar.xaml b/Tests/WpfFlyleafHost/FlyleafMEwithSideBar.xaml
index 82a65ab..d5db5e2 100644
--- a/Tests/WpfFlyleafHost/FlyleafMEwithSideBar.xaml
+++ b/Tests/WpfFlyleafHost/FlyleafMEwithSideBar.xaml
@@ -76,22 +76,6 @@
                                 FontStyle="{Binding UIConfig.SubsFontStyle}"
                                 Text="{Binding Player.Subtitles.SubsText}">
                                     </flwpf:OutlinedTextBlock>
-
-                                    <!--Playlist SideBar-->
-                                    <StackPanel HorizontalAlignment="Right">
-                                        <TextBlock Text="Sidebar" FontSize="20" Background="DarkGray" />
-                                        <ListView Width="200" FontSize="20" Background="Transparent" TextElement.Foreground="White">
-                                            <ListViewItem>
-                                                aaaaaaaaaaaaaa
-                                            </ListViewItem>
-                                            <ListViewItem>
-                                                bbbbbbbbbbbbbb
-                                            </ListViewItem>
-                                            <ListViewItem>
-                                                ccccccccccccccc
-                                            </ListViewItem>
-                                        </ListView>
-                                    </StackPanel>

                                     <!--Flyleaf Bar-->
                                     <flwpf:FlyleafBar Player="{Binding Player}"/>
@@ -104,5 +88,28 @@
         </Style>
     </Window.Resources>

-    <flwpf:FlyleafME Player="{Binding Player}" Style="{StaticResource FlyleafMECustom}"/>
+    <Grid>
+        <Grid.ColumnDefinitions>
+            <ColumnDefinition Width="*" />
+            <ColumnDefinition Width="Auto" />
+        </Grid.ColumnDefinitions>
+
+        <flwpf:FlyleafME Grid.Column="0" Player="{Binding Player}" Style="{StaticResource FlyleafMECustom}"/>
+
+        <StackPanel Grid.Column="1"  HorizontalAlignment="Right">
+            <TextBlock Text="Sidebar" FontSize="20" Background="DarkGray" />
+            <ListView Width="200" FontSize="20">
+                <ListViewItem>
+                    aaaaaaaaaaaaaa
+                </ListViewItem>
+                <ListViewItem>
+                    bbbbbbbbbbbbbb
+                </ListViewItem>
+                <ListViewItem>
+                    ccccccccccccccc
+                </ListViewItem>
+            </ListView>
+        </StackPanel>
+    </Grid>
+
 </Window>