make it easier to test mouse events

[trebahl]

In automated tests, I would like to be able to create instances of MouseEventArgs, MouseButtonEventArgs, ... with specific coordinates, then use RaiseEvent to test the reaction of my controls to the event.

But this is not possible because MouseEventArgs does not contain coordinates; instead GetPosition has to query the mousedevice for the position. It doesn't look possible to define a fake mousedevice for my purpose. I don't understand why this choice was made: the coordinates are available in the mouse window messages, so it should be possible to extract them in the WndProc and pass them to the mouseeventargs.

I understand a fundamental change of logic is not possible for backwards compatibility reasons, but I can think of two workarounds that wouldn't break bytecode compatibility:

1. Add a publically settable Point field in MouseEventArgs that would be returned by GetPosition if it has been assigned. Pretty ugly, but would be very handy.

    public Point GetPosition(IInputElement relativeTo)
    {
        return GetPositionImpl(relativeTo); // thunking to the virtual function to keep GetPosition non virtual
    }
    protected virtual Point GetPositionImpl(IInputElement relativeTo)
    {
        return MouseDevice.GetPosition(relativeTo);
    }

Then we could create derived classes that would override GetPositionImpl to return what we want. That would be less handy because we'd have to define one derived class for each class derived from MouseEventArgs, and every programmer would have to reinvent the square wheel on their side.

[miloush]

Creating MouseEventArgs is a very limited way of testing a behavior, because a lot of things are happening during mouse events, activation, capture, changing status of related dependency properties through the whole tree etc. It's would be of very limited use to enable this if the event handler would not see effects of all the above.

GetPosition is also more complicated then returning the point from window message. It takes into account all transforms of the elements in the tree.

Could you use the SendInput native API to send messages to the window?

Related: #5743

In terms of WPF, I think a better option would be opening up the input system to allow 3rd party input providers to report input.

[trebahl]

It's would be of very limited use to enable this if the event handler would not see effects of all the above.

I disagree. First, I can explicitly send all the events that would be sent IRL. Second, in my controls I would typically handle only a very specific set of events, the others being ignored so I don't need to send them.

GetPosition is also more complicated then returning the point from window message. It takes into account all transforms of the elements in the tree.

Yes, it needs to be converted to the specific visual. Just like what is done now, except the screen position is queried from the device while it could be fed from the message.

Could you use the SendInput native API to send messages to the window?

I used to do that in windows forms, but it's horribly hackish, requires a window to be displayed, and is generally frail. I thought WPF was supposed to spare us from all low level horrors. Plus I don't even think it would work since the mouse position is queried from the device instead of being read from the message: I would probably need to explicitly set the pointer position. I have done this in the past but it's super frail, it means no one must do anything on the machine while the test is running.

At the moment, the only workaround I see is something like

    protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e)
    {
        OnMouseLeftButtonDownImpl(e.GetPosition(this);
    }
    public void OnMouseLeftButtonDownImpl(Point pos)
    {
        // do the job
    }

Then in tests I can call OnMouseLeftButtonDownImpl with the position I like. That's awkward and only works if I have total control of the class.

[miloush]

Also related: #9243

I can explicitly send all the events

Not any more than you can simulate mouse now (i.e. only using reflection), for example IsMouseOver, IsMouseDirectlyOver etc. are read-only.

my controls

Hence the limited use.

the mouse position is queried from the device instead of being read from the message

Note that the mouse device reports the last position reported by the message.

wpf/src/Microsoft.DotNet.Wpf/src/PresentationCore/System/Windows/Input/MouseDevice.cs

Line 134 in 673c354

ptScreen = PointUtil.ClientToScreen(_lastPosition, activeSource);

wpf/src/Microsoft.DotNet.Wpf/src/PresentationCore/System/Windows/Input/MouseDevice.cs

Line 1703 in 673c354

_lastPosition = ptClient;

wpf/src/Microsoft.DotNet.Wpf/src/PresentationCore/System/Windows/Input/MouseDevice.cs

Line 1497 in 673c354

Point ptClient = new Point(rawMouseInputReport.X, rawMouseInputReport.Y);

Your current workaround looks like the best and easiest option if it works for you.

[trebahl]

Note that the mouse device reports the last position reported by the message.

OK, then it's not as bad as I feared. I guess I could use reflection to modify _lastPosition, rather than having to fight with SendInput.

[trebahl]

I think I got it to work, at least in not too complicated cases.

        public static void SetMousePosition(this FrameworkElement v, double x, double y)
        {
            var ps = PresentationSource.FromVisual(v);
            var proot = translatePoint.Invoke(null, new object[] { new Point(x, y), v, ps.RootVisual });
            var pcl = roottoclient.Invoke(null, new object[] { proot, ps });
            mousepos.SetValue(Mouse.PrimaryDevice, pcl);
            activesource.SetValue(Mouse.PrimaryDevice, seccrit.Invoke(new object[] { ps }));
        }
// MouseDevice fields to modify
        static FieldInfo mousepos = typeof(MouseDevice).GetField("_lastPosition", BindingFlags.NonPublic | BindingFlags.Instance);
        static FieldInfo activesource = typeof(MouseDevice).GetField("_inputSource", BindingFlags.NonPublic | BindingFlags.Instance);
// required methodinfos because so much stuff is internal
        static MethodInfo roottoclient = typeof(PresentationSource).Assembly.GetType("MS.Internal.PointUtil")
            .GetMethod("RootToClient", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static, null,
            new[] { typeof(Point), typeof(PresentationSource) }, new ParameterModifier[] { });
        static MethodInfo translatePoint = typeof(PresentationSource).Assembly.GetType("System.Windows.Input.InputElement")
            .GetMethod("TranslatePoint", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static, null,
            new[] { typeof(Point), typeof(DependencyObject), typeof(DependencyObject) }, new ParameterModifier[] { });
        static ConstructorInfo seccrit = (typeof(Point).Assembly.GetType("MS.Internal.SecurityCriticalDataClass`1").MakeGenericType(typeof(PresentationSource)))
            .GetConstructors(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)[0];

I guess a cleaner way would be to feed MouseDevice.PreNotifyInput with an ad-hoc NotifyInputEventArgs but I'm not sure how to construct it.

[trebahl]

After passing (10, 20) for a control inside a Window, I got back (10, 20.04) for the same control, I suppose I should accept that as experimental error.

[trebahl]

I suppose adding a function MouseDevice.SetPosition(Visual, Point) that would set _position to the supplied Visual's PointToScreen of the supllied point is out of the question?

[miloush]

I don't find that very likely, no.

[trebahl]

However, the state of buttons is not stored inside the mousedevice, but queried from the system. The function is virtual though, so all hope is not lost. Would it be ok to define a class derived from MouseDevice (defining GetButtonStateFromSystem), instantiate it, and pass it to my MouseEventArgs, or will something explode?

Same question about KeyboardDevice and GetKeyStatesFromSystem BTW.

[miloush]

Saw #5743? Seems like you can't do that now for MouseDevice, but you can for KeyboardDevice.

[trebahl]

Thanks!

[trebahl]

I went to great lengths to try and circumvent this internal restriction on GetButtonStateFromSystem, but nothing worked.
I went so far as to write the derived class in IL and compile it with ilasm. The compilation went fine, but when I try and instantiate the class I defined there I get a TypeLoadException.

If I had a penny for every hour of my life I lost because of an internal...

[trebahl]

May I ask how wpf is tested internally by Microsoft? I'm not seeing much in the tests directory in the published sources.

[miloush]

Have you checked the https://github.com/dotnet/wpf-test repo? There might be some mouse-related tests.

[trebahl]

I think I have devised a way around the impossibility of defining GetButtonStateFromSystem in a custom MouseDevice class. At the same time, it spares me from having to define a custom KeyboardDevice class.

I'm using Detours to hook GetKeyState which is used for both GetButtonStateFromSystem and GetKeyStatesFromSystem!

I have this code in a C++/CLI library:

#include <array>
#include <windows.h>
#include <detours.h>



short (WINAPI *TrueGetKeyState)(int nVirtKey) = &GetKeyState;


class GetKeyStateHook
{
public:
    static GetKeyStateHook instance;

    GetKeyStateHook()
    {
        Clear();
    }

    void Clear()
    {
        States.fill(0);
    }

    std::array<unsigned short, 256> States;

    void Press(int key)
    {
        States[key] = 0x8000;
    }
    void Release(int  key)
    {
        States[key] = 0;
    }
};
GetKeyStateHook GetKeyStateHook::instance;

extern "C"
{
    static short FakeGetKeyState(int nVirtKey)
    {
        return GetKeyStateHook::instance.States[nVirtKey];
    }
}


public ref class GetKeyStateDetour
{
public:
    GetKeyStateDetour()
    {
        DetourRestoreAfterWith();
        DetourTransactionBegin();
        DetourUpdateThread(GetCurrentThread());
        DetourAttach(&(PVOID&)TrueGetKeyState, FakeGetKeyState);
        DetourTransactionCommit();
    }

    ~GetKeyStateDetour()
    {
        DetourTransactionBegin();
        DetourUpdateThread(GetCurrentThread());
        DetourDetach(&(PVOID&)TrueGetKeyState, FakeGetKeyState);
        DetourTransactionCommit();
    }

    void Press(int key) { GetKeyStateHook::instance.Press(key); }
    void Release(int key) { GetKeyStateHook::instance.Release(key); }
    void Clear() { GetKeyStateHook::instance.Clear(); }
};

Then in my test code, I can type:

            using (var d = new GetKeyStateDetour())
            {
                d.Press(KeyInterop.VirtualKeyFromKey(Key.T));
                Console.WriteLine(Keyboard.PrimaryDevice.GetKeyStates(Key.T));
                Console.WriteLine(Keyboard.PrimaryDevice.GetKeyStates(Key.A));
            }
            Console.WriteLine(Keyboard.PrimaryDevice.GetKeyStates(Key.T));
            Console.WriteLine(Keyboard.PrimaryDevice.GetKeyStates(Key.A));

which outputs Down None None None as expected.

[trebahl]

An extra subtlety: the mousedevice must be made active by assigning its _inputSource field for GetButtonStateFromSystem to work. I wrote the following code for that.

        public class MouseDeviceInputSource
        {
            static readonly FieldInfo _inputSource = typeof(MouseDevice).GF("_inputSource");
            static readonly ConstructorInfo secCritCtor;

            static MouseDeviceInputSource()
            {
                if (_inputSource.FieldType == typeof(PresentationSource)) return;
                // case where it's a SecurityCriticalDataClass<PresentationSource>
                secCritCtor = _inputSource.FieldType.GC(typeof(PresentationSource)); //
            }


            public static void Set(MouseDevice dev, UIElement elt)
            {
                var src = PresentationSource.FromVisual(elt);
                object val = secCritCtor == null ? src : secCritCtor.Invoke(new object[] { src });
                _inputSource.SetValue(dev, val);
            }
        }

With this taken care of, you can set a button down with GetKeyStateDetour.Press(1) (1 == VK_LBUTTON).