Support for multiple pointers / mice

[hyblocker]

I'm using RmlUI in a VR environment, where a pointer / mouse is defined by a ray shot from a user's controller towards a 2D surface displaying the UI. Since in VR people can use both hands independently, It would be wise to be able to support using multiple cursors simultaneously. This work would likely be beneficial for touch devices as well and other input systems.

Would re-factoring the Rml::Context class such that all cursor events take in a numbered index representing the pointer index (so that drag, hover, etc. events still work) make sense? I'm willing to draft a PR should this approach be desired.

[mikke89]

I am definitely interested in supporting this scenario. Would it make sense to use a (multi-)touch api for this? See #60 and the linked PR within.

[hyblocker]

Yeah, using a multi-touch API for my use case is probably better as it would then allow for touch screen support and novel hardware support such as my case, as long as some arbitrary amount of "mice" may be used in the system.

[mikke89]

Great. I do wonder more generally about what kind of behavior you are expecting in terms of focus/active/hover state and the like, can you elaborate a bit more on this?

We should take some inspiration from the HTML touch events. As far as I understand, things like hover, and focus are always singular in HTML, even with multi-touch.

[hyblocker]

In the case of VR I'd expect the focus/active/hover states to be handled independently for both cursors. e.g. if I define a page with some buttons and I'm pointing at two different ones with both hands, VR users expect both of them to transition to the hover state etc.

I'm unsure if this would be desirable for all cases however. Would it make sense to define this as an opt-in feature (something like a maxSimultaneousMouseStateEvents variable for instance) so that it could accommodate both use cases?

[mikke89]

I'm not super excited about that change. Having multiple active and hover elements probably break a lot of assumptions, and diverge from HTML.

I would rather consider that we mainly implement the multi-touch events, and rather leave it up to the user to set their own similar states on the elements, for example by setting custom classes, based on those events.

[hyblocker]

That's fair. I completely forgot that I could indicate highlighted elements with my own classes too haha.