Draw things behind the UI

[lthms]

Hi there,

I am no expert here, but I think it might be useful to add a layer behind the UI one, where it is possible to draw things (basically, the game). A good behavior should be that UI catches keystrokes first, then if it can do nothing with it, gives it back to this layer.

Thus, in “HUD UI”, arrows are transfered. But when Escap is hit, it switches to menu and arrows are catches. That way, character is not longer moving.

Is something like that already existing in OIL?

I want to write a PoC client and I need something like that!

[Nemikolh]

I think you are mixing two problems here:

• Drawing things behind the UI
• Passing user events such as key or mouse between the UI and a library user custom handler

Drawing things behind the GUI is possible today because of the way OpenGL works. Every draw calls that happens before the ones done by oil when you call Router::render_views are going to be behind the UI (If I'm correct oil disable depth testing when drawing). Ideally we could improve that a bit by drawing on a given FrameBuffer provided by the library user. It would also allow to have the UI drawn on a texture that could then be reused anywhere.

Oil has currently no real support for user events in any ways. The examples use a simple match on keys to switch the focus. We need to think first about a design that would be flexible enough to cover each use cases without compromising too much the user experience of the UI designer.

Your use case is really interesting, and I see multiple requirements to solve it:

• Use a FSM and forward the events first to the UI or to your game based on the FSM state. Note that nothing prevents you from sending the event to both if you need to.
• Oil need also to implements an event bubbling system, such as the one that exists in browsers, and give back the event to the library user when the event hasn't been consumed.

That might be a good first start.

[lthms]

The question is to decide whenever the UI needs to use an event.

Moreover, we have to think to an intermediate event system. Indeed, let's imagine the UI is an inventory and the player wants to delete an item (let says he has to hit X to do so).

How to deal with that? Forward the X input is a non sense, it won't help the underlying library at all. Like we can implement some sort of signal, the same way Qt does. It might be tricky and might require to add to UI definition some hidden fields that can be latter used as signal arguments. To continue with the inventory example, the hidden field can be the object ID.

[Nemikolh]

Ah I see, you're right, the library need some kind of metadata when the event is consumed. As buttons are designed in the doc, there's an action attribute that allow to call a custom handler when "something" happend on the button. As it is right now, it doesn't provide any meta data. It is basically just a string name.

We could eventually redesign it to be something like:

<button action="bar({player.name})" />

Where {player.name} is a data-bindings that would then be forwarded to the callback with the action - event - string name.
Thus, the callback receiver would have the name of the action, the data binding specific value and the user event (key pressed, key released, mouse click, ...).

[lthms]

Thus, the callback receiver would have the name of the action, the data binding specific value and the user event (key pressed, key released, mouse click, ...).

I disagree: key pressed, released etc. is to UI oriented event.

I propose to add something like <button signalOnClick="signal_name(args) signalOnEtc="...">. Dealing with args might be (very) difficult though. An even better idea is to separate signals declaration from the rest.

After that, code registers to signal it wants.

(sorry for the spam)

[Nemikolh]

I think you are reacting a bit to quickly here.

Several points:

• First I like the action naming and I think we should keep it. Especially because whatever solution we choose it won't be anything like signals in Qt. Signals in Qt are used only when we talk about Qt Widget and they are not used anymore for the UI starting with QML.
• Secondly, knowing which key is pressed will always be needed. Any game might have a control menu where the key could be changed to another one, so having something like signalOnKeyA would just not work. And I don't see the point of removing the fact that the key is pressed or not. I have never met yet a framework that remove that information to the handler. Angular, GWT, the default browser API, all provide that information because for some reason the library user might want to know that.
• I'm not sure what you want to see in the arguments, but so far apart from data bindings or constants there not much to do. So in a first time we could stick to at most one argument that could only be a data bindings. We can then figure out the rest later. I also like this solution because it match nicely with the rest of the design of the library.

[lthms]

I think it's better if you divide this issue into two (as you first pointed out). I'll need to write a little my thoughts, because I's messy in my head for now.