modal widgets

[cmyr]

Doing modal dialogs correctly involves a lot of platform coordination, and so I would like as a stopgap to add support for modal widgets; these will be usable for things like a drop-down combo box or a setting panel, allowing us to implement working versions of these things much more easily than if we try and hook platform windows for all of them.

There are some API questions: for instance, a combo box needs to be presented with its current selection aligned at a particular location, and also needs to be maximally visible since we won't be able to draw outside the bounds of the containing window.

The general architecture, though, should be pretty simple: Only one modal will be allowed at a time, it will exist at the window level, and it will communicate back to the main widget tree by sending Commands when an item is selected. We could also add life cycle events for showing and hiding a modal.

I'm not sure where exactly this is on the roadmap, but it's probably something we should do soon. I know that @xStrom has been working on some related stuff, and so if they're interested in exploring this they would be very welcome; otherwise (unless anyone else is interested) I'll take it up sometime after the life cycle work.

[luleyleo]

I decided to write down my thoughts on the topic for #1009 and it got a bit longer and less related to the specific implementation proposed there than expected, thus I'm posting this here instead.

Requirements

The use cases for modals that come to my mind are tool-tips, dialogs, pop-up menus and general overlays such as drawers or completions lists for example.

Due to those, modal should allow nesting, for example a dialog containing a combo box or a button with a tool-tip.

Positioning

There are three types of positioning that would be required to fulfill the mentioned use cases:

• Whole window (dialogs)

• Widget relative (pop-ups, overlays)

• Cursor relative (tool-tips, context menus)

The max size of the modal would be pretty much unbound for relative ones and the windows size for those like dialogs.

For the relative ones, an automatic 'keep-in-window' / 'keep-in-screen' would be needed. Maybe we could do this by passing the remaining down-right space as max bounds and adjust position if it returns a size larger than those max bounds? This would be fairly easy but somewhat breaking the idea of max bounds, maybe there is a better solution.

Input Events

Obviously, modals must receive input events first and should then be able to decide about their propagation. Tool-tips for example should never swallow events and there are cases such as a completion list for a text box, which would only want to catch mouse input but not block the keyboard

Focus cycling (tab) should be contained inside the modal. For this we will probably need a separate focus list for each modal.

Painting

In theory, painting is pretty normal with the only difference that modal always paint over (after) their parent.

Also, dialogs often want to dim the main window, while tool-tips don't and would need a way to control their background.

API

My preferred modal API is a bit different to what #1009 is proposing and would allow modals to be normal, in-tree widgets, representing the same data as the containing widget. This would be the least surprising to users and would avoid doing the app-state-dance which we need to do with menus already and hopefully can get rid of at some point.

For this druid would provide a ModalHost, which can either contain just a modal in the case of dialogs for example, but also a modal plus the widget the modal is associated with.

This dialog will be provided with some way to open and close the modal, which could either be a function Data -> bool or a selector.

To give some ideas what I imagine:

fn modal_dialog() -> impl Widget<String> {
    Modal::dialog(
        slectors::SHOW_MY_DIALOG,
        Label::dynamic(|data, _| data.clone())
    )
}

fn button_with_menu() -> impl Widget<bool> {
    Modal::relative(
        |data| data,
        Menu::from(description),
        Button::new("Menu"),
    )
}

fn button_with_tooltip() -> impl Widget<()> {
    Modal::cursor(
        selector::HOVERED,
        Label::new("Tooltip"),
        Button::new("Button"),
    )
    .controller(OnHover(selector::HOVERED))
}

Implementation

There are two ways how we can implement modals, either simulate modals inside a single window or using platform windows for modals, allowing them to leave the window they are attached to.

One thing I'm uncertain about is whether the single window version has any advantage over native windows, or if we can replace it completely if native windows are available.

Implementation wise I see a couple differences:

Single Window	Native Windows
All events must be routed to the modal first in order to find out if it is interested.	Modals should have access to events in their parent window.
Painting has to be deferred artificially until all parents have done theirs.	Due to paining on a different surface, parents are not affected by modals.
When no space is left in the window, relative modals must be repositioned in the window. Also, dialogs can never be larger than the window containing them.	Same as for single window modals, but in relation to the screen instead of the window. Dialogs could be larger than their parent window, at least in theory.
Window decoration must be offered by druid.	Decoration can be provided by the platform (Wayland being special).
Transparency and blur should be doable with piet normally. So no blur right now, to my knowledge at least.	Transparency and blur might be offered by the platform or not. If transparency is offered but no blur, piet won't be able to do it either.

I would imagine menu bars to be another issue, but as a long time Gnome user I'm certainly not the best one to give advice about how those should work.

The single window version will probably be overall easier to implement but it causes some issues with painting. Native windows will be quite challenging to make reliable and iron out the platform specific quirks, but those aside they seem rather straight forward if we have single window modals working (famous last words).

I've had some experiments with this, and it started fairly well. The lifecycle and update work out of the box, events can be passed to modals first by adding an InternalEvent::RouteModalEvent.

Layout also seems rather easy with this approach. Relative positioning would be just like normal layout only that the modal would be able to leave its parents box. For window relative modals like dialogs, I think it would be sufficient if we add some way to position a widget relative to the window instead of its parent.

The major pain point with in-tree, single window modals is painting. The issue here is that we somehow need to defer it until all parents finished painting, which makes in-tree modals currently impossible. What we would need here is either support for layers which would allow us to composite them in the order we want after all the drawing was done normally, or we implement some sort of 'buffering' in piet, e.g. we record what would have been drawn and replay that later. This feels rather hacky though and I would certainly prefer using layers for this. If we can resolve this issue, then I think in-tree modals are feasible.

[futurepaul]

In regards to deferring painting, isn't that what z-index does?

[jneem]

This sounds good! I think an in-tree API will be much nicer than the global version in #1009 (which I'll close). I didn't follow the API part, though -- based on the tooltip example, my understanding was that Modal would be a wrapper around a normal widget, which might occasionally also decide to display a modal widget in addition. But that understanding doesn't work with the dialog example -- where does that widget live in the tree?

I also didn't understand how the selectors are supposed to work. Is the intention that on a hover, the OnHover controller sends a command to its child? If that's the case, why not have OnHover take a callback instead (which would get a &mut EventCtx, a &mut ChildWidget, and maybe some Data and Env?

[luleyleo]

@jneem

my understanding was that Modal would be a wrapper around a normal widget, which might occasionally also decide to display a modal widget in addition.

My idea (which is nowhere near final) was that a modal is first and foremost just that, a modal. If the modal is associated with some content, which is only relevant for relative modals such as tooltips, then the modal would act as a wrapper. Writing this down it sounds a bit irritating though, probably we can do better / more consistent.

That thing with selectors was just some spit balling really. The idea was that you could provide some sort of activation selector, that can be submitted to show the modal. So instead of the modal mandating what selector should be used to show it, you tell it what selector it should react to. The OnHover was just some ad-hoc controller that emits the provided selector when the widget is hovered, purely meant as an example, not as part of the API.

---

@futurepaul

In regards to deferring painting, isn't that what z-index does?

The problem I head with that was, that I wanted to draw an entire child tree of widgets on a different z-axis which I though was not possible. After some more experimentation it turns out to be possible, but pretty awful as I have to wrap the content widget in a Rc<RefCell<..>> and clone + move that as well as data and the Env into the paint closure. But I got something working with that approach and realized that invalidation might be the most troublesome offender with my (admittedly still very hacky) design, that will need some more investigation though.

[cmyr]

Excuse my having skimmed a bit, but I want to suggest that tooltips and modals are fundamentally different. As I see things, a tooltip is just something a widget draws that can occlude other widgets, and (possibly) be drawn outside the window, but is basically part of that widget. Importantly it doesn't have any influence on event handling.

Modals are basically entirely separate widget graphs (and with platform modals, separate windows) that have their own event handling behaviour, their own focus chains, and fundamentally behave at the level of a druid::Window. The ModalHost solution from runebender is a hacky way of getting a fairly simple 'modal-like' behaviour, but is definitely not the ideal solution.

[xStrom]

Great summary @Finnerale!

Single window vs platform windows

I wasn't a fan of the single window apporach before and reading the summary here I think I'm even less of a fan. It just seems such a detour. To enable proper window-overflowing behavior we will need platform windows eventually anyway and it seems that for the single window approach there are just too many problems that need solving which won't be relevant for the eventual platform window based solution. Thus I personally wouldn't touch the single window approach at all, but if someone wants to I would suggest at least being super hacky and having a destined-for-the-bin attitude about it. No sense in solving a bunch of the issues for code that will exist for just a few months at most.

Full-blown modal dialogs

I'm not sure that they should be anything different than a bool option when creating a new window. That's what they are for the platform and this would keep the number of concepts down that the developers have to learn.

Giving that modal dialog access to only a subset of Data can be useful, but the same is true with non-modal windows, and thus I think this is something that can be thought about in a more generic multi-window context.

Modal elements (comboboxes, tooltips etc)

I think having them look to the developer as close to a normal widget as possible would be awesome - again it keeps the number of concepts low.

In that sense the API that should be the most ergonomic is the one that developers are mostly using.

Button::new("Button").tooltip("Click me!");
Combobox::new(vec!["One", "Two"]);
// etc ..

The API underneath that which enables this high-level API could trade some ergonomics for power because it would only be used by us and by developers building advanced custom widgets.

Input events

I think the modal windows could just receive all input events (as they mostly do by default already) and then decide which they will send to their parent window.