Rendering Scenes, Text, SVGs and Lotties with bevy_ui coordinates and bevy_core_2d coordinates

[RobertBrewitz]

There are a few cases that are not supported in the current extract/prepare/render "pipeline".

1. Render VelloScenes/SVG/Lottie in bevy_ui coordinates without a bevy_ui Node component

Why is it needed? To render anything derived from transforms computed by bevy_ui, e.g. animating a VelloScene between two UI Nodes.

What is needed? A non-transposed ndc_to_pixel_matrix for VelloScene if it is supposed to render in bevy_ui coordinate space.

2. Render VelloTextSection in "UI space" with a bevy_ui Node component

Why is it needed? To render text in and on bevy_ui Nodes

What is needed? If a VelloTextSection has a Node component it should not apply a "vello_matrix", simply:

            let transform: [f32; 16] = world_transform.compute_matrix().to_cols_array();

            // | a c e |
            // | b d f |
            // | 0 0 1 |
            let transform: [f64; 6] = [
                transform[0] as f64,  // a
                -transform[1] as f64, // b
                -transform[4] as f64, // c
                transform[5] as f64,  // d
                transform[12] as f64, // e
                transform[13] as f64, // f
            ];

How the component composition would be handled I am not sure, personally I am using two separate master vello scenes, one for "UI space" and one for "2d/world space" rendering and then have VelloUiText, VelloWorldText, VelloUiScene, VelloWorldScene, VelloUiSvg and VelloWorldSvg.

Visual example of what I mean, here I render a VelloScene and VelloSvg in UI space without a Node, based on transforms set by bevy_ui, I also render VelloTextSection based on bevy_ui calculated transforms.

[simbleau]

Currently if you attach a Node to any VelloSvgHandle or VelloLottieHandle, it will map the asset to the node and render in "Ui space". I think it's a good pattern to follow.

Text and Scenes don't have this behavior because:

• Bevy already had good UI text, so there was no reason to use bevy_vello Text in screen space. (Now we have good bevy_vello text, so this point is moot)
• Vello scenes have no way to determine the bounding box, unlike SVGs/Lotties which have a width/height, so I didn't know how how to map the scene onto a node surface.

If we push forward, I think the presense of a Node extracted with a VelloTextSection or VelloScene would be a good pattern to follow, and determine the behavior of how we translate the coordinates.

[RobertBrewitz]

Currently if you attach a Node to any VelloSvgHandle or VelloLottieHandle, it will map the asset to the node and render in "Ui space". I think it's a good pattern to follow.

This locks the UI space to Taffy layouting as far as I can tell, and is what I am experiencing. If I want to render fancy animations in UI space, it is not possible with only bevy_vello, e.g. Transform's translations are locked to taffy.