Specialized UI transform (#16615)

# Objective

Add specialized UI transform `Component`s and fix some related problems:
* Animating UI elements by modifying the `Transform` component of UI
nodes doesn't work very well because `ui_layout_system` overwrites the
translations each frame. The `overflow_debug` example uses a horrible
hack where it copies the transform into the position that'll likely
cause a panic if any users naively copy it.
* Picking ignores rotation and scaling and assumes UI nodes are always
axis aligned.
* The clipping geometry stored in `CalculatedClip` is wrong for rotated
and scaled elements.
* Transform propagation is unnecessary for the UI, the transforms can be
updated during layout updates.
* The UI internals use both object-centered and top-left-corner-based
coordinates systems for UI nodes. Depending on the context you have to
add or subtract the half-size sometimes before transforming between
coordinate spaces. We should just use one system consistantly so that
the transform can always be directly applied.
* `Transform` doesn't support responsive coordinates.

## Solution

* Unrequire `Transform` from `Node`.
* New components `UiTransform`, `UiGlobalTransform`:
- `Node` requires `UiTransform`, `UiTransform` requires
`UiGlobalTransform`
- `UiTransform` is a 2d-only equivalent of `Transform` with a
translation in `Val`s.
- `UiGlobalTransform` newtypes `Affine2` and is updated in
`ui_layout_system`.
* New helper functions on `ComputedNode` for mapping between viewport
and local node space.
* The cursor position is transformed to local node space during picking
so that it respects rotations and scalings.
* To check if the cursor hovers a node recursively walk up the tree to
the root checking if any of the ancestor nodes clip the point at the
cursor. If the point is clipped the interaction is ignored.
* Use object-centered coordinates for UI nodes.
* `RelativeCursorPosition`'s coordinates are now object-centered with
(0,0) at the the center of the node and the corners at (±0.5, ±0.5).
* Replaced the `normalized_visible_node_rect: Rect` field of
`RelativeCursorPosition` with `cursor_over: bool`, which is set to true
when the cursor is over an unclipped point on the node. The visible area
of the node is not necessarily a rectangle, so the previous
implementation didn't work.

This should fix all the logical bugs with non-axis aligned interactions
and clipping. Rendering still needs changes but they are far outside the
scope of this PR.

Tried and abandoned two other approaches:
* New `transform` field on `Node`, require `GlobalTransform` on `Node`,
and unrequire `Transform` on `Node`. Unrequiring `Transform` opts out of
transform propagation so there is then no conflict with updating the
`GlobalTransform` in `ui_layout_system`. This was a nice change in its
simplicity but potentially confusing for users I think, all the
`GlobalTransform` docs mention `Transform` and having special rules for
how it's updated just for the UI is unpleasently surprising.
* New `transform` field on `Node`. Unrequire `Transform` on `Node`. New
`transform: Affine2` field on `ComputedNode`.
This was okay but I think most users want a separate specialized UI
transform components. The fat `ComputedNode` doesn't work well with
change detection.

Fixes #18929, #18930

## Testing

There is an example you can look at: 
```
cargo run --example ui_transform
```

Sometimes in the example if you press the rotate button couple of times
the first glyph from the top label disappears , I'm not sure what's
causing it yet but I don't think it's related to this PR.

##  Migration Guide
New specialized 2D UI transform components `UiTransform` and
`UiGlobalTransform`. `UiTransform` is a 2d-only equivalent of
`Transform` with a translation in `Val`s. `UiGlobalTransform` newtypes
`Affine2` and is updated in `ui_layout_system`.
`Node` now requires `UiTransform` instead of `Transform`. `UiTransform`
requires `UiGlobalTransform`.

In previous versions of Bevy `ui_layout_system` would overwrite UI
node's `Transform::translation` each frame. `UiTransform`s aren't
overwritten and there is no longer any need for systems that cache and
rewrite the transform for translated UI elements.

`RelativeCursorPosition`'s coordinates are now object-centered with
(0,0) at the the center of the node and the corners at (±0.5, ±0.5). Its
`normalized_visible_node_rect` field has been removed and replaced with
a new `cursor_over: bool` field which is set to true when the cursor is
hovering an unclipped area of the UI node.

---------

Co-authored-by: Alice Cecile <[email protected]>

Author: ickshonpe

Cargo.toml

@@ -3543,6 +3543,17 @@ description = "Illustrates how to use 9 Slicing for TextureAtlases in UI"
 category = "UI (User Interface)"
 wasm = true
 
+[[example]]
+name = "ui_transform"
+path = "examples/ui/ui_transform.rs"
+doc-scrape-examples = true
+
+[package.metadata.example.ui_transform]
+name = "UI Transform"
+description = "An example demonstrating how to translate, rotate and scale UI elements."
+category = "UI (User Interface)"
+wasm = true
+
 [[example]]
 name = "viewport_debug"
 path = "examples/ui/viewport_debug.rs"

crates/bevy_ui/src/accessibility.rs

@@ -1,6 +1,7 @@
 use crate::{
     experimental::UiChildren,
     prelude::{Button, Label},
+    ui_transform::UiGlobalTransform,
     widget::{ImageNode, TextUiReader},
     ComputedNode,
 };
@@ -13,11 +14,9 @@ use bevy_ecs::{
     system::{Commands, Query},
     world::Ref,
 };
-use bevy_math::Vec3Swizzles;
-use bevy_render::camera::CameraUpdateSystems;
-use bevy_transform::prelude::GlobalTransform;
 
 use accesskit::{Node, Rect, Role};
+use bevy_render::camera::CameraUpdateSystems;
 
 fn calc_label(
     text_reader: &mut TextUiReader,
@@ -40,12 +39,12 @@ fn calc_bounds(
     mut nodes: Query<(
         &mut AccessibilityNode,
         Ref<ComputedNode>,
-        Ref<GlobalTransform>,
+        Ref<UiGlobalTransform>,
     )>,
 ) {
     for (mut accessible, node, transform) in &mut nodes {
         if node.is_changed() || transform.is_changed() {
-            let center = transform.translation().xy();
+            let center = transform.translation;
             let half_size = 0.5 * node.size;
             let min = center - half_size;
             let max = center + half_size;

crates/bevy_ui/src/focus.rs

@@ -1,18 +1,21 @@
-use crate::{CalculatedClip, ComputedNode, ComputedNodeTarget, ResolvedBorderRadius, UiStack};
+use crate::{
+    picking_backend::clip_check_recursive, ui_transform::UiGlobalTransform, ComputedNode,
+    ComputedNodeTarget, Node, UiStack,
+};
 use bevy_ecs::{
     change_detection::DetectChangesMut,
     entity::{ContainsEntity, Entity},
+    hierarchy::ChildOf,
     prelude::{Component, With},
     query::QueryData,
     reflect::ReflectComponent,
     system::{Local, Query, Res},
 };
 use bevy_input::{mouse::MouseButton, touch::Touches, ButtonInput};
-use bevy_math::{Rect, Vec2};
+use bevy_math::Vec2;
 use bevy_platform::collections::HashMap;
 use bevy_reflect::{std_traits::ReflectDefault, Reflect};
 use bevy_render::{camera::NormalizedRenderTarget, prelude::Camera, view::InheritedVisibility};
-use bevy_transform::components::GlobalTransform;
 use bevy_window::{PrimaryWindow, Window};
 
 use smallvec::SmallVec;
@@ -67,12 +70,12 @@ impl Default for Interaction {
     }
 }
 
-/// A component storing the position of the mouse relative to the node, (0., 0.) being the top-left corner and (1., 1.) being the bottom-right
-/// If the mouse is not over the node, the value will go beyond the range of (0., 0.) to (1., 1.)
+/// A component storing the position of the mouse relative to the node, (0., 0.) being the center and (0.5, 0.5) being the bottom-right
+/// If the mouse is not over the node, the value will go beyond the range of (-0.5, -0.5) to (0.5, 0.5)
 ///
 /// It can be used alongside [`Interaction`] to get the position of the press.
 ///
-/// The component is updated when it is in the same entity with [`Node`](crate::Node).
+/// The component is updated when it is in the same entity with [`Node`].
 #[derive(Component, Copy, Clone, Default, PartialEq, Debug, Reflect)]
 #[reflect(Component, Default, PartialEq, Debug, Clone)]
 #[cfg_attr(
@@ -81,18 +84,17 @@ impl Default for Interaction {
     reflect(Serialize, Deserialize)
 )]
 pub struct RelativeCursorPosition {
-    /// Visible area of the Node relative to the size of the entire Node.
-    pub normalized_visible_node_rect: Rect,
+    /// True if the cursor position is over an unclipped area of the Node.
+    pub cursor_over: bool,
     /// Cursor position relative to the size and position of the Node.
     /// A None value indicates that the cursor position is unknown.
     pub normalized: Option<Vec2>,
 }
 
 impl RelativeCursorPosition {
     /// A helper function to check if the mouse is over the node
-    pub fn mouse_over(&self) -> bool {
-        self.normalized
-            .is_some_and(|position| self.normalized_visible_node_rect.contains(position))
+    pub fn cursor_over(&self) -> bool {
+        self.cursor_over
     }
 }
 
@@ -133,11 +135,10 @@ pub struct State {
 pub struct NodeQuery {
     entity: Entity,
     node: &'static ComputedNode,
-    global_transform: &'static GlobalTransform,
+    transform: &'static UiGlobalTransform,
     interaction: Option<&'static mut Interaction>,
     relative_cursor_position: Option<&'static mut RelativeCursorPosition>,
     focus_policy: Option<&'static FocusPolicy>,
-    calculated_clip: Option<&'static CalculatedClip>,
     inherited_visibility: Option<&'static InheritedVisibility>,
     target_camera: &'static ComputedNodeTarget,
 }
@@ -154,6 +155,8 @@ pub fn ui_focus_system(
     touches_input: Res<Touches>,
     ui_stack: Res<UiStack>,
     mut node_query: Query<NodeQuery>,
+    clipping_query: Query<(&ComputedNode, &UiGlobalTransform, &Node)>,
+    child_of_query: Query<&ChildOf>,
 ) {
     let primary_window = primary_window.iter().next();
 
@@ -234,46 +237,30 @@ pub fn ui_focus_system(
             }
             let camera_entity = node.target_camera.camera()?;
 
-            let node_rect = Rect::from_center_size(
-                node.global_transform.translation().truncate(),
-                node.node.size(),
-            );
-
-            // Intersect with the calculated clip rect to find the bounds of the visible region of the node
-            let visible_rect = node
-                .calculated_clip
-                .map(|clip| node_rect.intersect(clip.clip))
-                .unwrap_or(node_rect);
-
             let cursor_position = camera_cursor_positions.get(&camera_entity);
 
+            let contains_cursor = cursor_position.is_some_and(|point| {
+                node.node.contains_point(*node.transform, *point)
+                    && clip_check_recursive(*point, *entity, &clipping_query, &child_of_query)
+            });
+
             // The mouse position relative to the node
-            // (0., 0.) is the top-left corner, (1., 1.) is the bottom-right corner
+            // (-0.5, -0.5) is the top-left corner, (0.5, 0.5) is the bottom-right corner
             // Coordinates are relative to the entire node, not just the visible region.
-            let relative_cursor_position = cursor_position.and_then(|cursor_position| {
+            let normalized_cursor_position = cursor_position.and_then(|cursor_position| {
                 // ensure node size is non-zero in all dimensions, otherwise relative position will be
                 // +/-inf. if the node is hidden, the visible rect min/max will also be -inf leading to
                 // false positives for mouse_over (#12395)
-                (node_rect.size().cmpgt(Vec2::ZERO).all())
-                    .then_some((*cursor_position - node_rect.min) / node_rect.size())
+                node.node.normalize_point(*node.transform, *cursor_position)
             });
 
             // If the current cursor position is within the bounds of the node's visible area, consider it for
             // clicking
             let relative_cursor_position_component = RelativeCursorPosition {
-                normalized_visible_node_rect: visible_rect.normalize(node_rect),
-                normalized: relative_cursor_position,
+                cursor_over: contains_cursor,
+                normalized: normalized_cursor_position,
             };
 
-            let contains_cursor = relative_cursor_position_component.mouse_over()
-                && cursor_position.is_some_and(|point| {
-                    pick_rounded_rect(
-                        *point - node_rect.center(),
-                        node_rect.size(),
-                        node.node.border_radius,
-                    )
-                });
-
             // Save the relative cursor position to the correct component
             if let Some(mut node_relative_cursor_position_component) = node.relative_cursor_position
             {
@@ -284,7 +271,8 @@ pub fn ui_focus_system(
                 Some(*entity)
             } else {
                 if let Some(mut interaction) = node.interaction {
-                    if *interaction == Interaction::Hovered || (relative_cursor_position.is_none())
+                    if *interaction == Interaction::Hovered
+                        || (normalized_cursor_position.is_none())
                     {
                         interaction.set_if_neq(Interaction::None);
                     }
@@ -334,26 +322,3 @@ pub fn ui_focus_system(
         }
     }
 }
-
-// Returns true if `point` (relative to the rectangle's center) is within the bounds of a rounded rectangle with
-// the given size and border radius.
-//
-// Matches the sdf function in `ui.wgsl` that is used by the UI renderer to draw rounded rectangles.
-pub(crate) fn pick_rounded_rect(
-    point: Vec2,
-    size: Vec2,
-    border_radius: ResolvedBorderRadius,
-) -> bool {
-    let [top, bottom] = if point.x < 0. {
-        [border_radius.top_left, border_radius.bottom_left]
-    } else {
-        [border_radius.top_right, border_radius.bottom_right]
-    };
-    let r = if point.y < 0. { top } else { bottom };
-
-    let corner_to_point = point.abs() - 0.5 * size;
-    let q = corner_to_point + r;
-    let l = q.max(Vec2::ZERO).length();
-    let m = q.max_element().min(0.);
-    l + m - r < 0.
-}

crates/bevy_ui/src/layout/convert.rs

@@ -448,6 +448,8 @@ impl RepeatedGridTrack {
 
 #[cfg(test)]
 mod tests {
+    use bevy_math::Vec2;
+
     use super::*;
 
     #[test]
@@ -523,7 +525,7 @@ mod tests {
             grid_column: GridPlacement::start(4),
             grid_row: GridPlacement::span(3),
         };
-        let viewport_values = LayoutContext::new(1.0, bevy_math::Vec2::new(800., 600.));
+        let viewport_values = LayoutContext::new(1.0, Vec2::new(800., 600.));
         let taffy_style = from_node(&node, &viewport_values, false);
         assert_eq!(taffy_style.display, taffy::style::Display::Flex);
         assert_eq!(taffy_style.box_sizing, taffy::style::BoxSizing::ContentBox);
@@ -661,7 +663,7 @@ mod tests {
     #[test]
     fn test_into_length_percentage() {
         use taffy::style::LengthPercentage;
-        let context = LayoutContext::new(2.0, bevy_math::Vec2::new(800., 600.));
+        let context = LayoutContext::new(2.0, Vec2::new(800., 600.));
         let cases = [
             (Val::Auto, LengthPercentage::Length(0.)),
             (Val::Percent(1.), LengthPercentage::Percent(0.01)),