Mouse raycast with Rapier

[ghost]

Hey there !

I know there is plugin which allows to do that but it only works with meshes and I'd need to interact with collision boxes that may be outside meshes.

Here is the setup that I got, I took most of the code from 3D scene example.

use bevy::{
    input::mouse::{MouseButtonInput, MouseMotion, MouseWheel},
    prelude::*,
    window::CursorMoved,
};
use bevy_rapier3d::prelude::*;

fn main() {
    App::new()
        .insert_resource(Msaa { samples: 4 })
        .add_plugins(DefaultPlugins)
        .add_plugin(RapierPhysicsPlugin::<NoUserData>::default())
        .add_plugin(RapierDebugRenderPlugin::default())
        .add_startup_system(setup)
        .add_system(print_mouse_events_system)
        .run();
}

/// set up a simple 3D scene
fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    // plane
    commands.spawn_bundle(PbrBundle {
        mesh: meshes.add(Mesh::from(shape::Plane { size: 5.0 })),
        material: materials.add(Color::rgb(0.3, 0.5, 0.3).into()),
        ..default()
    });
    // cube
    commands.spawn_bundle(PbrBundle {
        mesh: meshes.add(Mesh::from(shape::Cube { size: 1.0 })),
        material: materials.add(Color::rgb(0.8, 0.7, 0.6).into()),
        transform: Transform::from_xyz(0.0, 0.5, 0.0),
        ..default()
    })
    .insert(Collider::cuboid(0.55, 0.55, 0.55))
    .insert_bundle(TransformBundle::from(Transform::from_xyz(0.0, 0.5, 0.0)));

    // light
    commands.spawn_bundle(PointLightBundle {
        point_light: PointLight {
            intensity: 1500.0,
            shadows_enabled: true,
            ..default()
        },
        transform: Transform::from_xyz(4.0, 8.0, 4.0),
        ..default()
    });
    // camera
    commands.spawn_bundle(PerspectiveCameraBundle {
        transform: Transform::from_xyz(-2.0, 2.5, 5.0).looking_at(Vec3::ZERO, Vec3::Y),
        ..default()
    });
}

fn cast_ray(rapier_context: Res<RapierContext>) {
    let ray_pos = Vec3::new(0.0, 0.5, 1.0);
    let ray_dir = Vec3::new(0.0, 0.0, -1.0);
    let max_toi = 4.0;
    let solid = true;
    let groups = InteractionGroups::all();
    let filter = None;

    if let Some((entity, toi)) = rapier_context.cast_ray(
        ray_pos, ray_dir, max_toi, solid, groups, filter
    ) {
        // The first collider hit has the entity `entity` and it hit after
        // the ray travelled a distance equal to `ray_dir * toi`.
        let hit_point = ray_pos + ray_dir * toi;
        println!("Entity {:?} hit at point {}", entity, hit_point);
    }
}

fn print_mouse_events_system(
    mut _mouse_button_input_events: EventReader<MouseButtonInput>,
    mut _mouse_motion_events: EventReader<MouseMotion>,
    mut cursor_moved_events: EventReader<CursorMoved>,
    mut _mouse_wheel_events: EventReader<MouseWheel>,
) {
    for event in cursor_moved_events.iter() {
        event.position;
        println!("Mouse position {}", event.position);
    }
}

How would I cast the ray so that it hits an object under the mouse ? I know that ray_pos needs to be the location of the camera but how can I calculate what ray_dir should be ?

[HackerFoo]

I use a ray starting from the screen, which is at Z=1 in screen space, pointing towards the infinite far plane at Z=0.

To do this, with Vec2 normalized device coordinates ndc, calculate the starting point with ndc.extend(1.) towards another point at ndc.extend(0.5) (halfway to infinity, hah), and use projection_matrix.inverse().project_point3(...) to take them from NDC space to camera space, and apply the camera transform to take them to world space.

You could also just compute a ray from the camera's origin through the point on the screen for perspective projections, but that won't work for orthographic projections.

[HackerFoo]

To make a ray starting from the camera, you can create a ray from camera_transform.translation to the point computed from point_1 above. This is a ray from the camera focal point to the screen, you can think of it as a point on the camera lens, although it's not curved like a lens's surface.

[ghost]

Thanks a lot ! I got it working !

[zwazel]

@Count-Badgerston can you post your solution somewhere? Would be very helpful

[tim-blackbird]

@zwazel Bevy now has a helper method for getting a ray from a cursor position Camera::viewport_to_world.

[zwazel]

ooh thanks @devil-ira!