Render-less Cameras for NPC perception

[0x0539]

bevy::render::view::check_visibility seems perfect for what I'm trying to do, which is to attach a Camera to each NPC and then use the ViewVisibility component to detect what objects the NPC can see.

This seems to be almost working, but there's an issue I can't figure out. Basically, I get some warnings like

  WARN bevy_render::camera::camera: Camera order ambiguities detected for active cameras with the following priorities: {(0, Some(Window(NormalizedWindowRef(0v1))))}. To fix this, ensure there is exactly one Camera entity spawned with a given order for a given RenderTarget. Ambiguities should be resolved because either (1) multiple active cameras were spawned accidentally, which will result in rendering multiple instances of the scene or (2) for cases where multiple active cameras is intentional, ambiguities could result in unpredictable render results.

Another issue: with multiple Cameras rendering to the same RenderTarget, the render results look bad, and there's also a bunch of unnecessary rendering happening (I don't need to actually render the NPC perspective, just run object culling).

How can I add Cameras to the NPCs so that I get ViewVisibility without any further rendering? I didn't see a DummyrenderTarget or anything like that.

[alice-i-cecile]

I'd be interested in seeing a DummyRenderTarget PR honestly: this seems like an interesting application.

[0x0539]

I looked at the bevy_render code a bit. It seems like VisibilityPlugin only needs a reference to a Camera in order to check whether camera.is_active is true.

If Bevy makes this an Option<&Camera> then I might be able to get away without Cameras entirely, and instead just add the other parts of the bundle that I need to my NPC (Frustum, VisibleEntities, and Projection, I think?)

It would be great to get a second opinion from someone more familiar with this code than me.

[0x0539]

I think I was able to get something working. I'll put my code below in case it's useful to someone later.

1. Spawn an entity to represent the "NPC camera". It's a subset of the Camera3dBundle with just the bare necessities.

commands.entity(npc)
            .with_children(|parent| {
                parent.spawn((
                    SpatialBundle::default(),
                    VisibleEntities::default(),
                    Frustum::default(),
                    PerspectiveProjection::default(),
                    NPCCameraMarker,
                ));
            });

2. Define an Update system to mark visible entities every frame. This is just a copy of the check_visibility fn from Bevy but without the Camera and RenderLayers stuff:

/// System updating the visibility of entities each frame.
/// Camera-free version of `check_visibility` from bevy_render.
pub fn mark_visible_entities(
    mut thread_queues: Local<Parallel<Vec<Entity>>>,
    mut view_query: Query<(&mut VisibleEntities, &Frustum), With<NPCCameraMarker>>,
    mut visible_aabb_query: Query<(
        Entity,
        &InheritedVisibility,
        Option<&Aabb>,
        &GlobalTransform,
        Has<NoFrustumCulling>,
    )>,
    deterministic_rendering_config: Res<DeterministicRenderingConfig>,
) { ... }

3. Query the visible entities like this:

pub(crate) fn update_perception(
    query: Query<(&VisibleEntities, &NPCCameraMarker)>,
    players: Query<&Player>,
    mut commands: Commands
) {
    let mut num_cams = 0;
    for (visible_entities, camera_marker) in query.iter() {
        for visible_entity in visible_entities.iter() {
            if players.get(*visible_entity).is_ok() {
                num_cams += 1;
            }
        }
    }
    info!("player visible to {} npc cams", num_cams);
}

I'm not super confident this is correct but it seems to be working in my local.

One note for wayfarers: this code doesn't handle occlusions, just frustum culling, so it's really just a starting point for an NPC perception system. In my own version I think I'll run some ray casting on the culled entities to actually make a final decision on visibility.