Skip to content

Commit 7d3af2f

Browse files
staging-devin-ai-integration[bot]streamkit-devinstreamer45
authored
feat(compositor): add circular crop for Loom-style webcam PIP overlays (#176)
* feat(compositor): add circular crop for Loom-style webcam PIP overlays Add a crop_circle boolean field to compositor LayerConfig that clips the composited layer to an ellipse inscribed in the destination rect. When the rect is square this produces a perfect circle — ideal for webcam PIP overlays in the style of Loom. Backend: - config.rs, kernel.rs, mod.rs: thread crop_circle through LayerConfig, ResolvedLayer, LayerSnapshot, CompositeItem, and resolve_scene - blit.rs: add ellipse-masked blit path with anti-aliased edges (~1.5px smoothstep band) to both scale_blit_rgba and scale_blit_rgba_rotated; disable SIMD fast-forward when crop_circle is active - tests.rs: add 3 new tests for axis-aligned, rotated, and full composite_frame circular crop Frontend: - compositor-types.ts: add crop_circle to LayerConfig and ResolvedLayer - compositorConstants.ts: DEFAULT_CROP_CIRCLE = false - compositorLayerParsers.ts: parse/serialize cropCircle <-> crop_circle - compositorAtoms.ts: include cropCircle in layerEqual comparison - compositorOverlays.ts, useCompositorLayers.ts: thread cropCircle - compositorNodeWidgets.tsx: add Circle toggle (MirrorButton) in CropZoomControl; include in reset - compositorNodeInspector.tsx: pass cropCircle prop - compositorCanvasLayers.tsx: CSS border-radius: 50% preview - compositorServerSync.ts: sync cropCircle from server state Demo: - samples/pipelines/dynamic/video_moq_webcam_circle_pip.yml Signed-off-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix(compositor): disable skip_clear for crop_circle, fix mergeOverlayState comparator Address two bugs found by Devin Review: 1. kernel.rs: The skip_clear optimization now checks crop_circle — when true, pixels outside the ellipse must be cleared to transparent black. Without this, pooled buffers would leak stale frame data in the corners. Added a regression test using VideoFramePool with pre-filled garbage. 2. useCompositorLayers.ts: The hasExtraChanges comparator passed to mergeOverlayState now includes cropCircle, so remote config updates that only change cropCircle are no longer silently discarded. Signed-off-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix(compositor): remove overflow:hidden from crop circle preview Remove overflow:hidden from the VideoLayer container when crop_circle is enabled. The overflow:hidden was clipping resize handles positioned at corners since they fall outside the inscribed ellipse. The circular outline (via borderRadius: 50%) alone provides sufficient visual preview of the crop circle shape. Signed-off-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * refactor: replace crop_circle bool with crop_shape enum across full stack Replace the boolean crop_circle field with an extensible CropShape enum (Rect | Circle) throughout backend and frontend. This enables future shape variants (RoundedRect, Hexagon, etc.) without breaking changes. Backend: - Add CropShape enum with serde rename_all snake_case - Replace crop_circle: bool with crop_shape: CropShape in LayerConfig, ResolvedLayer, LayerSnapshot, CompositeItem, ResolvedSlotConfig - Kernel converts enum to bool at blit call site - Update all tests and benchmarks Frontend: - Replace cropCircle: boolean with cropShape: 'rect' | 'circle' in LayerState, parsers, atoms, overlays, server sync, drag resize - Replace On/Off MirrorButton toggle with segmented control (▭ Rect / ● Circle) matching existing design patterns - Update DEFAULT_CROP_CIRCLE to DEFAULT_CROP_SHAPE - Update all test files Pipeline: - Update demo pipeline YAML to use crop_shape: rect/circle Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * docs(compositor): fix misleading CropShape forward-compat doc comment The doc comment claimed unknown values deserialize as Rect, but there was no #[serde(other)] to back that up. Corrected the comment to accurately describe the actual behavior: field-level #[serde(default)] handles missing keys, but unknown variant strings will error. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * perf(compositor): hoist ellipse AA constants out of rotated blit hot loop ellipse_sx, ellipse_sy, aa_band, and aa_inner only depend on the destination rect dimensions (rw/rh) which are loop-invariant. Move them before the per-row closure, matching the axis-aligned path which already hoists these correctly. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> --------- Signed-off-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Signed-off-by: Devin AI <devin@devin.ai> Co-authored-by: StreamKit Devin <devin@streamkit.dev> Co-authored-by: Claudio Costa <cstcld91@gmail.com>
1 parent 3b5f4cd commit 7d3af2f

19 files changed

+631
-27
lines changed

crates/engine/benches/compositor_only.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ fn make_layer(
120120
crop_zoom: 1.0,
121121
crop_x: 0.5,
122122
crop_y: 0.5,
123+
crop_shape: streamkit_nodes::video::compositor::config::CropShape::Rect,
123124
})
124125
}
125126

@@ -414,6 +415,7 @@ fn build_scenarios(canvas_w: u32, canvas_h: u32) -> Vec<Scenario> {
414415
crop_zoom: 1.0,
415416
crop_x: 0.5,
416417
crop_y: 0.5,
418+
crop_shape: streamkit_nodes::video::compositor::config::CropShape::Rect,
417419
}),
418420
Some(LayerSnapshot {
419421
data: pip,
@@ -429,6 +431,7 @@ fn build_scenarios(canvas_w: u32, canvas_h: u32) -> Vec<Scenario> {
429431
crop_zoom: 1.0,
430432
crop_x: 0.5,
431433
crop_y: 0.5,
434+
crop_shape: streamkit_nodes::video::compositor::config::CropShape::Rect,
432435
}),
433436
]
434437
},

crates/nodes/src/video/compositor/config.rs

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,27 @@ use std::collections::HashMap;
1111

1212
// ── Configuration ───────────────────────────────────────────────────────────
1313

14+
/// Shape used to clip a composited layer.
15+
///
16+
/// `Rect` (the default) renders the layer as-is within its destination
17+
/// rectangle. `Circle` clips to an ellipse inscribed in the destination
18+
/// rect — when the rect is square this produces a perfect circle, ideal
19+
/// for Loom-style webcam PIP overlays.
20+
///
21+
/// New variants (e.g. `RoundedRect`, `Hexagon`) can be added in the
22+
/// future. The field-level `#[serde(default)]` on `LayerConfig` means a
23+
/// missing `crop_shape` key defaults to `Rect`.
24+
#[derive(Deserialize, Serialize, Debug, Clone, Copy, Default, PartialEq, Eq, JsonSchema)]
25+
#[cfg_attr(feature = "codegen", derive(ts_rs::TS))]
26+
#[serde(rename_all = "snake_case")]
27+
pub enum CropShape {
28+
/// No shape clipping — the layer fills its destination rectangle.
29+
#[default]
30+
Rect,
31+
/// Clip to an ellipse inscribed in the destination rectangle.
32+
Circle,
33+
}
34+
1435
const fn default_width() -> u32 {
1536
1280
1637
}
@@ -204,6 +225,10 @@ pub struct LayerConfig {
204225
/// visible effect when `crop_zoom > 1.0`. Default 0.5.
205226
#[serde(default = "default_crop_center")]
206227
pub crop_y: f32,
228+
/// Shape clipping applied to the layer. Default `Rect` (no clipping).
229+
/// Set to `Circle` for Loom-style circular webcam PIP overlays.
230+
#[serde(default)]
231+
pub crop_shape: CropShape,
207232
}
208233

209234
impl Default for LayerConfig {
@@ -218,6 +243,7 @@ impl Default for LayerConfig {
218243
crop_zoom: default_crop_zoom(),
219244
crop_x: default_crop_center(),
220245
crop_y: default_crop_center(),
246+
crop_shape: CropShape::default(),
221247
}
222248
}
223249
}
@@ -299,6 +325,8 @@ pub struct ResolvedLayer {
299325
pub crop_x: f32,
300326
/// Normalized crop tilt Y (0.0–1.0).
301327
pub crop_y: f32,
328+
/// Shape clipping applied to the layer.
329+
pub crop_shape: CropShape,
302330
}
303331

304332
/// Server-computed layout for a single overlay (text or image).

crates/nodes/src/video/compositor/kernel.rs

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,8 @@ pub struct LayerSnapshot {
218218
pub crop_x: f32,
219219
/// Normalized crop tilt Y (0.0–1.0). Default 0.5 (centred).
220220
pub crop_y: f32,
221+
/// Shape clipping applied to the layer.
222+
pub crop_shape: super::config::CropShape,
221223
}
222224

223225
/// Work item sent from the async loop to the persistent compositing thread.
@@ -252,6 +254,7 @@ pub struct CompositeResult {
252254

253255
/// A resolved, ready-to-composite item. Unifies video layers and decoded
254256
/// overlays into a single type for the z-sorted compositing loop.
257+
#[allow(clippy::struct_excessive_bools)]
255258
struct CompositeItem<'a> {
256259
src_data: &'a [u8],
257260
src_width: u32,
@@ -269,6 +272,8 @@ struct CompositeItem<'a> {
269272
/// Source sub-region in pixel coordinates `(x, y, w, h)`. `None` means
270273
/// sample the entire source. Used for virtual PTZ crop/zoom.
271274
src_region: Option<(u32, u32, u32, u32)>,
275+
/// Shape clipping applied to the composited layer.
276+
crop_shape: super::config::CropShape,
272277
}
273278

274279
/// Compute the source crop rectangle from normalised crop parameters.
@@ -361,7 +366,10 @@ pub fn composite_frame(
361366
layers.iter().enumerate().find_map(|(i, e)| e.as_ref().map(|l| (i, l))).is_some_and(
362367
|(_slot_idx, layer)| {
363368
// Quick checks that don't need the pixel data.
364-
if layer.opacity < 1.0 || layer.rotation_degrees.abs() >= 0.01 {
369+
if layer.opacity < 1.0
370+
|| layer.rotation_degrees.abs() >= 0.01
371+
|| layer.crop_shape != super::config::CropShape::Rect
372+
{
365373
return false;
366374
}
367375
let covers = layer.rect.as_ref().is_none_or(|r| {
@@ -446,6 +454,7 @@ pub fn composite_frame(
446454
mirror_horizontal: layer.mirror_horizontal,
447455
mirror_vertical: layer.mirror_vertical,
448456
src_region,
457+
crop_shape: layer.crop_shape,
449458
});
450459
insertion_order += 1;
451460
}
@@ -464,6 +473,7 @@ pub fn composite_frame(
464473
mirror_horizontal: ov.mirror_horizontal,
465474
mirror_vertical: ov.mirror_vertical,
466475
src_region: None,
476+
crop_shape: super::config::CropShape::Rect,
467477
});
468478
insertion_order += 1;
469479
}
@@ -482,6 +492,7 @@ pub fn composite_frame(
482492
mirror_horizontal: ov.mirror_horizontal,
483493
mirror_vertical: ov.mirror_vertical,
484494
src_region: None,
495+
crop_shape: super::config::CropShape::Rect,
485496
});
486497
insertion_order += 1;
487498
}
@@ -505,6 +516,7 @@ pub fn composite_frame(
505516
item.mirror_horizontal,
506517
item.mirror_vertical,
507518
item.src_region,
519+
item.crop_shape == super::config::CropShape::Circle,
508520
);
509521
}
510522

crates/nodes/src/video/compositor/mod.rs

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ struct InputSlot {
7070
/// Rebuilt only when compositor config or pin set changes, avoiding
7171
/// per-frame `HashMap` lookups and `sort_by` calls.
7272
#[derive(Clone)]
73+
#[allow(clippy::struct_excessive_bools)]
7374
struct ResolvedSlotConfig {
7475
rect: Option<config::Rect>,
7576
opacity: f32,
@@ -84,6 +85,7 @@ struct ResolvedSlotConfig {
8485
crop_zoom: f32,
8586
crop_x: f32,
8687
crop_y: f32,
88+
crop_shape: config::CropShape,
8789
}
8890

8991
/// Fully-resolved compositor scene for one configuration epoch.
@@ -152,6 +154,7 @@ fn resolve_scene(
152154
crop_zoom,
153155
crop_x,
154156
crop_y,
157+
crop_shape,
155158
) = if let Some(lc) = layer_cfg {
156159
(
157160
lc.rect,
@@ -164,6 +167,7 @@ fn resolve_scene(
164167
lc.crop_zoom,
165168
lc.crop_x,
166169
lc.crop_y,
170+
lc.crop_shape,
167171
)
168172
} else if idx > 0 && num_slots > 1 {
169173
// Auto-PiP: non-first layers without explicit config.
@@ -185,9 +189,10 @@ fn resolve_scene(
185189
1.0,
186190
0.5,
187191
0.5,
192+
config::CropShape::Rect,
188193
)
189194
} else {
190-
(None, 1.0, 0, 0.0, false, false, false, 1.0, 0.5, 0.5)
195+
(None, 1.0, 0, 0.0, false, false, false, 1.0, 0.5, 0.5, config::CropShape::Rect)
191196
};
192197

193198
// Build the view-data layer using the current latest_frame for
@@ -219,6 +224,7 @@ fn resolve_scene(
219224
crop_zoom,
220225
crop_x,
221226
crop_y,
227+
crop_shape,
222228
});
223229

224230
configs.push(ResolvedSlotConfig {
@@ -232,6 +238,7 @@ fn resolve_scene(
232238
crop_zoom,
233239
crop_x,
234240
crop_y,
241+
crop_shape,
235242
});
236243
}
237244

@@ -772,6 +779,7 @@ impl ProcessorNode for CompositorNode {
772779
crop_zoom: cfg.crop_zoom,
773780
crop_x: cfg.crop_x,
774781
crop_y: cfg.crop_y,
782+
crop_shape: cfg.crop_shape,
775783
}
776784
})
777785
})

0 commit comments

Comments
 (0)