Conversation
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
There was a problem hiding this comment.
Pull request overview
This pull request adds 3D Gaussian Splatting (3DGS) rasterization capability that enables rendering directly from world-space Gaussians rather than from 2D projections. This allows geometry gradients to flow through the rasterization step, which is particularly useful for non-pinhole camera models with distortion.
Changes:
- Adds new
render_images_from_world_3dgsmethod that performs ray-ellipsoid rasterization directly in world space - Introduces
CameraModelandRollingShutterTypeenums for flexible camera projection support - Implements CUDA kernels for forward and backward passes of the 3DGS rasterizer with support for multiple camera models (pinhole, orthographic, and OpenCV distortion variants)
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/fvdb/detail/ops/gsplat/GaussianCameraModels.h | Defines camera model and rolling shutter enums (contains critical duplication bug) |
| src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorld3DGS.cuh | Shared utilities for ray generation, distortion, and coordinate transformations |
| src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorld3DGSForward.cu | CUDA kernel for forward rasterization pass |
| src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorld3DGSBackward.cu | CUDA kernel for backward gradient computation |
| src/fvdb/detail/autograd/GaussianRasterizeFromWorld3DGS.{h,cpp} | PyTorch autograd wrapper for the rasterization operation |
| src/fvdb/GaussianSplat3d.{h,cpp} | Main API implementation routing to appropriate projection and rasterization paths |
| src/python/GaussianSplatBinding.cpp | Python bindings for the new enums and method |
| fvdb/gaussian_splatting.py | Python wrapper for the new rendering method |
| fvdb/enums.py | Python enum definitions for camera models and rolling shutter types |
| fvdb/_fvdb_cpp.pyi, fvdb/init.pyi, fvdb/init.py | Type stubs and exports for the new functionality |
| tests/unit/test_rasterize_world3dgs.py | Unit tests for gradient correctness and finite difference validation |
| src/CMakeLists.txt | Build configuration for new source files |
| src/fvdb/detail/ops/gsplat/GaussianProjectionUT.h | Refactored to use shared enum definitions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorld3DGSForward.cu
Outdated
Show resolved
Hide resolved
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Francis Williams <fwilliams@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorldForward.cu
Outdated
Show resolved
Hide resolved
src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorldBackward.cu
Outdated
Show resolved
Hide resolved
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
Signed-off-by: Francis Williams <francis@fwilliams.info>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/fvdb/detail/ops/gsplat/GaussianRasterizeFromWorldForward.cu
Outdated
Show resolved
Hide resolved
|
Could we get a description of the PR please? |
| const torch::Tensor &distortionCoeffs, // [C, K] (K=0 or 12) | ||
| const RollingShutterType rollingShutterType, | ||
| const CameraModel cameraModel, | ||
| // Render settings |
There was a problem hiding this comment.
We should use the RenderSettings object here like the other dispatchGaussianRasterize functions.
We should also just roll CameraModel and RollingShutterType into RenderSettings as well to help transport them all together.
2 participants
No description provided.