Markov Chain Monte Carlo Gaussian splat optimizer

[harrism]

Closes #181
Closes #182
Closes #183
Part of #180

@fwilliams contributed to this PR.

This pull request introduces support for a new MCMC-based optimizer for Gaussian Splat radiance field reconstruction, refactors the optimizer registration and deserialization system for extensibility, and adds shared test utilities for Gaussian Splat models. The changes enable users to select between classic and MCMC optimizer strategies from the frgs CLI, ensure optimizer classes are registered and deserialized robustly, and provide reusable testing infrastructure.

it's not possible to train (for example) the garden scene with the MCMC optimizer with a target maximum number of Gaussians of 3M via the following example command. (To train with the "classic" optimizer just change reconstruct-mcmc to reconstruct).

frgs reconstruct-mcmc ./data/360_v2/garden/ --opt.max_gaussians 3000000

New MCMC Optimizer Support

• Added GaussianSplatOptimizerMCMC and GaussianSplatOptimizerMCMCConfig classes, and exposed them in the package API (__init__.py) for use in reconstruction workflows. [1] [2] [3]
• Introduced the ReconstructMCMC CLI command (frgs reconstruct-mcmc), allowing users to reconstruct scenes using the MCMC optimizer strategy via the frgs CLI. [1] [2]
• Updated the reconstruction pipeline to select the optimizer type (classic or MCMC) based on the optimizer config provided, and improved docstrings to reflect this. [1] [2]
• Python tests for the MCMC optimizer and for training using it.
• Basic mcmc unit benchmarks

Optimizer Registration and Serialization Refactor

• Implemented a splat_optimizer decorator and optimizer registry to ensure all optimizer classes are registered for (de)serialization, and refactored from_state_dict to use the registry for robust deserialization. [1] [2] [3] [4] [5] [6]
• Updated serialization to include the optimizer class name in the state dict, supporting round-trip saving/loading across optimizer types. [1] [2]

Testing Infrastructure Improvements

• Added a tests/__init__.py to make the test directory a package, enabling shared test helpers.
• Introduced reusable Gaussian Splat test utilities in tests/unit/common.py, including scene loading, model initialization, and rendering helpers, as well as a mixin for test cases.

[Copilot]

Pull request overview

This pull request introduces support for a Markov Chain Monte Carlo (MCMC) based optimizer for Gaussian Splat reconstruction, alongside a refactored optimizer registration and deserialization system. The changes enable users to choose between the classic optimization strategy and the new MCMC strategy via both the Python API and CLI, with full serialization support for checkpoint saving and loading across optimizer types.

Key changes:

• New GaussianSplatOptimizerMCMC class implementing the MCMC optimization strategy based on the paper "Gaussian Splatting as Markov Chain Monte Carlo"
• Decorator-based optimizer registry system (@splat_optimizer) for robust optimizer (de)serialization
• Shared test infrastructure with GettysburgGaussianSplatTestCase mixin to reduce code duplication across tests
• New ReconstructMCMC CLI command for easy access to MCMC optimization

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
tests/__init__.py	Makes tests directory a package to enable shared helper imports
tests/unit/common.py	Adds shared Gaussian Splat test utilities including scene loading, model initialization, and GettysburgGaussianSplatTestCase mixin
tests/unit/test_gaussian_splat_optimizer.py	Refactored to use shared test infrastructure, removing duplicated setup code
tests/unit/test_gaussian_splat_optimizer_mcmc.py	New test suite for MCMC optimizer covering serialization, relocation, and insertion
tests/unit/test_gaussian_splat_reconstruction_optimizer_registry.py	Tests optimizer registry and checkpoint roundtrip for both classic and MCMC optimizers
tests/unit/test_training.py	Adds integration tests for MCMC optimizer in training loop
fvdb_reality_capture/radiance_fields/base_gaussian_splat_optimizer.py	Introduces @splat_optimizer decorator and registry system for optimizer (de)serialization
fvdb_reality_capture/radiance_fields/gaussian_splat_optimizer.py	Adds @splat_optimizer decorator, make_optimizer() factory method, and state dict name field
fvdb_reality_capture/radiance_fields/gaussian_splat_optimizer_mcmc.py	New MCMC optimizer implementation with noise injection, relocation, and insertion operations
fvdb_reality_capture/radiance_fields/gaussian_splat_reconstruction.py	Updates to use config-based optimizer instantiation via make_optimizer()
fvdb_reality_capture/radiance_fields/__init__.py	Exports new MCMC optimizer classes
fvdb_reality_capture/cli/frgs/_reconstruct.py	Adds ReconstructMCMC command class
fvdb_reality_capture/cli/frgs/__init__.py	Registers ReconstructMCMC command in CLI

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Import of 'GaussianSplatOptimizerMCMC' is not used.
Import of 'GaussianSplatOptimizerMCMCConfig' is not used.

[harrism]

Replaced by #214