Clone the repository with the submodules by using:
git clone --recursive [email protected]:Zhanpeng1202/Instant4D.gitUpdate requirements.txt with correct CUDA version for PyTorch and cuUML, i.e., replacing cu126 and cu12 with your CUDA version.
conda create -n instant4d python=3.10
conda activate instant4d
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126 # change to your CUDA version
pip install -r requirement.txt
pip install xformers # read below
# Note: mega-sam requires xformers for Unidepth, but there is a dependancy issue for newer pytroch
# But Gaussian Splatting require CUDA version match.
# therefore one workaround is to use a different virtual environment only for Unidepth metric depth estimaitonTo install mega-sam, run the following command:
Note: change the .type() to scalar_type() in mega-sam\base\src\altcorr_kernel, mega-sam\base\src\correlation_kernels and mega-sam/base/thirdparty/lietorch/lietorch/src/lietorch_gpu.cu if using torch >2.7, refer this issue.
cd SLAM/mega-sam/base
python setup.py install
cd ../../../../To install Gaussian Splatting accelerating package, run the following command:
cd submodule
pip install fussed-ssim
pip install simple-knn
cd pointops2
python setup.py install
cd ../..Noted that the gaussian splatting package will be compile during the first running.
-
Download DepthAnything checkpoint to mega-sam/Depth-Anything/checkpoints/depth_anything_vitl14.pth
-
Download and include RAFT checkpoint at mega-sam/cvd_opt/raft-things.pth
We provide a lightweight websocket remote viewer to visualize 4DGS training process. Users can train 4DGS on a server and hope to view it on local computer.
On the local computer
# download these file in local computer
git clone [email protected]:Zhanpeng1202/gaussian_splatting_websocket_viewer.git
# Connect Server with SSH with vscode
vscode ssh server
#set up forward port in vscode
Terminal -> Ports -> Forward a Ports -> 6119On the server
# clone the official gaussain splatting repository
git clone [email protected]:graphdeco-inria/gaussian-splatting.git --recursive
# put networkGUI_Websocket.py in to correct location inside the cloned repository
<location>
|---gaussian_splatting
| |---gaussain_render
| | |---network_gui.py
| | |---network_gui_websocket.py
| |---train.py
# replace train.py with that provided in this repositorymkdir dataset
cd dataset
Download the pre-processed data by DynamicNeRF.
mkdir Nvidia
wget --no-check-certificate https://filebox.ece.vt.edu/~chengao/free-view-video/data.zip
unzip data.zip
rm data.zip
DAVIS or custom sequences
We provide sample videos under examples/, one can start from reproduce them.
Change the path and weight in script/reconstruct.sh and change the config accordingly in the script/optmize
source script/reconstruct.sh
python -m script.prune
python -m script.optmize
This work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!
If you find this project useful in your research, please consider citing:
@article{luo2025instant4d,
title={Instant4d: 4d gaussian splatting in minutes},
author={Luo, Zhanpeng and Ran, Haoxi and Lu, Li},
journal={Advances in neural information processing systems},
year={2025}
}