Running the demo.py file

[kamiradi]

Hello authors of Bayes3D @sritchie @nishadgothoskar ,

I am trying to get this code running on a simple video recorded in sim. I am using the latest genjax (0.10.3) with the associated libraries. However, it seems like I am running into issues over a particular binary called by nvdiffrast_jax. Would it be possible to get some indicators on documentation/details on how the nvdiffrast_jax library is generated so that I can make it run for the latest genjax.

A minimum working setup of my installation is as follows

• cuda 12.2
• genjax 0.10.3
• jax[cuda12] 0.5.3
• jaxlib 0.5.3
• python 3.11.12
• torch 2.4.0
• torchvision 0.19.0

On running the demos/demo.py I get the following error

  File "/root/workspace/demos/demo.py", line 160, in test_demo                                                                                                                                
    trace, _ = importance_jit(                                                                                                                                                                
               ^^^^^^^^^^^^^^^                                                                                                                                                                
RuntimeError: Cuda error: 304[cudaGraphicsGLRegisterBuffer(&s.cudaPosBuffer, s.glPosBuffer, cudaGraphicsRegisterFlagsWriteDiscard);]                                                          
Exception raised from rasterizeResizeBuffers at /root/workspace/src/inverse_graphics/renderer/nvdiffrast_jax/nvdiffrast/common/rasterize_gl.cpp:371 (most recent call first):                 
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x770f91acbf86 in /root/.pyenv/versions/inv_env/lib/python3.11/site-packages/torch/lib/libc10.so)                       
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x64 (0x770f91a7ad10 in /root/.pyenv/versions/inv_env/lib/python3.11/site-packages/torch/l
ib/libc10.so)                                                                                                                                                                                 
frame #2: rasterizeResizeBuffers(int, RasterizeGLState&, bool&, int, int, int, int, int) + 0x36c (0x770edc33eb0a in /root/.cache/torch_extensions/py311_cu121/nvdiffrast_plugin_original_gl/nv
diffrast_plugin_original_gl.so)                                                                                                                                                               
frame #3: _rasterize_fwd_gl(CUstream_st*, RasterizeGLStateWrapper&, float const*, int const*, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, float*, float*) 
+ 0x1e5 (0x770edc36d1fd in /root/.cache/torch_extensions/py311_cu121/nvdiffrast_plugin_original_gl/nvdiffrast_plugin_original_gl.so)                                                          
frame #4: jax_rasterize_fwd_gl(CUstream_st*, void**, char const*, unsigned long) + 0x29b (0x770edc36d59c in /root/.cache/torch_extensions/py311_cu121/nvdiffrast_plugin_original_gl/nvdiffrast
_plugin_original_gl.so) 

Your help is appreciated. I'll put a PR/ Docker file that can reproduce the setup for running b3d demo.py.

PS: If there is a way to skip the xla version of the renderer and just use CPU rendering, could you point me towards the relevant change in code? I'd like to run a simple inference over custom sim generated video.