Building on Ubuntu 20.04 - errors

[nyue]

I am trying to build both the head and v3.0.2 on Ubuntu 20.04

It has Python 3.8 and GCC 9.4 which I believe meets the requirements.

nyue@Compaq-8200-Elite:~/projects/mitsuba/mitsuba3/build$ which python
/usr/bin/python
nyue@Compaq-8200-Elite:~/projects/mitsuba/mitsuba3/build$ python --version
Python 3.8.10

nyue@Compaq-8200-Elite:~/projects/mitsuba/mitsuba3/build$ gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

However, I keep encountering the following errors and wonder if it is something I need to address on my system in order to build.

CMake Error at CMakeLists.txt:137 (message):
  Could not run resources/configure.py script: File
  "/home/nyue/projects/mitsuba/mitsuba3/mitsuba3_git/resources/configure.py",
  line 177

      with open(os.path.join(f"{sys.argv[3]}/ext/drjit/include/drjit/fwd.h")) as f:
                                                                          ^

  SyntaxError: invalid syntax

[Speierers]

Hi @nyue ,

It would be great if you could share the full CMake log. It looks like it is using a different Python to execute this script.

[nyue]

@Speierers you are right, it seems that cmake configuration is picking up a different python version (2.7.18) even though the default python is 3.8.10

-- The CXX compiler identification is GNU 9.4.0
-- The C compiler identification is GNU 9.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Mitsuba v3.0.2 
-- Mitsuba: setting build type to 'Release' as none was specified.
-- Mitsuba: targeting the native CPU architecture (specify MI_NATIVE_FLAGS to change this).
-- Found Python: /usr/bin/python2.7 (found version "2.7.18") found components: Interpreter Development Development.Module Development.Embed 
-- Looking for C++ include pybind11/pybind11.h
-- Looking for C++ include pybind11/pybind11.h - not found
-- pybind11 v2.8.0 dev2
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
CMake Error at CMakeLists.txt:137 (message):
  Could not run resources/configure.py script: File
  "/home/nyue/projects/mitsuba/mitsuba3/mitsuba3_git/resources/configure.py",
  line 177

      with open(os.path.join(f"{sys.argv[3]}/ext/drjit/include/drjit/fwd.h")) as f:
                                                                          ^

  SyntaxError: invalid syntax



-- Configuring incomplete, errors occurred!

How can we hint to cmake make to use python3 in Mitsuba's cmake configuration ?

[Speierers]

The Python installation discovery is handled to Pybind11. Check the following links to better understand how this is done:

https://pybind11.readthedocs.io/en/stable/compiling.html#findpython-mode

https://cmake.org/cmake/help/latest/module/FindPython.html

[nyue]

I fixed it by using

	  -D Python_EXECUTABLE=/usr/bin/python3

[nyue]

After the success with configuration, I proceeded to build, almost till the end, I encountered this compilation error

[1122/1129] Generating ../../python/drjit/__init__.pyi
FAILED: python/drjit/__init__.pyi /home/nyue/projects/mitsuba/mitsuba3/build/python/drjit/__init__.pyi 
cd /home/nyue/projects/mitsuba/mitsuba3/build/src/python && /home/nyue/systems/cmake/3.23.2/bin/cmake -E env PYTHONPATH=/home/nyue/projects/mitsuba/mitsuba3/build/python: /usr/bin/python3 -Xutf8 /home/nyue/projects/mitsuba/mitsuba3/mitsuba3_git/ext/drjit/resources/generate_stub_files.py /home/nyue/projects/mitsuba/mitsuba3/build/python/drjit
Traceback (most recent call last):
  File "/home/nyue/projects/mitsuba/mitsuba3/mitsuba3_git/ext/drjit/resources/generate_stub_files.py", line 347, in <module>
    import drjit as dr
  File "/home/nyue/projects/mitsuba/mitsuba3/build/python/drjit/__init__.py", line 11, in <module>
    import drjit.config as _config
ModuleNotFoundError: No module named 'drjit.config'
[1123/1129] Building CXX object src/python/CMakeFiles/mitsuba_ext.dir/__/render/python/phase.cpp.o
[1124/1129] Building CXX object src/python/CMakeFiles/mitsuba_ext.dir/__/render/python/spiral.cpp.o
[1125/1129] Building CXX object src/python/CMakeFiles/mitsuba_ext.dir/__/render/python/film.cpp.o
[1126/1129] Linking CXX shared module python/mitsuba/mitsuba_cuda_ad_rgb_ext.cpython-38-x86_64-linux-gnu.so
[1127/1129] Linking CXX shared module python/mitsuba/mitsuba_llvm_ad_rgb_ext.cpython-38-x86_64-linux-gnu.so
ninja: build stopped: subcommand failed.

[njroussel]

Once you reach the build error, can you perform the following steps for me please:

1. Can you confirm that the file mitsuba3/ext/drjit/drjit/config.py exists? What about your_build_folder/python/drjit/config.py
2. Does it build successfully if you run the ninja command again? (Without cleaning the existing build in between).

[nyue]

(1) Can you confirm that the file mitsuba3/ext/drjit/drjit/config.py exists?
YES
(2) What about your_build_folder/python/drjit/config.py
NO
(3) Does it build successfully if you run the ninja command again? (Without cleaning the existing build in between).
NO

Ninja output from running it again without cleaning the existing build

-- Build files have been written to: /home/nyue/projects/mitsuba/mitsuba3/build
[2/104] Generating ../../python/drjit/__init__.pyi
FAILED: python/drjit/__init__.pyi /home/nyue/projects/mitsuba/mitsuba3/build/python/drjit/__init__.pyi 
cd /home/nyue/projects/mitsuba/mitsuba3/build/src/python && /home/nyue/systems/cmake/3.23.2/bin/cmake -E env PYTHONPATH=/home/nyue/projects/mitsuba/mitsuba3/build/python: /usr/bin/python3 -Xutf8 /home/nyue/projects/mitsuba/mitsuba3/mitsuba3_git/ext/drjit/resources/generate_stub_files.py /home/nyue/projects/mitsuba/mitsuba3/build/python/drjit
Traceback (most recent call last):
  File "/home/nyue/projects/mitsuba/mitsuba3/mitsuba3_git/ext/drjit/resources/generate_stub_files.py", line 347, in <module>
    import drjit as dr
  File "/home/nyue/projects/mitsuba/mitsuba3/build/python/drjit/__init__.py", line 11, in <module>
    import drjit.config as _config
ModuleNotFoundError: No module named 'drjit.config'
[4/104] Linking CXX shared library libembree3.so
ninja: build stopped: subcommand failed.

[njroussel]

That's surprising. You can maybe manually copy the config.py file to that location, that should at least get you unstuck for now.

Are you using the latest master commit? If you have this following line there shouldn't really be a reason why the file isn't copied to your build folder.

[nyue]

My bad. I wasn't on the latest master.

Git pull and all building fine now.

Apologies for the noise.