Shading frame n, s, t is all nan

[tomas16]

Hi, I'm tracing some rays in a loop. Below code is the core of the loop body:

si = scene.ray_intersect(ray, mi.RayFlags.ShadingFrame, coherent=False, active=active)
active &= si.is_valid()

# Refract and generate ray for next iteration
costi = si.wi.z
_, costt, _, eta_ti = mi.fresnel(costi, eta)
raydir_local = mi.refract(si.wi, costt, eta_ti)
raydir = si.to_world(raydir_local)
ray = si.spawn_ray(raydir)

I have one case where everything essentially turns into nan. I've traced the problem to si.sh_frame.n, si.sh_frame.s and si.sh_frame.t being all nan, so in turn si.wi and costi are nan, costt is 0, raydir_local is [nan, nan, 0] and raydir is [nan, nan, nan].

I used si.prim_index to find the vertex normals of the face that's being intersected, and they are:

In [7]: vertex_normals
Out[7]: 
array([[ 0.01501265, -0.94263654, -0.33348309],
       [-0.00228912, -0.96046938, -0.27837622],
       [-0.00276447, -0.91487202, -0.40373449]])
In [8]: np.linalg.norm(vertex_normals, axis=1) - 1
Out[8]: array([ 0.00000000e+00, -2.22044605e-16,  0.00000000e+00])

My understanding is that the shading normal si.sh_frame.n is constructed by interpolating between these vertex normals, and si.sh_frame.s and si.sh_frame.t are then constructed by finding perpendicular vectors.

Why would these shading frame basis vectors be nan if the vertex normals they're interpolated/constructed from appear correct?

[njroussel]

Hi @tomas16

Nothing obvious jumps to my mind.
What shape are you intersecting? A triangle mesh?

[njroussel]

Nevermind, if you have vertex_normals it's a triangle mesh...

My only suggestion would be to compile the project yourself and print the intermediate variables in Mesh::compute_surface_interaction (https://github.com/mitsuba-renderer/mitsuba3/blame/master/src/render/mesh.cpp#L792-L835).

You'll need to turn off megakernel-mode:

dr.set_flag(dr.JitFlag.LoopRecord, False)
dr.set_flag(dr.JitFlag.VCallRecord, False)

[tomas16]

Thanks for pointing me in that direction.

I went and looked at that function and implemented the steps it uses to compute the shading normal in my python code, after saving the relevant data output from drjit.

# prim_uv is 1x2
b1 = prim_uv[0, 0]
b2 = prim_uv[0, 1]
b0 = np.float32(1) - b1 - b2
b = np.array([b0, b1, b2]).reshape((3, 1))

# shading normal
sh_n = np.sum(vertex_normals * b, axis=0)
sh_n /= np.linalg.norm(sh_n)

Still nothing out of the ordinary:

In [39]: vertex_normals
Out[39]: 
array([[ 0.01501265, -0.94263654, -0.33348309],
       [-0.00228912, -0.96046938, -0.27837622],
       [-0.00276447, -0.91487202, -0.40373449]])
In [40]: b
Out[40]: 
array([[0.1013805],
       [0.6243217],
       [0.2742978]], dtype=float32)
In [41]: sh_n
Out[41]: array([-6.66595788e-04, -9.47788667e-01, -3.18898415e-01])

Also computed dp0 and dp1 and the angle between them

In [53]: # vertices is 3x3
    ...: dp0 = vertices[1, :] - vertices[0, :]
    ...: dp1 = vertices[2, :] - vertices[0, :]
    ...: np.arccos(np.sum(dp0 * dp1)) / np.pi * 180
Out[53]: 91.253565309657

This does appear to be a very elongated triangle, but the face normal gets computed correctly and I don't think there really should be an issue with the triangle itself.

Unfortunately I have to prioritize some other work before I can dive much deeper into this. I'm also aware that my meshes have some issues that need to get solved, so part of me is hoping that changing the meshes may fix this problem. Though based on the above I don't see a reason to be too hopeful about that.

I think the other thing that may be interesting is to try different variants (the scalar one!) of mitsuba, just to make sure that the drjit/llvm interaction isn't causing any of this. Obviously in the above I used python to rule out obvious issues, but the computation that gets executed is ultimately different. But if I need to compile mitsuba myself, I may as well run it in the debugger as you suggested.