Normal Maps DX or GL orientation?

[Disgruntoad]

Hey, I'm not sure whether mitsuba uses the dx or gl orientation for normal maps. Which one is correct?

Thanks!

[shinyoung-yi]

UV conventions in Direct X and OpenGL

In my understanding, V-axis of UV coordinates indicates top-to-bottom direction in texture images in DX but bottom-to-top in OpenGL right?

• https://www.gamedev.net/forums/topic/682989-texture-mapping-coordinates-for-opengl-and-directx/

Then regarding "red" and "green" in normal maps as U and V axes and according to that convention, positive "green" values in normal maps indicate -Y and +Y in an object coordinates for DX and OpenGL, respectively.

• https://www.texturecan.com/post/3/DirectX-vs-OpenGL-Normal-Map/

TL;DR

Mitsuba 3 seems to follow Direct X's convention.

In Mitsuba 3?

The convention for UV coordinates of textures

For test, I will use the following texture image:
dummy_texture.png

Using the following script, we observe that Mitsuba 3 regards the U-axis as the top-to-bottom direction, similar to Direct X

import mitsuba as mi
mi.set_variant('cuda_ad_rgb', 'llvm_ad_rgb')

scene_dict = mi.cornell_box()
scene_dict.pop('floor')
scene_dict.pop('ceiling')
scene_dict.pop('back')
scene_dict.pop('green-wall')
scene_dict.pop('red-wall')
scene_dict.pop('small-box')
scene_dict.pop('large-box')
scene_dict['plane'] = {'type': 'rectangle',
                       'bsdf': {'type': 'diffuse',
                                'reflectance': {'type': 'bitmap',
                                                'filename': 'dummy_texture.png'}},
                       }
scene_dict['light']['to_world'] = mi.ScalarTransform4f([[-1, 0, 0, 0],
                                                        [0, 1, 0, 0],
                                                        [0, 0, -1, 5],
                                                        [0, 0, 0, 1]])
scene = mi.load_dict(scene_dict)

img = mi.render(scene)
mi.Bitmap(img)

Note that we can also check that UV = (XY+1)/2 at each vertex of the rectangle.

one = 1-1e-4
ray = mi.Ray3f(mi.Point3f([-one, -one, one, one],
                          [-one, one, -one, one],
                          [one, one, one, one]), mi.Vector3f([0, 0, -1]))
si = scene.ray_intersect(ray)
print(f"{si.p = }")
print(f"{si.uv = }")

# Outputs
# si = SurfaceInteraction[
# <omitted>
#   p = [[-0.9999, -0.9999, 0],
#        [-0.9999, 0.9999, 0],
#        [0.9999, -0.9999, 0],
#        [0.9999, 0.9999, 0]],
#   shape = [0x27384cd1dd0, 0x27384cd1dd0, 0x27384cd1dd0, 0x27384cd1dd0],
#   uv = [[5.00083e-05, 5.00083e-05],
#         [5.00083e-05, 0.99995],
#         [0.99995, 5.00083e-05],
#         [0.99995, 0.99995]],
# <omitted>

The convention for normal map values for RGB textures

From this UV convention, we can guess that normal map will be also evaluated in the Direct X's convention.
The normal map data (https://github.com/mitsuba-renderer/mitsuba-data/blob/1a5732ba82b0fdc92bb014be4f57c26342d11739/common/textures/floor_tiles_normalmap.jpg) included in Mitsuba 3 also seems to regard "green" color as the top-to-bottom direction in the image space.

For testing, first I saved two normal map images from https://www.texturecan.com/post/3/DirectX-vs-OpenGL-Normal-Map/ , which introduce the conventions for Direct X and OpenGL. I saved the images as normalmap_dx.png and normalmap_gl.png, resp.
Using the following script:

file_normal = "normalmap_dx.png"
# file_normal = "normalmap_gl.png"
scene_dict['plane']['bsdf'] = { # `scene_dict` from the previous script
    'type': 'normalmap',
    'normalmap': {
        'type': 'bitmap',
        'raw': True,
        'filename': file_normal},
    'bsdf': {'type': 'roughplastic'}
}
scene_dict['cube'] = {'type': 'cube',
                      'to_world': mi.ScalarTransform4f.scale(0.1),
                      'bsdf':{'type': 'diffuse',
                              'reflectance': {'type': 'rgb', 'value': [0, 1, 1]}}}
scene_dict['light']['to_world'] = mi.ScalarTransform4f([[-1, 0, 0, 3.5],
                                                        [0, 1, 0, 3.5],
                                                        [0, 0, -1, 5],
                                                        [0, 0, 0, 1]])
scene_dict['light']['emitter']['radiance']['value'] = [40, 32, 21] # [18.387, 13.9873, 6.75357]
scene_dict['sensor']['film']['width'] = 1200
scene_dict['sensor']['film']['height'] = 600
scene_dict['sensor']['fov'] = 16

scene = mi.load_dict(scene_dict)
img = mi.render(scene)
mi.Bitmap(img)

Mitsuba 3 produces the following images
Normal map texture: normalmap_dx.png

Normal map texture: normalmap_gl.png

We finally observe that the normal map shading of the rendered image with normalmap_dx.png is consistent with the direction of the shadow of the cube object, but the one with normalmap_gl.png is not.

[Disgruntoad]

Thank you very much for the detailed explanation and example!