Local transformations for Mitsuba shapes in XML

[mohanamisra]

I wish to rotate a shape, but the rotations are around the world origin, while I need the rotations around the object's centre.

In the transformation toolbox page in the docs, I do see a way to convert to/from the local frame. However, if I wish to automate the rotations to simulate different room scene configurations, I'd need to get the shape's coordinates, convert it to the local frame, apply the rotations I want, convert it back to the world frame and then apply it to the xml.

Is there any way to do it natively in XML?

I'd also be open to ideas that simply make this to-and-fro mapping less convoluted.

Thanks !

[njroussel]

Hi @mohanamisra

As far as I know, there isn't a built-in operation for this in XML. If you only want to edit the XML, I guess your only choice is to parse the world transform, then edit the XML to apply the inverse transform,, apply your rotation, and then apply the world transform again.

[mohanamisra]

Thanks for the reply @njroussel!

Could you please clarify what you meant by the steps you listed, or perhaps link to the relevant sections in the documentation for me to work through?

Also, what would the non-XML way to do it be? And can that be integrated with my scene.xml ? I don't have to programmatically alter anything in the scene apart from these rotations and now that I think about it maybe doing it in Python rather than XML would be better, especially if I can keep the rest of the XML scene intact.

[njroussel]

I'm guessing that in your XML you have to_world transforms that are described as raw matrices. My suggestion was to parse these with whatever XML tool you like, and then rewrite/replace them in the original XML after having computed the corrected (with your extra rotation) matrix.
This doesn't use any parts of mitsuba, this is just XML editing.

Assuming that your shape is a Mesh, the non-xml way would be to do these same steps, i.e undo initial wold transform then apply the rotation and re-apply the world transform, but using the mi.traverse() interface to update the mesh.vertex_positions.

[mohanamisra]

I have the shape defined as follows, to rotate around x axis by 60 degrees (as an example):

	<shape type="ply" id="elm__7" name="elm__7">
		<string name="filename" value="meshes/Mirror.ply"/>
		<boolean name="face_normals" value="true"/>
		<ref id="mat-Material.001" name="bsdf"/>
		<transform name="to_world">
			<matrix value="1.000000 0.000000 0.000000 0.000000
						  0.000000 0.500000 -0.866025 0.683013
						  0.000000 0.866025 0.500000 -0.183013
						  0.000000 0.000000 0.000000 1.000000"/>
		</transform>
		<sensor type="irradiancemeter"/>
	</shape>

I'm a bit unsure what you mean by

rewrite/replace them in the original XML after having computed the corrected (with your extra rotation) matrix

and

undo initial wold transform then apply the rotation and re-apply the world transform

I have been trying to translate my shape to the "world origin", rotate it and then translate it back to the original object centre. That hasn't worked unfortunately, as in it still rotates around the world origin. However, in that approach too, I just had one single rotation matrix that I was trying to work with, and no extra matrices.

EDIT: It worked. It was a problem with getting the centre of the shape accurately and then translating it to the origin, rotating and translating it back.

[mohanamisra]

I finally got it working, and the solution is relatively simple with the correct methods, order of operations and so on.

Here's the code for finding out the centre of the shape:

mirror_shape = mi.load_dict({
    'type': 'ply',
    'filename': '/<your file path>/Mirror.ply',
    'face_normals': True
})

# Get the bounding box of the shape
bbox = mirror_shape.bbox()

# Get the centre
center = bbox.center()

And then in XML you simply do the following:

        <transform name="to_world">
            <translate value="-2.17873, -1.79809, -0"/> # THE NEGATIVES OF YOUR CENTRE COORDINATES. MOVES TO ORIGIN
            <rotate value="1, 0, 0" angle="60"/> # EXAMPLE ROTATIONS
            <rotate value="0, 1, 0" angle="120"/>
            <rotate value="0, 0, 1" angle="30"/>
            <translate value="2.17873, 1.79809, 0"/> # MOVE IT BACK TO THE ORIGINAL OBJECT CENTRE.
        </transform>

To automate it, I'll probably use an XML parsing/editing solution like @njroussel mentioned.

[mohanamisra]

Note that unless you really need to update the XML files, @njroussel's suggestion of applying the transformation after loading the scene will be much more convenient

My shape is a .ply file, and I'm using load_dict to load it, applying a to_world property to it with the transformations chained as described in the Transformation Toolbox in the docs.

However, after that I am currently writing it to the disk and then simply making sure the XML file uses the updated .ply next time around. This doesn't feel very efficient and I went through the Mesh I/O and manipulation section for help, but I'm not sure how to use mi.traverse() and mesh.vertex_positions() to update the .ply file so that it doesn't have to generate a new file and use it for rendering?

Here's a snippet of what I'm doing currently:

new_mirror = mi.load_dict({
    'type': 'ply',
    'filename': 'meshes/Mirror.ply',
    'face_normals': True,
    "to_world": mi.ScalarTransform4f().translate(mi.ScalarPoint3f([2.17873, 1.79809, 0])).rotate(mi.ScalarPoint3f([0, 0, 1]), angle=60).translate(mi.ScalarPoint3f([-2.17873, -1.79809, -0])),
})

new_mirror.write_ply('meshes/new_mirror.ply')

[merlinND]

You would just load your scene with mi.load_file("your_xml.xml"), the get the scene properties from params = mi.traverse(scene), and apply the transformation to your mesh's vertices in e.g. params["my_mesh.vertex_positions"]. Finally, call params.update() to apply the changes.

https://mitsuba.readthedocs.io/en/stable/src/rendering/editing_a_scene.html

[mohanamisra]

I attempted the same and got the following error:

RuntimeError: dtype.unravel(): input array size is not divisible by 'dtype' shape!

My input array (which is the element's vertex positions, viz. elm__7.vertex_positions for me) size is 42. What shape is mi.Point3f expecting, for this error to be resolved?

[merlinND]

The resulting Point3f should have width mesh.vertex_count().

[mohanamisra]

Okay, the docs only show an example for translation.
If I follow the Transformation Toolbox section as described by the Editing a scene, I end up with the following code:

scene = mi.load_file("ver_2.xml")
params = mi.traverse(scene)
V = dr.unravel(mi.Point3f, params['elm__7.vertex_positions'])

# TRANSLATE TO ORIGIN -> ROTATE -> TRANSLATE BACK
V_transformed = mi.Transform4f().translate(mi.Point3f([2.17873, 1.79809, 0])).rotate(mi.Point3f([0, 1, 0]), angle=60).translate(mi.Point3f([-2.17873, -1.79809, -0]))

After which I'm doing the following on a hunch only (unfortunately). There are no errors but it isn't modifying the scene when I re-render.

params['elm__7.vertex_positions'] = dr.ravel(V_transformed @ V)
params.update()

Am I supposed to use some other way to transform the vertex positions?

EDIT: Nvm, it works! To check I need to re-render it in the same file rather than loading the XML again from scratch. Thanks a lot for the guidance and help.