Conventions for LinearTransformation (right-multiply vs left-multiply)

[beneisner]

Hi,

I was wondering if the conventions for the LinearTransformation class are. Specifically, how the linear transformation is actually applied.

In many settings, I'm familiar with the standard pre-multiplication convention for a rotation. For instance, if I have matrices

R = [ b1, b2, b3 ]  # 3x3 matrix with basis vectors
t = [ [a1], [a2], [a3] ]  # dx1 vector, i.e. a point in a pointcloud

then I would expect that the linear transformation of this vector t would be:

t_prime = R @ t
or
t_prime = torch.matmul(R, t)

Both of these yield dx1 matrices.

Understandably, when things are batched, you might want to use matmul slightly differently to maintain shapes. In the LinearTransformation class, data.pos has a shape of [N x d], where N is the number of points and d is the dimensionality of the points (d=2 or d=3, typically). When applying the transformation, the code in LinearTransformation applies:

# Truncated for simplicity...
data.pos = torch.matmul(pos, matrix)

which does in fact yield a [N x d] matrix, since matrix is square. However, this is a different operation than the convention I mentioned in the first example. In fact, if matrix is a rotation matrix (and thus orthonormal), it applies the inverse transformation (since R^(-1) = R.T). Modifying the above line to:

data.pos = torch.matmul(pos, matrix.T)

would put the result in line with my expectation, since it's equivalent to:

data.pos = (R @ pos.T).T

My question is: what is the intended convention in LinearTransformation? Because when consumers of LinearTransformation construct their rotation matrices, they seem to be constructing them in a way that expects the "left-multiply" convention I described (for instance, RandomRotation constructs rotations using standard left-multiply rotation matrix conventions).

Thanks again for a great library :)

[rusty1s]

Thanks for this issue. I think that you are right that it's more intuitive to first transpose the matrix before using it, e.g.:

self.matrix = matrix.t()

We use this linear transformation mostly for augmentation, in which it does not matter whether to apply the correct or inverse transformation. Nonetheless, I think we should fix this. Are you interested in contributing this?

[beneisner]

Ok, sounds good. Just submitted a PR #2777.