Question about QR decomposition

[Pixie8888]

Hi,

I have a question regarding the orthogonal basis that is obtained by QR decomposition. The QR decomposition is defined like this:

Why did you not take the whole "Q" as the basis but only use "P_C^(0)"?

[feyzaakyurek]

Hi @Pixie412,

Thanks for your interest in our work!

$Q'$ contains the basis for the nullspace of the $\eta_{C^{(0)}}$ while we are interested in the column space basis for the subspace spanned by the base class weights which is $P_{C^{(0)}}$. $P_{C^{(0)}}$ also corresponds to $Q$ in reduced QR decomposition [1,2]

[1] Trefethen, L. N., & Bau III, D. (1997). Numerical linear algebra (Vol. 50). Siam.
[2] Terao, T., Ozaki, K., & Ogita, T. (2020). LU-Cholesky QR algorithms for thin QR decomposition. Parallel Computing, 92, 102571.

[Pixie8888]

Hi,
Why not using SVD to get the left/right singular vectors as the basis?

[ekinakyurek]

Hi @Pixie412,

I am sorry for the late reply, we have missed the notification.

Yes, you can also use SVD to find the same orthogonal projection; they are equivalent. Please see the equivalence in the code below. Please let us know if you have further questions.

Let me also clarify our paper's specific notation: we concatanate our base representations into matrix $A \in R^{512 \times 64}$. Then, run the reduced QR $$A^{512 \times 64} = Q_r^{512 \times 64} R_r^{64 \times 64}$$ $Q_r=P_{C^{(0)}}$ in our paper. So, our actual projection matrix is $P=Q_rQ_r^{\top}$

As you suggested, reduced left singular matrix, $U_r$ in reduced SVD, can be used to calculate the same projection:
$$A = U_r^{512 \times 64}\Sigma_r^{64 \times 64} V_r^{*64 \times 64} $$ and you can calculate the same orthogonal projection $P = U_rU_r^{\top}$

Test:

julia> using LinearAlgebra

julia> A = randn(512, 64);

julia> function find_projection_with_QR(A)
           F = qr(A) # full QR
           Qr = F.Q[:, 1:size(A, 2)] # reduced Q
           return Qr * transpose(Qr)
       end
find_projection_with_QR (generic function with 1 method)

julia> function find_projection_with_SVD(A)
           F = svd(A; full=true) # full SVD
           Ur = F.U[:, 1:size(A, 2)] # reduced U
           return Ur * transpose(Ur)
       end
find_projection_with_SVD (generic function with 1 method)

julia> find_projection_with_QR(A) ≈ find_projection_with_SVD(A)
true