Double check nuclear norm LMO / add capped SDP cone

[pokutta]

TODO

• validate nuclear norm LMO and add clean unit tests (add also various versions: ARPACK, iterative solvers, etc)
• add capped SDP cone from CGSDP.jl
• add block-LMO from SDP cone

specifics:

Nuclear norm LMO implementation needs to be checked and also:

• support various implementations of the svds: ARPACK / iterative solvers just to name two

Baseline / working function for the nuclear norm:

###############################
# nuclear norm


function compute_lmo_factor_nuclear_norm(M, tol=1e-8)
    # println("M: $M")
    Z = Arpack.svds(M, nsv=1, tol=tol)[1]
    # println("Z.U: $(Z.U)")
    # println("Z.V: $(Z.V)")
    # println("Z.Vt: $(Z.Vt)")
    u = view(Z.U, :)
    rank_one_v = FrankWolfe.RankOneMatrix(u, - Z.Vt[:])
    # println("Rank-1 factor: $rank_one_v")    

    return rank_one_v
end

Unit test that should work

M = [2 0; 0 1]
F = compute_lmo_factor_nuclear_norm(M)
println(eigvals(Matrix(F)))

M = [0 0 1; 2 0 0; 0 -3 0]
F = compute_lmo_factor_nuclear_norm(M)
println(eigvals(Matrix(F)))

And add capped SDP cone. Should also support ARPACK and iterative solvers etc. in particular we should use LOBPCG for this one as these are psd matrices. See also CGSDP.jl for three implementations that we should move into the main package.