MatrixAlgebraKit support tracker

[lkdvos]

Summarizing some of the ideas and discussions around #230

• Have the MatrixAlgebraKit functions overloaded and working for AbstractTensorMap, more or less the way it is implemented right now.
• The specializations for DiagonalTensorMap would ideally not be needed, and moved into MatrixAlgebraKit
• The old functions leftorth, rightorth and tsvd could remain in their current form, but have deprecation warnings and simply pass on to the new ones
• The old structs SDD, QR, ... might just have to be removed, since I'm not sure what the correct way is to deprecate a struct
• The old truncation schemes might have to be removed?
• Separate out the factorization tests from the regular tensor tests, and have a bit more of them. Currently we are testing various cases of 5-leg tensors, which is both expensive because they become quite large, and very inconvenient to write new tests with these exact same spaces. In principle, there are some edge cases (empty tensors, 1 leg tensors, etc) that are relevant, and otherwise none of this really depends on the number of legs, so we can probably make these tests both lighter and cover more grounds
• At some point we would probably also have to actually investigate some multithreading for all of these things, but I'm not sure if we want to immediately add this, since its a very annoying thing to handle: for many cases BLAS is just very efficient with its threading, so it is very easy to make things slower by adding more multithreading, and the actual good approach depends heavily on the distribution of block sizes
• Update the docs to reflect all of the changes
• Test truncated eigenvalue decomposition, and check if this works with autodiff

[lkdvos]

@kshyatt pinging you here

[lkdvos]

Addendum for truncation schemes:

• lvl 1: supporting dense truncated decompositions
• lvl 2: supporting abelian truncated decompositions. This effectively means (but doesn't have to be implemented as such) that we have a bunch of individual vectors of values, we want to truncate them together, and then re-scatter that information back to the individual vectors to truncate these accordingly.
• lvl 3: supporting generic truncated decompositions. This is similar to the abelian case, but additionally has to include some kind of weight for each of the vectors of values.

Importantly, the difficulty depends quite a lot on the chosen truncation scheme, in the sense that there are schemes that require global information, and others that can simply be implemented block-by-block. Also there are schemes that do/don't depend on the weights, and the actual value of the weight additionally depends on the scheme as well.

The main focus should probably be on the following truncation schemes, which are the ones that were previously supported and are more or less the same now (with the minor addition of having relative tolerances as well):

• TruncationKeepSorted: keep the first howmany values, sorted by some metric
• TruncationKeepAbove(atol, rtol, p, by) and TruncationKeepBelow(atol, rtol, p, by): keep the values before or after a threshold value atol, typically by=abs (which is probably enough for now)
• TruncationError (currently not in MatrixAlgebraKit, see TensorAlgebra's implementation: truncate as many values as possible until the error in the factorization exceeds some threshold
• TruncationSpace: TensorKit-specific: keep x values in each sector