Documentation confusion between preconditioners and the problem being solved

[timholy]

Several methods---at least LSQR, LSLQ, LSMR, LNLQ, CRAIG, CRAIGMR, TriCG, and TriMR, and partially CR---conflate "preconditioner" with problem specification. (I believe it's fair to say that in conventional usage, "preconditioner" primarily refers to a device for speeding convergence, but it should not grossly change the nature of the solution.) For example, LSQR with $\lambda = 0$ minimizes $\lVert b - Ax \rVert_M$, where $\lVert v \rVert_M^2 = v^TMv$ for positive-definite $M$; in the documentation M is described first and foremost as a preconditioner, but it's actually a core part of the problem specification: you may get different answers for x given different M even if b and A remain unchanged.

julia> A = [1.0 0; 0 1; 1 1]; b = [1.0, 1, 0];

julia> lsqr(A, b)
([0.33333333333333337, 0.33333333333333337], SimpleStats
 niter: 1
 solved: true
 inconsistent: true
 indefinite: false
 npcCount: 0
 residuals: []
 Aresiduals: []
 κ₂(A): []
 timer: 22.30μs
 status: found approximate minimum least-squares solution
)

julia> lsqr(A, b; M = Diagonal([1e-8, 1, 1]))
([-0.9999999599999996, 0.9999999799999998], SimpleStats
 niter: 2
 solved: true
 inconsistent: true
 indefinite: false
 npcCount: 0
 residuals: []
 Aresiduals: []
 κ₂(A): []
 timer: 10.20μs
 status: found approximate minimum least-squares solution
)

(Stated another way, weighted least-squares returns solutions that in general depend on the weights.) Both of these answers are correct because they solve different problems, but to me this goes well beyond M serving as a mere preconditioner.

Moreover, there is consistent confusion about how M should be interpreted depending on whether ldiv = true or ldiv = false; in other words, the actual problem being solved depends on both M and the setting of ldiv. The documentation does not always fully clarify this point, and tends to make strong assumptions that ldiv=false.

Altogether, I think it's best to be clear about the problem specification first, and then describe how to construct arguments to solve that problem second.

I made some attempts to resolve this for TriCG and TriMR in #1043, but that was only for those two methods and that PR also brought in changes to warm-starting. If there is agreement by the maintainers that this documentation confusion needs to be resolved, I can submit a documentation-only PR that addresses a wider array of methods. (Please do list any I may have omitted.)

[amontoison]

I added the kwarg ldiv after the implementation of many methods. Only mul! was available before for the preconditioner.
It was the documentation tends to make a strong assumptions on ldiv=true.
I agree that the word "preconditioner" should be replaced by "weight" in LN / LS context.

I don't understand the problem with CR. We only have a centered preconditioning for the Lanczos process:
https://jso.dev/Krylov.jl/dev/preconditioners/

[timholy]

Sounds like we're on the same page, right? I might take a crack at fixing this. (I often think documentation is best polished by someone other than the code-author, but in practice I've rarely been the lucky recipient of that kind of help myself.)

I don't understand the problem with CR.

I included it only because of this line of the documentation:

M also indicates the weighted norm in which residuals are measured.

which presumably suffers from the same ldiv assumption?

[timholy]

Though I should point out that perhaps the right way to solve this problem also ropes in #1047, so perhaps I should wait on your response to that issue first.