Change tolerances to a more realistic value for the CI tests

[ktbolt]

Use Case

The tolerances used in the CI tests are unrealistically small, typically 1e-12.

Problem

There is no valid reason for having such small tolerances. And we have had multiple instances of users using the CI tests as templates for their own simulations. This has caused those simulations to fail with convergence problems which in turn causes confusion in the user community.

Solution

I will change all of the tolerances in the CI tests to be more representative of the tolerances used in real simulations.

Alternatives considered

none

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct and Contributing Guidelines

[dcodoni]

@ktbolt I agree on this and also I would like to point out that for LV_HolzapfelOgdenModifiedAnisotropy_passive
we are setting relative tolerance for the linear solver to 1e-16 which is less than the default absolute tolerance. I do not even know if numerically we can reduce residual 16 order of magnitudes less. I observed that neither fsils nor trilinos can reach this level.
I agree on having some tight tolerances to push the solver to a certain limit, but I think we should keep this limit in the 'realistic' range.

[mrp089]

Two reasons for tight testing tolerances are:

• Avoid problems when running the test on different machines or with multiple processors (Parallel simulations no longer work #62, Can't run simulations on Apple Silicon #92)
• Highlight convergence issues in our code (Slow nonlinear convergence in ustruct/LV_Guccione_active #197, Slow convergence in porous media flow #357, Slow nonlinear convergence when using URIS valve in fluid equation #448)

I agree that unachievable linear or nonlinear solver tolerances don't make sense. I outlined the steps to set up the tests in our testing guide. If you come up with a better strategy, you should update the steps for creating a new test.

I prefer adding additional "real-world" examples to the repository over changing our integration tests. These can be linked in the main README, include documentation of the simulation setup and results, and can be sourced from the SimVascular examples that are uploaded elsewhere.

[dcodoni]

@mrp089 I also noticed few things about the linear solver tolerances:

• the tolerances of the linear solver are not checked. I looked a bit at the most recent successful Ubuntu tests and many tests are not reaching desired convergence for linear solver (maxing out iterations) and some such as LV_Guccione_passive have this type of linear convergence (relative tolerance set to 1e-14 only satisfied by the first non-linear iterations):
  ST 1-1 1.350e-01 [0 1.000e+00 1.000e+00 8.271e-14] [152 -6 6]
  ST 1-2 2.690e-01 [-55 1.761e-03 1.761e-03 6.266e-11] [130 -3 5]
  ST 1-3 3.980e-01 [-197 1.337e-10 1.337e-10 9.105e-04] [41 -2 2]
  ST 1-4s 5.280e-01 [-209 3.251e-11 3.251e-11 3.869e-03] [21 -1 1]
• the default absolute tolerance is 1e-10, so if the tolerances are 1e-14 or 1e-16, it may happen the absolute tolerance rules (almost no tests explicitly set absolute tolerance)
• the amount of screen outputs: [Vector] WARNING: Index checking is enabled, makes it extremely hard to read the tests output.

So even if we have strict tolerances (1e-14 or 1e-16 should not be used in my opinion), I am wondering how many times in the tests we actually reached them until now.

[ktbolt]

@dcodoni I mentioned checking tolerances of the linear solver here #403

It is good to understand out how absolute tolerance and tolerance interact.

We should add a bool variable to print the WARNING: Index checking message just once.

[mrp089]

I agree to change nonsensical linear tolerances to something reasonable. Also, #403 sounds like a good idea (if the histor.dat outputs are stable over several runs)!

Can we change the warning to appear only once? We should also add A to our pytest options to list all tests (passed and failing) at the end (currently, you have to go through the whole log).

@ktbolt, do you also want to change nonlinear tolerances?