Trilinos refactoring and updating (SimVascular#424)

* Trilinos updated:
* main packages updates: Epetra->Tpetra; AztecOO->Belos; Ifpack->Ifpack2; ML->MueLu
* refactor linear algebra backend to use Teuchos and Kokkos:
- wrap Tpetra vectors and matrices with Teuchos::RCP for safer memory management and automatic reference counting.
- use Kokkos (via Tpetra::Node/KokkosDeviceWrapperNode) to abstract parallel execution (same code to run efficiently on CPU and GPU theoretically)
* removed all static objects
* change of IC/ICT preconditioners in favor of RILUK(0)/RILUK(1)

* Updating dockerfiles:
* removed ubuntu20 and ubuntu22 folders. Created ubunut/ folder that contains dockerfile based on ubuntu-latest
* modified the solver/dockerfile to use the ubuntu-latest docker container as a base

* Updated dockerfiles and workflow file to use the latest docker container:
* simvascular/libraries:latest is the image containing the latest trilinos built

* Update the input files of trilinos related test cases for fluid and fsi
* increased the maximum iteration number for linear solver
* add the fluid case iliac artery with ml preconditioner

* Modifications and improvements:
* kokkos::finalize has been moved inside the TrilinosImpl class, fixing the compilation errors whenever trilinos is not used
* trilinos linear solver residual test has been properly set, fixing first non-linear iteration problem of maxing out the linear iterations reducing excessively and unnecessarily the residual

* Updating Trilinos packages list in README.md

* Final update on dockerfile

* Fix buffer overflow in genBC_Integ_X (set_bc.cpp):
* string length was computed incorrectly, leading to silent buffer overflow.
* this only surfaced as a critical memory crash when linking with the latest Trilinos libraries.

* Changes to CMake to fix coverage and unit test mismatch

* Modifying cmake file t ofix the coverage error

* Fix coverage target for GCC (lcov/geninfo):
- Added --ignore-errors mismatch to avoid geninfo line mismatch errors
- Added --rc geninfo_unexecuted_blocks=1 to silence GCC warnings

* fix coverage: correct path expansion in 'lcov --remove ' command

* Removing the unit tests from coverage

* Refactor CMake: unify solver build, exclude unit tests from coverage
* Built solver sources into a shared object library (solver_objs)
  so they can be reused by both the main executable and unit tests.
  This avoids recompiling solver sources twice when ENABLE_UNIT_TEST
  is enabled.
* Excluded unit tests from code coverage by removing coverage flags
  from the run_all_unit_tests target (-fno-profile-arcs, -fno-test-coverage).
  This prevents .gcda files from being generated for test code,
  ensuring coverage only reflects solver sources.

* Removing the unit tests folder '*/CMakeFiles/run_all_unit_tests.dir/*'
from the coverage.

* Remove '*/CMakeFiles/run_all_unit_tests.dir/*' from the coverage list

* Adding the following flags in the Cmake file under the enable_coverage section:
* --ignore-errors mismatch --rc geninfo_unexecuted_blocks=1
* --ignore-errors unused
These flags allowed to save a coverage file even in presence of errors of mismatch type.
Fix some parametres setting in the Trilinos GMRES solver: krylov space, max iteration and max restarts.

* Updates in new trilinos implementation:
* add timer for the linear solver in Trilinos
* add the possibility to read the MueLu Options for the ML preconditioner from file mueluOptions.xml
  if file is not present built-in parameters are used

* Using fsils for the mesh equation in FSI test cases

* Fixed allocation problem in the graph creation:
* the localt to global map was created in sorted order, while the graph allocated in unsorted order
* the graph is now created using the sorted order like the map

Author: dcodoni

.github/workflows/tests.yml

@@ -39,7 +39,7 @@ jobs:
   test-ubuntu:
     name: Test Ubuntu
     runs-on: ubuntu-latest
-    container: simvascular/libraries:ubuntu22
+    container: simvascular/libraries:latest
     needs: check-for-changes
     if: ${{ needs.check-for-changes.outputs.run_full_tests == 'true' }}
     steps:

Code/Source/solver/CMakeLists.txt

@@ -290,9 +290,9 @@ if(ENABLE_COVERAGE)
   # add coverage target
   add_custom_target(coverage
     # gather data
-    COMMAND ${LCOV} --directory . --capture --output-file coverage.info
+    COMMAND ${LCOV} --directory . --capture --output-file coverage.info --ignore-errors mismatch --rc geninfo_unexecuted_blocks=1
     # exclude externals
-    COMMAND ${LCOV} --remove coverage.info -o coverage.info '/usr/*' '/opt/*' '${PROJECT_SOURCE_DIR}/ThirdParty/*' '/Library/*' 'v1/*'
+    COMMAND ${LCOV} --remove coverage.info -o coverage.info '/usr/*' '/opt/*' "${PROJECT_SOURCE_DIR}/ThirdParty/*" '/Library/*' 'v1/*' --ignore-errors unused
     # generate report
     COMMAND ${GENHTML} --demangle-cpp -o coverage coverage.info
     WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
@@ -371,5 +371,4 @@ if(ENABLE_UNIT_TEST)
   # gtest_discover_tests(runUnitTest)
   add_test(NAME all_unit_tests COMMAND run_all_unit_tests)
 
-endif()
-
+endif()

Code/Source/solver/FsilsLinearAlgebra.cpp

@@ -79,6 +79,12 @@ void FsilsLinearAlgebra::initialize(ComMod& com_mod, eqType& lEq)
   // Nothing is needed to initialize FSILS.
 }
 
+/// @brief Finalize framework.
+void FsilsLinearAlgebra::finalize()
+{
+  // Nothing is needed to finalize FSILS.
+}
+
 /// @brief Set the linear algebra package for assmbly.
 void FsilsLinearAlgebra::set_assembly(consts::LinearAlgebraType atype)
 {

Code/Source/solver/FsilsLinearAlgebra.h

@@ -23,6 +23,7 @@ class FsilsLinearAlgebra : public virtual LinearAlgebra {
     virtual void solve(ComMod& com_mod, eqType& lEq, const Vector<int>& incL, const Vector<double>& res);
     virtual void set_assembly(consts::LinearAlgebraType atype);
     virtual void set_preconditioner(consts::PreconditionerType prec_type);
+    virtual void finalize();
 
   private:
     /// @brief A list of linear algebra interfaces that can be used for assembly.

Code/Source/solver/LinearAlgebra.h

@@ -27,7 +27,8 @@ class LinearAlgebra {
     virtual void set_assembly(consts::LinearAlgebraType assembly_type) = 0;
     virtual void set_preconditioner(consts::PreconditionerType prec_type) = 0;
     virtual void solve(ComMod& com_mod, eqType& lEq, const Vector<int>& incL, const Vector<double>& res) = 0;
-
+    virtual void finalize() = 0;
+    
     virtual consts::LinearAlgebraType get_interface_type() { return interface_type; }
 
     consts::LinearAlgebraType interface_type = consts::LinearAlgebraType::none;

Code/Source/solver/PetscLinearAlgebra.cpp

@@ -90,6 +90,11 @@ void PetscLinearAlgebra::initialize(ComMod& com_mod, eqType& lEq)
   impl->initialize(com_mod, lEq);
 }
 
+void PetscLinearAlgebra::finalize()
+{
+  // No need to finalize PETSc.
+}
+
 /// @brief Initialize an FsilsLinearAlgebra object used for assembly.
 void PetscLinearAlgebra::initialize_fsils(ComMod& com_mod, eqType& lEq)
 {

Code/Source/solver/PetscLinearAlgebra.h

@@ -22,6 +22,7 @@ class PetscLinearAlgebra : public virtual LinearAlgebra {
     virtual void solve(ComMod& com_mod, eqType& lEq, const Vector<int>& incL, const Vector<double>& res);
     virtual void set_assembly(consts::LinearAlgebraType assembly_type);
     virtual void set_preconditioner(consts::PreconditionerType prec_type);
+    virtual void finalize();
 
   private:
     static std::set<consts::LinearAlgebraType> valid_assemblers;

Code/Source/solver/TrilinosLinearAlgebra.cpp

@@ -22,6 +22,7 @@ class TrilinosLinearAlgebra::TrilinosImpl {
     void set_preconditioner(consts::PreconditionerType prec_type) {};
     void solve(ComMod& com_mod, eqType& lEq, const Vector<int>& incL, const Vector<double>& res) {};
     void solve_assembled(ComMod& com_mod, eqType& lEq, const Vector<int>& incL, const Vector<double>& res) {};
+    void finalize(){};
 };
 #endif
 
@@ -113,6 +114,12 @@ void TrilinosLinearAlgebra::initialize(ComMod& com_mod, eqType& lEq)
   impl->initialize(com_mod);
 }
 
+/// @brief Finalize Trilinos.
+void TrilinosLinearAlgebra::finalize()
+{
+  impl->finalize();
+}
+
 /// @brief Create an fsils linear algebra interface for assembly.
 void TrilinosLinearAlgebra::initialize_fsils(ComMod& com_mod, eqType& lEq)
 {

Code/Source/solver/TrilinosLinearAlgebra.h

@@ -23,6 +23,7 @@ class TrilinosLinearAlgebra : public virtual LinearAlgebra {
     virtual void set_assembly(consts::LinearAlgebraType atype);
     virtual void set_preconditioner(consts::PreconditionerType prec_type);
     virtual void solve(ComMod& com_mod, eqType& lEq, const Vector<int>& incL, const Vector<double>& res);
+    virtual void finalize();
 
   private:
     static std::set<consts::LinearAlgebraType> valid_assemblers;

Code/Source/solver/consts.cpp

@@ -255,8 +255,8 @@ const std::set<PreconditionerType> trilinos_preconditioners = {
   PreconditionerType::PREC_TRILINOS_BLOCK_JACOBI,
   PreconditionerType::PREC_TRILINOS_ILU,
   PreconditionerType::PREC_TRILINOS_ILUT,
-  PreconditionerType::PREC_TRILINOS_IC,
-  PreconditionerType::PREC_TRILINOS_ICT,
+  PreconditionerType::PREC_TRILINOS_RILUK0,
+  PreconditionerType::PREC_TRILINOS_RILUK1,
   PreconditionerType::PREC_TRILINOS_ML
 };
 
@@ -286,8 +286,8 @@ const std::map<std::string,PreconditionerType> preconditioner_name_to_type =
   {"trilinos-blockjacobi", PreconditionerType::PREC_TRILINOS_BLOCK_JACOBI},
   {"trilinos-ilu", PreconditionerType::PREC_TRILINOS_ILU},
   {"trilinos-ilut", PreconditionerType::PREC_TRILINOS_ILUT},
-  {"trilinos-ic", PreconditionerType::PREC_TRILINOS_IC},
-  {"trilinos-ict", PreconditionerType::PREC_TRILINOS_ICT},
+  {"trilinos-riluk0", PreconditionerType::PREC_TRILINOS_RILUK0},
+  {"trilinos-riluk1", PreconditionerType::PREC_TRILINOS_RILUK1},
   {"trilinos-ml", PreconditionerType::PREC_TRILINOS_ML},
 
   {"petsc-jacobi", PreconditionerType::PREC_PETSC_JACOBI},
@@ -304,8 +304,8 @@ const std::map<PreconditionerType, std::string> preconditioner_type_to_name {
   {PreconditionerType::PREC_TRILINOS_BLOCK_JACOBI, "trilinos-blockjacobi"}, 
   {PreconditionerType::PREC_TRILINOS_ILU, "trilinos-ilu"}, 
   {PreconditionerType::PREC_TRILINOS_ILUT, "trilinos-ilut"}, 
-  {PreconditionerType::PREC_TRILINOS_IC, "trilinos-ic"}, 
-  {PreconditionerType::PREC_TRILINOS_IC, "trilinos-ict"}, 
+  {PreconditionerType::PREC_TRILINOS_RILUK0, "trilinos-riluk0"}, 
+  {PreconditionerType::PREC_TRILINOS_RILUK1, "trilinos-riluk1"}, 
   {PreconditionerType::PREC_TRILINOS_ML, "trilinos-ml"},
   {PreconditionerType::PREC_PETSC_JACOBI, "petsc-jacobi"},
   {PreconditionerType::PREC_PETSC_RCS, "petsc-rcs"}