feature: parallel solve subspace diagonalization in dav_subspace

[pxlxingliang]

• Add function Diago_HS_para() to diago the H/S by genelpa or scalapack. The function will perform 2D block distribution of data, call elpa/scalapack for solution, and collect eigenvectors to rank0.
• Add a unified function interface pxxxgvx_diag() for calling scalapack to solve the generalized eigenvalue problem with double/complex/float/complex.
• Add a new INPUT parameter diag_subspace_method, which support to set the subspace diagonalization method in dav_subspace to use elpa or scalapack to solve.

Reminder

• Have you linked an issue with this pull request?
• Have you added adequate unit tests and/or case tests for your pull request?
• Have you noticed possible changes of behavior below or in the linked issue?
• Have you explained the changes of codes in core modules of ESolver, HSolver, ElecState, Hamilt, Operator or Psi? (ignore if not applicable)

Linked Issue

Fix #5480

Unit Tests and/or Case Tests for my changes

• A unit test is added for each new feature or bug fix.

What's changed?

• Example: My changes might affect the performance of the application under certain conditions, and I have tested the impact on various scenarios...

Any changes of core modules? (ignore if not applicable)

• Example: I have added a new virtual function in the esolver base class in order to ...

[haozhihan]

Diago_HS_para implements parallel solution for the subspace process in dav-subspace. If this is efficient, can we just fix one solution method to reduce the burden of users?

[haozhihan]

I have a brief insight regarding this PR:

This process involves a transformation of parallel strategy.

• from basis parallelism to 2D block parallelism
• from 2D block parallelism to basis parallelism

Can this transformation of parallel strategy be more general?

If the H and S matrices of LCAO are solved by iterative method (like cg, dav, and so on, usually used for plane wave basis), it will also involve almost the same transformation of parallel strategy.

[pxlxingliang]

Diago_HS_para implements parallel solution for the subspace process in dav-subspace. If this is efficient, can we just fix one solution method to reduce the burden of users?

The parallel diagonization is not always more efficient, it is related to the system size, parallel cores, and efficiency of parallel communication, etc. I have done some tests in #5480.

[pxlxingliang]

I have a brief insight regarding this PR:

This process involves a transformation of parallel strategy.

• from basis parallelism to 2D block parallelism
• from 2D block parallelism to basis parallelism

Can this transformation of parallel strategy be more general?

If the H and S matrices of LCAO are solved by iterative method (like cg, dav, and so on, usually used for plane wave basis), it will also involve almost the same transformation of parallel strategy.

The transformation of different 2D block distribution can be realized easily by call the scalapack function Cpigemr2d() (a uniform interface for different data type is here: https://github.com/deepmodeling/abacus-develop/blob/develop/source/module_base/scalapack_connector.h#L158). While the transformation for basis parallelism is strongly related to the self-defined class in ABACUS (like psi?), may be a class function of psi to do the transformation is better.

[pxlxingliang]

This feature use parallel_2d to deal with the parallel of H/S, but now parallel_2d is only compiled with ENABLE_LCAO=ON. I suggest to move parallel_2d.cpp to module_base to be a basic class.

@mohanchen, @jinzx10, @kirk0830, what's your suggestions?

[mohanchen]

Reopen it if needed

[mohanchen]

What's the status now? @pxlxingliang

[pxlxingliang]

What's the status now? @pxlxingliang

@mohanchen All modifications have been completed, and need your review again.