Skip to content

CMake support#664

Open
mtaillefumier wants to merge 40 commits intoetmc:masterfrom
mtaillefumier:cmake_support
Open

CMake support#664
mtaillefumier wants to merge 40 commits intoetmc:masterfrom
mtaillefumier:cmake_support

Conversation

@mtaillefumier
Copy link
Copy Markdown
Contributor

@mtaillefumier mtaillefumier commented Feb 5, 2026

tmlQCD can be compiled with CMake. So far the build supports

  • QUDA (with all QUDA releated options)
  • CUDA
  • HIP
  • MPI
  • OpenMP
  • lime
  • lemon
  • hdf5
  • fftw
  • QphiX
  • lapack
  • DDalphaAMG
  • Rename DFLAGS with TM_USE_BLA
  • Vectorization alignment
  • Documentation
  • cleanup
  • ci/cd update
  • cscs ci/cd update
  • Remove TM_USE_QUDA_EXPERIMENTAL and TM_QUDA_FERMIONIC_FORCES #ifdef statements but not the code itself.
  • HIP_PATH HIP_ROOT
  • Add autodetection
  • Remove KOJAX
  • use fetch_content
  • remove bison
  • CMAKE_BUILD_TYPE
  • Rewrite the README.md
  • HIP+QUDA+tmLQCD
  • test compilation on beverin / AMD GPU (can be done separately, not cmake specific)

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

@kostrzewa and @chaoos :

  • I reorganized the source tree and moved all library files in src/lib and executable in src/bin. Other layouts can be used
  • Compilation fails for some of the tests for instance test_locallity.c.

@kostrzewa
Copy link
Copy Markdown
Member

@kottnad @chaoos in preparation for the discussion it would be great if you could give this a look and see if there are things where we should use the opportunity to do them differently

@mtaillefumier mtaillefumier force-pushed the cmake_support branch 3 times, most recently from 2d50b54 to 8f39dd2 Compare February 11, 2026 07:58
chaoos
chaoos previously requested changes Feb 11, 2026
Copy link
Copy Markdown
Contributor

@chaoos chaoos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with the changes from my suggestions below, the build still fails for me with various compiler errors and warnings, like e.g.:

/home/roman/projects/tmp/tmLQCD/src/lib/io/gauge_read_binary.c: In function ‘read_binary_gauge_data’:
/home/roman/projects/tmp/tmLQCD/src/lib/io/gauge_read_binary.c:72:17: error: ‘g_cart_grid’ undeclared (first use in this function); did you mean ‘g_cart_id’?
   72 |     MPI_Barrier(g_cart_grid);
      |                 ^~~~~~~~~~~
      |                 g_cart_id
/home/roman/projects/tmp/tmLQCD/src/lib/io/gauge_read_binary.c:72:17: note: each undeclared identifier is reported only once for each function it appears in
/home/roman/projects/tmp/tmLQCD/src/lib/io/utils_construct_writer.c: In function ‘construct_writer’:
/home/roman/projects/tmp/tmLQCD/src/lib/io/utils_construct_writer.c:17:28: error: ‘g_cart_grid’ undeclared (first use in this function); did you mean ‘g_cart_id’?
   17 |     status = MPI_File_open(g_cart_grid, filename,
      |                            ^~~~~~~~~~~
      |                            g_cart_id

One can put these variables as extern, but then at link time it fail to resolve with undefined reference to g_cart_grid'`.

@chaoos
Copy link
Copy Markdown
Contributor

chaoos commented Feb 12, 2026

Even with the changes from my suggestions below, the build still fails for me with various compiler errors and warnings, like e.g.:

/home/roman/projects/tmp/tmLQCD/src/lib/io/gauge_read_binary.c: In function ‘read_binary_gauge_data’:
/home/roman/projects/tmp/tmLQCD/src/lib/io/gauge_read_binary.c:72:17: error: ‘g_cart_grid’ undeclared (first use in this function); did you mean ‘g_cart_id’?
   72 |     MPI_Barrier(g_cart_grid);
      |                 ^~~~~~~~~~~
      |                 g_cart_id
/home/roman/projects/tmp/tmLQCD/src/lib/io/gauge_read_binary.c:72:17: note: each undeclared identifier is reported only once for each function it appears in
/home/roman/projects/tmp/tmLQCD/src/lib/io/utils_construct_writer.c: In function ‘construct_writer’:
/home/roman/projects/tmp/tmLQCD/src/lib/io/utils_construct_writer.c:17:28: error: ‘g_cart_grid’ undeclared (first use in this function); did you mean ‘g_cart_id’?
   17 |     status = MPI_File_open(g_cart_grid, filename,
      |                            ^~~~~~~~~~~
      |                            g_cart_id

One can put these variables as extern, but then at link time it fail to resolve with undefined reference to g_cart_grid'`.

Apologies, if I build lemon with the new cmake system as static library, I can build tmlqcd properly.

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

mtaillefumier commented Feb 12, 2026

Sorry, the current version does not support the lemon configure build as the lemon cmake build system makes this more reliable. I need to check if I generate a pkg-config file for lemon

endif()

# check for fftw3 (rely on pkgconfig).
if(TM_USE_FFTW)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kostrzewa check if FFTW still needed

@kostrzewa
Copy link
Copy Markdown
Member

remove LAPH_ev.c

set_target_properties(hmc PROPERTIES VERSION ${PROJECT_VERSION} SOVERSION 1)

# define a library and add the dependencies
target_link_libraries(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kostrzewa test if this works with CVC

@kostrzewa
Copy link
Copy Markdown
Member

please add a couple of example cmake calls to a doc/cmake-examples directory

@kostrzewa
Copy link
Copy Markdown
Member

I'm afraid some compilation paths have been broken. For example,

-DTM_USE_HALFSPINOR=OFF

does not compile because the hopping.h header has disappeared from src/lib/operator.

@kostrzewa
Copy link
Copy Markdown
Member

kostrzewa commented Mar 27, 2026

This happened during e591af5 and I think hopping.h was mangled before that during the SSE / BGL / XLC removal.

See hopping.h at state https://github.com/etmc/tmLQCD/blob/b4f5fbba6cded50237b8ee8507beec1d247b08e5/operator/hopping.h -> #else branch.

@kostrzewa
Copy link
Copy Markdown
Member

We could add an additional github workflow which builds (but does not run) a set of compilation flag combinations.

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

-DTM_USE_HALFSPINOR=OFF is working now

- Restored hopping.h
- Rename hmc with tmlqcd
- Fix TM_USE_DDalphaAMG when necessary
- Remove header guards in git_hash.c
mtaillefumier and others added 5 commits March 27, 2026 09:13
Co-authored-by: chaoos <chaoos@users.noreply.github.com>
- Restored hopping.h
- Rename hmc with tmlqcd
- Fix TM_USE_DDalphaAMG when necessary
- Remove header guards in git_hash.c
- Update Spack package
…ge.py

Co-authored-by: chaoos <chaoos@users.noreply.github.com>
@mtaillefumier
Copy link
Copy Markdown
Contributor Author

@kostrzewa: I removed the duplicated file hopping_body_dbl.c and the comment in 'src/lib/xchange/xchange_halffield.c'. Unfortunately I force pushed which mean the commit you commented out is still there with a different reference.

@kostrzewa
Copy link
Copy Markdown
Member

It would be great if we could try to avoid force-pushes if possible.

For example the work in #666 has been based on cmake_support and I worry that there will be some issues when the PTBC stuff becomes ready to be tested.

@kostrzewa
Copy link
Copy Markdown
Member

Note that in tm_[sub,times]_Hopping_Matrix.c the current head commit here still refers to hopping_body_dbl.c instead of hopping_body_dbl.inc.

Regarding your comment about the insertion of halfspinor_body.inc and hopping_body_dbl.inc I agree that it's not the prettiest solution but it does enable three variations (which are essentiallly kernel fusions)

Hopping_Matrix.c, tm_sub_Hopping_Matrix.c and tm_times_Hopping_Matrix.c to be implemented without code duplication.

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

mtaillefumier commented Mar 27, 2026

Note that in tm_[sub,times]_Hopping_Matrix.c the current head commit here still refers to hopping_body_dbl.c instead of hopping_body_dbl.inc.

Regarding your comment about the insertion of halfspinor_body.inc and hopping_body_dbl.inc I agree that it's not the prettiest solution but it does enable three variations (which are essentiallly kernel fusions)

Hopping_Matrix.c, tm_sub_Hopping_Matrix.c and tm_times_Hopping_Matrix.c to be implemented without code duplication.

can we reach this with inlining. I may not want to include this in this PR though. I modified the last remaning file as well

@kostrzewa
Copy link
Copy Markdown
Member

can we reach this with inlining. I may not want to include this in this PR though.

let's not include it here

I modified the last remaning file as well

okay, should I close the corresponding PR then? #672

@kostrzewa
Copy link
Copy Markdown
Member

I'm going to continue testing the build on different machines. It will take a few days but I would like to complete this before easter as the teaching period starts again on the 13th of April and I'll have too much to do then...

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

mtaillefumier commented Mar 27, 2026

can we reach this with inlining. I may not want to include this in this PR though.

let's not include it here

I modified the last remaning file as well

okay, should I close the corresponding PR then? #672

[ignore] Actually it might be better to merge your PR first and then I can rebase afterwards. This fix needs to be applied to the last version as well.

I merged your PR and I will fix potential conflicts. I need to check that I did not delete the hopping.h file in the "old" tree.

include hopping.h and hoping_body_dbl.inc in tm_[sub,times]_Hopping_Matrix.c
@mtaillefumier
Copy link
Copy Markdown
Contributor Author

mtaillefumier commented Mar 27, 2026

@chaoos @kostrzewa

A short recap of changes

  • Restored invert.c
  • Restored hopping.h
  • Fix compilation issue with HALF_SPINOR is OFF
  • Fix TM_USE_DDalphaAMG
  • Renamed hmc to tmlqcd (small caps)
  • Remove unnecessary comments
  • create a tag for lemon and update the Spack package
  • check that hopping.h is present in the old build system

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

i opened a PR in the lemon github repo updating the version to 1.2.0. I can either merge it manually and then create the tag accordingly or let someone else do it

@kostrzewa
Copy link
Copy Markdown
Member

check that hopping.h is present in the old build system

no it's not, we should probably also restore it there and then tag that as rel-6-0-2

the cmake_support branch will then become rel-6-1-0

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

mtaillefumier commented Mar 27, 2026

check that hopping.h is present in the old build system

no it's not, we should probably also restore it there and then tag that as rel-6-0-2

the cmake_support branch will then become rel-6-1-0

I am already working on it. I will open a PR in a short while.

@mtaillefumier
Copy link
Copy Markdown
Contributor Author

Pr opened to fix hopping.h. Once merged I will rebase this PR from the master branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants