Skip to content

Commit 3d80b33

Browse files
joeatoddjoeatodd
authored
Adding Kokkos test case (#24)
* Initial commit of Kokkos material from internal repo * Updates to Kokkos example for distribution * Update README: explain hwloc & gpu arch flags
1 parent ebd92ed commit 3d80b33

File tree

9 files changed

+415
-0
lines changed

9 files changed

+415
-0
lines changed

‎README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,13 @@ This demonstrates using SYCL's `host_task` for CUDA interoperability, calling CU
7575

7676
This example combines the MPI and SGEMM Interop examples to distribute a matrix multiplication problem between MPI ranks.
7777

78+
[Kokkos](examples/kokkos)
79+
--------------------------------------------
80+
81+
[Kokkos](https://github.com/kokkos/kokkos) is a middle-layer for scientific computing which features a SYCL backend. This example
82+
shows a small Kokkos test case (vector-matrix-vector multiplication), adapted from a test case in the Kokkos repo;
83+
there is no SYCL code in the example, but it includes scripts to build Kokkos with SYCL support.
84+
7885
[Hashing Algorithms](examples/hashing)
7986
--------------------------------------------
8087

‎examples/kokkos/CMakeLists.txt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
cmake_minimum_required (VERSION 3.10)
2+
cmake_policy(SET CMP0074 NEW)
3+
project (Kokkos_Test_Case)
4+
5+
set(Kokkos_DIR "$ENV{Kokkos_ROOT}" CACHE STRING "Kokkos root directory")
6+
find_package(Kokkos REQUIRED)
7+
8+
add_executable(test_case test_case.cpp)
9+
target_link_libraries(test_case Kokkos::kokkos)
10+
11+

‎examples/kokkos/README.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
Simple Test Case for Kokkos
2+
----
3+
4+
This is a simple standalone test case taken from the Kokkos repository & packaged up here for use with SYCL.
5+
It's doing a vector-matrix-vector product. It's an identity matrix with two vectors of 1s, so the expected answer
6+
is just equal to the problem size.
7+
8+
Building the test case
9+
-----
10+
11+
test_case.cpp contains a simple kernel which has been copied straight from the Kokkos Tutorials (Exercises/02/Solution).
12+
13+
Build it with build.sh, after setting the environment variable:
14+
```
15+
Kokkos_ROOT="[your/kokkos/installation]/lib/cmake/Kokkos"
16+
```
17+
18+
Running the test case
19+
----
20+
21+
Just launch it! There are optional flags:
22+
23+
-N : number of rows
24+
-M : number of columns
25+
-S : total size
26+
-nrepeat : how many times to repeat the test (default 100)
27+
28+
Obviously, not all of N, M & S should be set. The test case will sanity check your args anyway.
29+
30+
Building Kokkos
31+
------
32+
33+
In case you don't have an existing Kokkos build, there are some build scripts in `./kokkos_build_scripts`.
34+
There are scripts for building Kokkos with SYCL, or CUDA (nvcc or clang).
35+
36+
Set the following environment variables:
37+
```
38+
KOKKOS_INSTALL_DIR=[/your/install/dir]
39+
KOKKOS_SOURCE_DIR=[/your/source/dir]
40+
HWLOC_DIR=[/your/hwloc/dir]
41+
```
42+
43+
HWLOC
44+
------
45+
46+
The [Portable Hardware Locality](https://www.open-mpi.org/projects/hwloc/) (hwloc) package is an optional dependency which enables Kokkos to query the hardware topology of the system on which it is running. If you do not have a HWLOC installation, this option can be removed & Kokkos will be built without HWLOC support.
47+
48+
SYCL backend
49+
-------------
50+
51+
Kokkos should work with any SYCL backend, though the focus of this examples repo is SYCL-For-CUDA.
52+
Previous work at Codeplay has involved running Kokkos with SYCL on Nvidia hardware with Ampere architecture, hence the flag:
53+
```
54+
-DKokkos_ARCH_AMPERE80=ON \
55+
```
56+
This flag is not strictly necessary, but it enables Ahead of Time (AoT) compilation, which can give a significant performance gain when building large projects built on Kokkos.
57+
You should modify the cmake command for your GPU arch.
58+
59+
60+

‎examples/kokkos/build.sh‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
3+
rm -rf build
4+
mkdir build
5+
cd build
6+
7+
# Set the environment variable Kokkos_ROOT="[your/kokkos/installation]/lib/cmake/Kokkos"
8+
CXXFLAGS="-Xsycl-target-frontend -O3" \
9+
LDFLAGS="-Xsycl-target-frontend -O3" \
10+
cmake .. -G Ninja \
11+
-DCMAKE_BUILD_TYPE=Debug \
12+
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
13+
-DCMAKE_CXX_COMPILER=clang++ \
14+
-DCMAKE_C_COMPILER=clang
15+
16+
ninja
17+
18+
cd ..

‎examples/kokkos/kokkos_build_scripts/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
These build scripts are provided for illustration only. They will almost certainly require modification before they work elsewhere.

‎examples/kokkos/kokkos_build_scripts/kokkos_build.sh‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/bash
2+
# Install Kokkos w/ sycl-cuda support
3+
4+
set -x #echo on
5+
6+
# Set:
7+
# KOKKOS_INSTALL_DIR=[/your/install/dir]
8+
# KOKKOS_SOURCE_DIR=[/your/source/dir]
9+
# HWLOC_DIR=[/your/hwloc/dir]
10+
11+
# Configure & build kokkos
12+
mkdir kokkos-build
13+
cd kokkos-build
14+
15+
CXXFLAGS="-Xsycl-target-frontend -O3 -fgpu-inline-threshold=100000 -Wno-unknown-cuda-version -Wno-deprecated-declarations -Wno-linker-warnings -ffast-math" \
16+
LDFLAGS="-Xsycl-target-frontend -O3" \
17+
cmake $KOKKOS_SOURCE_DIR -G Ninja \
18+
-DCMAKE_BUILD_TYPE=Release \
19+
-DCMAKE_CXX_STANDARD=17 \
20+
-DCMAKE_CXX_COMPILER=clang++ \
21+
-DCMAKE_INSTALL_PREFIX=$KOKKOS_INSTALL_DIR \
22+
-DKokkos_CXX_STANDARD=17 \
23+
-DKokkos_ENABLE_SYCL=ON \
24+
-DKokkos_ARCH_HSW=ON \
25+
-DKokkos_ARCH_AMPERE80=ON \
26+
-DKokkos_ENABLE_HWLOC=ON \
27+
-DKokkos_ENABLE_UNSUPPORTED_ARCHS=ON \
28+
-DKokkos_ENABLE_TESTS=OFF \
29+
-DKokkos_HWLOC_DIR=$HWLOC_DIR
30+
31+
ninja install
32+
33+
cd ..

‎examples/kokkos/kokkos_build_scripts/kokkos_clang_cuda_build.sh‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash
2+
# Install Kokkos w/ sycl-cuda support
3+
4+
set -x #echo on
5+
6+
# Set:
7+
# KOKKOS_INSTALL_DIR=[/your/install/dir]
8+
# KOKKOS_SOURCE_DIR=[/your/source/dir]
9+
# HWLOC_DIR=[/your/hwloc/dir]
10+
11+
# Configure & build kokkos
12+
mkdir kokkos-build
13+
cd kokkos-build
14+
15+
CXXFLAGS="-Xsycl-target-frontend -O3 -fgpu-inline-threshold=100000" \
16+
LDFLAGS="-Xsycl-target-frontend -O3" \
17+
cmake $KOKKOS_SOURCE_DIR -G Ninja \
18+
-DCMAKE_BUILD_TYPE=Release \
19+
-DCMAKE_CXX_STANDARD=17 \
20+
-DCMAKE_CXX_COMPILER=clang++ \
21+
-DCMAKE_INSTALL_PREFIX=$KOKKOS_INSTALL_DIR \
22+
-DKokkos_CXX_STANDARD=17 \
23+
-DKokkos_ENABLE_SYCL=OFF \
24+
-DKokkos_ENABLE_CUDA=ON \
25+
-DKokkos_ARCH_HSW=ON \
26+
-DKokkos_ARCH_AMPERE80=ON \
27+
-DKokkos_ENABLE_HWLOC=ON \
28+
-DKokkos_ENABLE_UNSUPPORTED_ARCHS=ON \
29+
-DKokkos_ENABLE_TESTS=OFF \
30+
-DKokkos_HWLOC_DIR=$HWLOC_DIR
31+
32+
ninja install
33+
34+
cd ..

‎examples/kokkos/kokkos_build_scripts/kokkos_nvcc_cuda_build.sh‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/bash
2+
# Install Kokkos w/ sycl-cuda support
3+
4+
set -x #echo on
5+
6+
# Set:
7+
# KOKKOS_INSTALL_DIR=[/your/install/dir]
8+
# KOKKOS_SOURCE_DIR=[/your/source/dir]
9+
# HWLOC_DIR=[/your/hwloc/dir]
10+
11+
# Configure & build kokkos
12+
mkdir kokkos-build
13+
cd kokkos-build
14+
15+
cmake $KOKKOS_SOURCE_DIR -G Ninja \
16+
-DCMAKE_BUILD_TYPE=Release \
17+
-DCMAKE_CXX_STANDARD=17 \
18+
-DCMAKE_CXX_COMPILER=g++ \
19+
-DCMAKE_CUDA_COMPILER=nvcc \
20+
-DCMAKE_INSTALL_PREFIX=$KOKKOS_INSTALL_DIR \
21+
-DKokkos_CXX_STANDARD=17 \
22+
-DKokkos_ENABLE_SYCL=OFF \
23+
-DKokkos_ENABLE_CUDA=ON \
24+
-DKokkos_ARCH_HSW=ON \
25+
-DKokkos_ARCH_AMPERE80=ON \
26+
-DKokkos_ENABLE_HWLOC=ON \
27+
-DKokkos_ENABLE_UNSUPPORTED_ARCHS=ON \
28+
-DKokkos_ENABLE_TESTS=OFF \
29+
-DKokkos_HWLOC_DIR=$HWLOC_DIR
30+
31+
ninja install
32+
33+
cd ..

0 commit comments

Comments
 (0)