|
| 1 | +# GPU Math Conformance Tests |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This test suite provides a framework to systematically measure the accuracy of math functions on GPUs and verify their conformance with standards like OpenCL. |
| 6 | + |
| 7 | +While the primary focus is validating the implementations in the C standard math library (LLVM-libm), these tests can also be executed against other math library providers, such as CUDA Math and HIP Math, for comparison. |
| 8 | + |
| 9 | +The goals of this project are to empower LLVM-libm contributors with a robust tool for validating their implementations and to build trust with end-users by providing transparent accuracy data. |
| 10 | + |
| 11 | +### Table of Contents |
| 12 | + |
| 13 | +- [Getting Started](#getting-started) |
| 14 | +- [Running the Tests](#running-the-tests) |
| 15 | +- [Adding New Tests](#adding-new-tests) |
| 16 | + |
| 17 | +## Getting Started |
| 18 | + |
| 19 | +This guide covers how to build the necessary dependencies, which include the new Offload API and the C standard library for both host and GPU targets. |
| 20 | + |
| 21 | +### System Requirements |
| 22 | + |
| 23 | +Before you begin, ensure your system meets the following requirements: |
| 24 | + |
| 25 | +- A system with an AMD or NVIDIA GPU. |
| 26 | +- The latest proprietary GPU drivers installed. |
| 27 | +- The corresponding development SDK for your hardware: |
| 28 | + - **AMD:** [ROCm SDK](https://rocm.docs.amd.com) |
| 29 | + - **NVIDIA:** [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) |
| 30 | + |
| 31 | +### Building the Dependencies |
| 32 | + |
| 33 | +The official documentation for building LLVM-libc for GPUs provides a detailed guide and should be considered the primary reference. Please follow the instructions in the **"Standard runtimes build"** section of that guide: |
| 34 | + |
| 35 | +- [Building the GPU C library (Official Documentation)](https://libc.llvm.org/gpu/building.html) |
| 36 | + |
| 37 | +> [!IMPORTANT] |
| 38 | +> For the conformance tests, the standard `cmake` command from the official documentation must be adapted slightly. You must also add `libc` to the main `-DLLVM_ENABLE_RUNTIMES` list. This is a crucial step because the tests need a host-side build of `libc` to use as the reference oracle for validating GPU results. |
| 39 | +
|
| 40 | +## Running the Tests |
| 41 | + |
| 42 | +### Default Test |
| 43 | + |
| 44 | +To build and run the conformance test for a given function (e.g., `logf`) against the default C standard math library `llvm-libm` provider, use the following command. This will execute the test on all available and supported platforms. |
| 45 | + |
| 46 | +```bash |
| 47 | +ninja -C build/runtimes/runtimes-bins offload.conformance.logf |
| 48 | +``` |
| 49 | + |
| 50 | +### Testing Other Providers |
| 51 | + |
| 52 | +Once the test binary has been built, you can run it against other math library providers using the `--test-configs` flag. |
| 53 | + |
| 54 | +- **For `cuda-math` on an NVIDIA GPU:** |
| 55 | + |
| 56 | + ```bash |
| 57 | + ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=cuda-math:cuda |
| 58 | + ``` |
| 59 | + |
| 60 | +- **For `hip-math` on an AMD GPU:** |
| 61 | + |
| 62 | + ```bash |
| 63 | + ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=hip-math:amdgpu |
| 64 | + ``` |
| 65 | + |
| 66 | +You can also run all available configurations for a test with: |
| 67 | + |
| 68 | +```bash |
| 69 | +./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=all |
| 70 | +``` |
| 71 | + |
| 72 | +## Adding New Tests |
| 73 | + |
| 74 | +To add a conformance test for a new math function, follow these steps: |
| 75 | + |
| 76 | +1. **Implement the Device Kernels**: Create a kernel wrapper for the new function in each provider's source file. For CUDA Math and HIP Math, you must also add a forward declaration for the vendor function in `/device_code/DeviceAPIs.hpp`. |
| 77 | + |
| 78 | +2. **Implement the Host Test**: Create a new `.cpp` file in `/tests`. This file defines the `FunctionConfig` (function and kernel names, as well as ULP tolerance) and the input generation strategy. |
| 79 | + |
| 80 | + - Use **exhaustive testing** (`ExhaustiveGenerator`) for functions with small input spaces (e.g., half-precision functions and single-precision univariate functions). This strategy iterates over every representable point in the input space, ensuring complete coverage. |
| 81 | + - Use **randomized testing** (`RandomGenerator`) for functions with large input spaces (e.g., single-precision bivariate and double-precision functions), where exhaustive testing is computationally infeasible. Although not exhaustive, this strategy is deterministic, using a fixed seed to sample a large, reproducible subset of points from the input space. |
| 82 | + |
| 83 | +3. **Add the Build Target**: Add a new `add_conformance_test(...)` entry to `/tests/CMakeLists.txt` to make the test buildable. |
0 commit comments