Skip to content

Commit 24527a1

Browse files
committed
[SYCL][E2E] Add a test to track UNSUPPORTED tests
Similar to XFAIL tests (no-xfail-without-tracker.cpp) adding a requirement to follow UNSUPPORTED by UNSUPPORTED-TRACKER. Adding a test to track the amount and the list of improperly UNSUPPORTED tests. Also update some tests to check the script.
1 parent 003fa58 commit 24527a1

9 files changed

+591
-9
lines changed

sycl/test-e2e/Assert/assert_in_kernels.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
// REQUIRES: linux
22

3-
// https://github.com/intel/llvm/issues/7634
43
// UNSUPPORTED: hip
4+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/7634
55
//
66
// XFAIL: (opencl && gpu)
77
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/11364

sycl/test-e2e/Assert/assert_in_multiple_tus.cpp

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
// REQUIRES: linux
22

3-
// https://github.com/intel/llvm/issues/7634
43
// UNSUPPORTED: hip
4+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/7634
55
//
6-
// https://github.com/intel/llvm/issues/8832
76
// UNSUPPORTED: cuda
7+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/8832
88
//
99
// XFAIL: (opencl && gpu)
1010
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/11364
1111

12-
// https://github.com/intel/llvm/issues/15029
1312
// UNSUPPORTED: gpu-intel-dg2
13+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/15029
1414

1515
// RUN: %{build} -DSYCL_FALLBACK_ASSERT=1 -I %S/Inputs %S/Inputs/kernels_in_file2.cpp -o %t.out
1616
// RUN: %{run} %t.out &> %t.txt ; FileCheck %s --input-file %t.txt %if fpga %{ --check-prefix=CHECK-ACC %}

sycl/test-e2e/Assert/assert_in_multiple_tus_one_ndebug.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
// REQUIRES: linux
22

3-
// https://github.com/intel/llvm/issues/7634
43
// UNSUPPORTED: hip
4+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/7634
55
//
6-
// https://github.com/intel/llvm/issues/8832
76
// UNSUPPORTED: cuda
7+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/8832
88
//
99
// XFAIL: (opencl && gpu)
1010
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/11364

sycl/test-e2e/Assert/assert_in_one_kernel.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
// REQUIRES: linux
22

3-
// https://github.com/intel/llvm/issues/7634
43
// UNSUPPORTED: hip
4+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/7634
55
//
66
// XFAIL: (opencl && gpu)
77
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/11364

sycl/test-e2e/Assert/assert_in_simultaneously_multiple_tus_one_ndebug.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// FIXME flaky fail on CUDA
2-
// FIXME HIP: https://github.com/intel/llvm/issues/7634
32
// UNSUPPORTED: cuda, hip
3+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/7634
44
//
55
// XFAIL: (opencl && gpu)
66
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/11364

sycl/test-e2e/Basic/kernel_max_wg_size.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
// https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/experimental/sycl_ext_intel_grf_size.asciidoc
66
// REQUIRES: arch-intel_gpu_pvc || gpu-intel-dg2
77
// UNSUPPORTED: cuda || hip
8+
// UNSUPPORTED-TRACKER: INTENDED - This extension is currently implemented in DPC++ only for Intel GPU devices and only when using the Level Zero backend or OpenCL backend.
89

910
// clang-format off
1011
#include <sycl/detail/core.hpp>

sycl/test-e2e/README.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -344,3 +344,38 @@ Once the issue is created, you can update the test by adding `XFAIL` and
344344

345345
If you add `XFAIL` without `XFAIL-TRACKER` directive,
346346
`no-xfail-without-tracker.cpp` test will fail, notifying you about that.
347+
348+
## Marking tests as unsupported
349+
350+
Some tests may be considered unsupported, e.g.:
351+
* the test checks the feature that is not supported by some
352+
backend / device / OS / etc.
353+
* the test is flaky or hangs, so it can't be marked with `XFAIL`.
354+
355+
In these cases the test can be marked with `UNSUPPORTED`. This mark should be
356+
followed by `UNSUPPORTED-TRACKER`. There are two options:
357+
```
358+
UNSUPPORTED-TRACKER: INTENDED - <Free-text with reason why it's not intended to be supported on the given feature.>
359+
UNSUPPORTED-TRACKER: <github issue link / internal number>
360+
```
361+
362+
In the first case it should look like:
363+
```
364+
// UNSUPPORTED: cuda, hip
365+
// UNSUPPORTED-TRACKER: INTENDED - only supported by backends with SPIR-V IR
366+
367+
// UNSUPPORTED: gpu
368+
// UNSUPPORTED-TRACKER: INTENDED - the feature works only on CPU & FPGA
369+
```
370+
371+
For the second case the tracker should be provided:
372+
```
373+
// Sporadically fails on DG2.
374+
// UNSUPPORTED: gpu-intel-dg2
375+
// UNSUPPORTED-TRACKER: https://github.com/intel/llvm/issues/DDDDD
376+
// *OR*
377+
// UNSUPPORTED-TRACKER: PRJ-1234
378+
```
379+
380+
If you add `UNSUPPORTED` without `UNSUPPORTED-TRACKER` directive,
381+
`no-unsupported-without-tracker.cpp` test will fail, notifying you about that.

0 commit comments

Comments
 (0)