POC: Enable Proton for XPU; basis, not ready for user use (#2635)

Closes #1145

On LTS proton doesn't work: `zelTracerCreate` return `2013265921`
(ZE_RESULT_ERROR_UNINITIALIZED) code. Looks like relates to
#1953.

**4/2/25 status update:**

There is still no necessary interface in PTI, but work on it is in
progress and it seems to me there is little left (thanks @jfedorov!). I
believe that at least the main points of the implementation can be
reviewed and maybe even merged. The main attention should be paid to
dependencies, which include sycl, level_zero (including an additional
interface for calling callbacks, which is used temporarily) and PTI. I
added PTI headers directly to the Triton sources, since they will be
small in size and are used only for building Proton, I believe this will
not interfere with the main code. I tried to add dependencies so as not
to break the build if they are missing, also some paths are hardcoded,
because the main use case for now is just launching UTs in CI. There is
a lot of debug code left in the code, which outputs additional
information to stdout, I would like to leave it for now. I plan to clean
this code as soon as PTI implements the necessary functionality (the
integration of which will highlight many problems, I'm sure, since I
can't emulate the behavior of the PTI completely).

**12/12/24 status update:**

The most important problem is the lack of an interface for registering
callback functions that would be called before and after kernel
execution. Cupti and Roctracer backend of Proton use this interface to
register (via `callbackData->correlationId`) ​​kernel calls to build a
call tree and, upon profiler shutdown, to check that all created records
for running kernels were written to Proton storage structures from the
backend profiler storage system. The current workaround uses the
interface `<level_zero/layers/zel_tracing_api.h>`, which provides
functions for registering user functions via a pair of
`prologue_callbacks/epilogue_callbacks` for various events. However, the
problem of obtaining the record identifier in these callbacks that PTI
will create for registered kernels **has not been solved yet** (a
manually selected identifier is currently used for tests).

How it works in Cupti? (simplified)
```c++
void CuptiProfiler::CuptiProfilerPimpl::callbackFn(void *userData,
                                                   CUpti_CallbackDomain domain,
                                                   CUpti_CallbackId cbId,
                                                   const void *cbData) {
// ...
  if (callbackData->callbackSite == CUPTI_API_ENTER) {
      // scope registration
      auto scopeId = threadState.record();
      threadState.enterOp(scopeId);
      // ...
      // linking the internal profiler data ID to the external one that proton uses. 
      profiler.correlation.correlate(callbackData->correlationId, numInstances);
  } else if (callbackData->callbackSite == CUPTI_API_EXIT) {
     // ...
     // scope exit
     threadState.exitOp();
     // the submitted record should be taken into account when flushing data
     profiler.correlation.submit(callbackData->correlationId);
  }
// ...
}

void CuptiProfiler::CuptiProfilerPimpl::doStart() {
  // subscriber is `CUpti_SubscriberHandle`
  cupti::subscribe<true>(&subscriber, callbackFn, nullptr);            // not exists in PTI
  cupti::activityEnable<true>(CUPTI_ACTIVITY_KIND_CONCURRENT_KERNEL);  // exists in PTI
  cupti::activityRegisterCallbacks<true>(allocBuffer, completeBuffer); // exists in PTI
  // `setGraphCallbacks` do something like:
  //   CALLBACK_ENABLE(CUPTI_CBID_RESOURCE_GRAPHNODE_CREATED);
  //   CALLBACK_ENABLE(CUPTI_CBID_RESOURCE_GRAPHNODE_CLONED);
  //   ...
  setGraphCallbacks(subscriber, /*enable=*/true);                      // not exists in PTI
  // `setRuntimeCallbacks` do something like:
  //   CALLBACK_ENABLE(CUPTI_RUNTIME_TRACE_CBID_cudaLaunch_v3020);
  //   CALLBACK_ENABLE(CUPTI_RUNTIME_TRACE_CBID_cudaLaunchKernel_v7000);
  //   ...
  setRuntimeCallbacks(subscriber, /*enable=*/true);                    // not exists in PTI
  // `setDriverCallbacks` do something like:
  //   CALLBACK_ENABLE(CUPTI_DRIVER_TRACE_CBID_cuLaunch);
  //   CALLBACK_ENABLE(CUPTI_DRIVER_TRACE_CBID_cuLaunchGrid);
  //   ...
  setDriverCallbacks(subscriber, /*enable=*/true);                     // not exists in PTI
}
``` 

**Needs to be done:**
* Think of what to do with getting the PTI profiler record identifiers
in level zero callbacks or how to find a way without them.
* ​​~Synchronize the device to make a correct data flush.~ (done via
`sycl_queue.wait()`)
* What to do with the concept of cuda graph kernels?
* Obtain the device architecture.
* Enable unit tests.
  * [x] test_api.py
  * [x] test_lib.py
  * [x] test_profile.py (partially)
  * [x] test_viewer.py
* Enable tutorials.
* Final code cleanup.



https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/14202903723

---------

Signed-off-by: Anatoly Myachev <[email protected]>

Author: anmyachev

.github/workflows/build-test-reusable.yml

@@ -152,6 +152,13 @@ jobs:
             echo TRITON_TEST_CMD="bash -v -x scripts/test-triton.sh --warning-reports --skip-pytorch-install --reports-dir $GITHUB_WORKSPACE/reports ${{ inputs.ignore_errors && '--ignore-errors' || '' }} $skiplist"
           } | tee -a $GITHUB_ENV
 
+      - name: Run Proton tests
+        if: ${{ inputs.driver_version == 'rolling' }}
+        run: |
+          cd third_party/proton/test
+          pytest test_api.py test_lib.py test_profile.py test_viewer.py -s -v
+          cd ..
+
       - name: Run unit tests
         run: |
           ${{ env.TRITON_TEST_CMD }} --unit

.pre-commit-config.yaml

@@ -135,5 +135,6 @@ exclude: |
     ^third_party/amd/backend/include/roctracer/|
     ^third_party/amd/backend/lib/|
     ^third_party/nvidia/backend/include/cuda.h|
-    ^third_party/f2reduce
+    ^third_party/f2reduce|
+    ^third_party/intel/backend/proton/include
   )

python/setup.py

@@ -432,6 +432,10 @@ def get_proton_cmake_args(self):
         if roctracer_include_dir == "":
             roctracer_include_dir = os.path.join(get_base_dir(), "third_party", "amd", "backend", "include")
         cmake_args += ["-DROCTRACER_INCLUDE_DIR=" + roctracer_include_dir]
+        xpupti_include_dir = get_env_with_keys(["TRITON_XPUPTI_INCLUDE_PATH"])
+        if xpupti_include_dir == "":
+            xpupti_include_dir = os.path.join(get_base_dir(), "third_party", "intel", "backend", "proton", "include")
+        cmake_args += ["-DXPUPTI_INCLUDE_DIR=" + xpupti_include_dir]
         return cmake_args
 
     def build_extension(self, ext):

third_party/intel/backend/proton/include/pti/pti.h

@@ -0,0 +1,52 @@
+//==============================================================
+// Copyright (C) Intel Corporation
+//
+// SPDX-License-Identifier: MIT
+// =============================================================
+
+#ifndef INCLUDE_PTI_H_
+#define INCLUDE_PTI_H_
+
+#include "pti/pti_export.h"
+#include "pti/pti_version.h"
+
+/* clang-format off */
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/**
+ * @brief Return/Error codes
+ */
+typedef enum {
+  PTI_SUCCESS = 0,                        //!< success
+  PTI_STATUS_END_OF_BUFFER = 1,           //!< end of buffer reached, e.g., in ptiViewGetNextRecord
+  PTI_ERROR_NOT_IMPLEMENTED = 2,          //!< functionality not implemented
+  PTI_ERROR_BAD_ARGUMENT = 3,             //!< error code for invalid arguments
+  PTI_ERROR_NO_CALLBACKS_SET = 4,         //!< error due to no callbacks set via ptiViewSetCallbacks
+  PTI_ERROR_EXTERNAL_ID_QUEUE_EMPTY = 5,  //!< empty external ID-queue while working with
+                                          //!< PTI_VIEW_EXTERNAL_CORRELATION
+  PTI_ERROR_BAD_TIMESTAMP = 6,  //!< error in timestamp conversion, might be related with the user
+                                //!< provided TimestampCallback
+  PTI_ERROR_DRIVER = 50,        //!< unknown driver error
+  PTI_ERROR_TRACING_NOT_INITIALIZED = 51,  //!< installed driver requires tracing enabling with
+                                           //!< setting environment variable ZE_ENABLE_TRACING_LAYER
+                                           //!< to 1
+  PTI_ERROR_L0_LOCAL_PROFILING_NOT_SUPPORTED = 52,  //!< no Local profiling support in the installed
+                                                    //!< driver
+
+  PTI_ERROR_INTERNAL = 200  //!< internal error
+} pti_result;
+
+/**
+ * @brief Helper function to return stringified enum members for pti_result
+ *
+ * @return const char*
+ */
+PTI_EXPORT const char* ptiResultTypeToString(pti_result result_value);
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif  // INCLUDE_PTI_H_

third_party/intel/backend/proton/include/pti/pti_export.h

@@ -0,0 +1,42 @@
+
+#ifndef PTI_EXPORT_H
+#define PTI_EXPORT_H
+
+#ifdef PTI_STATIC_DEFINE
+#  define PTI_EXPORT
+#  define PTI_NO_EXPORT
+#else
+#  ifndef PTI_EXPORT
+#    ifdef pti_EXPORTS
+        /* We are building this library */
+#      define PTI_EXPORT __attribute__((visibility("default")))
+#    else
+        /* We are using this library */
+#      define PTI_EXPORT __attribute__((visibility("default")))
+#    endif
+#  endif
+
+#  ifndef PTI_NO_EXPORT
+#    define PTI_NO_EXPORT __attribute__((visibility("hidden")))
+#  endif
+#endif
+
+#ifndef PTI_DEPRECATED
+#  define PTI_DEPRECATED 
+#endif
+
+#ifndef PTI_DEPRECATED_EXPORT
+#  define PTI_DEPRECATED_EXPORT PTI_EXPORT PTI_DEPRECATED
+#endif
+
+#ifndef PTI_DEPRECATED_NO_EXPORT
+#  define PTI_DEPRECATED_NO_EXPORT PTI_NO_EXPORT PTI_DEPRECATED
+#endif
+
+#if 0 /* DEFINE_NO_DEPRECATED */
+#  ifndef PTI_NO_DEPRECATED
+#    define PTI_NO_DEPRECATED
+#  endif
+#endif
+
+#endif /* PTI_EXPORT_H */

third_party/intel/backend/proton/include/pti/pti_version.h

@@ -0,0 +1,51 @@
+//==============================================================
+// Copyright (C) Intel Corporation
+//
+// SPDX-License-Identifier: MIT
+// =============================================================
+#ifndef INCLUDE_PTI_VERSION_H_
+#define INCLUDE_PTI_VERSION_H_
+
+#include <stdint.h>
+
+#include "pti/pti_export.h"
+
+/* clang-format off */
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#if !defined(PTI_VERSION)
+#define PTI_VERSION 0.10.0
+#endif
+
+#define PTI_VERSION_STRING "0.10.0"
+#define PTI_VERSION_MAJOR 0
+#define PTI_VERSION_MINOR 10
+#define PTI_VERSION_PATCH 0
+
+typedef struct pti_version {
+  uint32_t _major;
+  uint32_t _minor;
+  uint32_t _patch;
+} pti_version;
+
+/**
+ * @brief Returns the compiled version of PTI
+ *
+ * @return c-string with compiled version of PTI
+ */
+PTI_EXPORT const char* ptiVersionString();
+
+/**
+ * @brief Returns the compiled version of PTI
+ *
+ * @return pti_version struct with compiled version of PTI
+ */
+pti_version PTI_EXPORT ptiVersion();
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif  // INCLUDE_PTI_VERSION_H_