task: add more env var documentation #2137

jharlow-intel · 2025-08-19T22:10:35Z

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
Have you added documentation for your changes, if necessary?
Have you added your changes to the changelog?
If this PR is a work in progress, are you opening the PR as a draft?

This PR adds some needed environment variable documentation to the git pages.

docs/doc_sources/user_guides/environment_variables.rst

github-actions · 2025-08-19T22:45:37Z

View rendered docs @ https://intelpython.github.io/dpctl/pulls/2137/index.html

github-actions · 2025-08-19T22:49:36Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_105 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

coveralls · 2025-08-19T22:55:11Z

coverage: 85.902%. remained the same
when pulling a25c161 on task/SAT-7838
into a6a0fdd on master.

github-actions · 2025-08-20T15:29:28Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_105 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-08-20T15:29:56Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_107 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

github-actions · 2025-08-20T16:01:59Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_105 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

github-actions · 2025-08-20T16:24:49Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_105 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

github-actions · 2025-08-20T17:54:07Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_105 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

github-actions · 2025-08-20T18:38:23Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_105 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

jharlow-intel · 2025-08-20T18:43:17Z

@ndgrigorian ready for another review

ndgrigorian · 2025-08-26T07:30:07Z

Noting:
Closes #2037

ndgrigorian · 2025-08-26T07:31:47Z

docs/doc_sources/user_guides/environment_variables.rst

+Variable ``ZE_AFFINITY_MASK``
+-------------------------------
+Allows users to mask specific devices from being used by SYCL applications.
+If we have ZE_FLAT_DEVICE_HIERARCHY set to COMPOSITE, we can have an AFFINITY of “1” for our application to only see device #1 - making system devices 0, and 2+, invisible. Etc.


may be worth mentioning how it works with FLAT and COMBINED
see: https://www.intel.com/content/www/us/en/developer/articles/technical/flattening-gpu-tile-hierarchy.html

I'm not sure if I've tested combined with affinity mask in retrospect...

I already linked that for the ZE_DEVICE_HIERARCHY docs, and I can't find any mention of how exactly ZE_AFFINITY_MASK interacts with COMBINED.

I will include the blurb about FLAT here as well, but I see nothing for COMBINED documented here: https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask

yeah I'm going to go ahead and test it myself, since it's not documented in the compiler composite device extension either

it seems to interact with ZE_AFFINITY_MASK as one might intuitively expect:
if all exposed tiles correspond to the same physical (i.e., composite) device, then that composite device is available and the exposed tiles are available as component devices

if the exposed tiles correspond to different physical devices, then a composite device is available for each physical device that has a tile exposed, and the exposed tiles are available as component devices of the corresponding physical device

this is all quite wordy so the language will need trimming here

github-actions · 2025-08-27T19:11:56Z

Array API standard conformance tests for dpctl=0.21.0dev0=py310h93fe807_111 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

antonwolfy · 2025-09-01T10:34:03Z

docs/doc_sources/user_guides/environment_variables.rst

+    * - ``FLAT``
+      - Level Zero devices with multiple tiles will be exposed as a set of root devices, each corresponding to an individual tile. Enabled by default.
+
+Read more `about device hierarchy here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#device-hierarchy>`_ and `here <https://www.intel.com/content/www/us/en/developer/articles/technical/flattening-gpu-tile-hierarchy.html>`_.


Would it be better to be more explicit in the link name? Something like

Suggested change

Read more `about device hierarchy here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#device-hierarchy>`_ and `here <https://www.intel.com/content/www/us/en/developer/articles/technical/flattening-gpu-tile-hierarchy.html>`_.

Read more about device hierarchy in `Level Zero Specification <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#device-hierarchy>`_ and `Intel GPU article <https://www.intel.com/content/www/us/en/developer/articles/technical/flattening-gpu-tile-hierarchy.html>`_.

antonwolfy · 2025-09-01T10:35:15Z

docs/doc_sources/user_guides/environment_variables.rst

+
+Additional examples to illustrate this are in the detailed documentation for ZE_AFFINITY_MASK.
+
+Read more about `affinity masks here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask>`_.


and here

Suggested change

Read more about `affinity masks here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask>`_.

Read more about affinity masks in `Level Zero Specification <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask>`_.

antonwolfy · 2025-09-01T10:39:10Z

docs/doc_sources/user_guides/environment_variables.rst

+If the system has four dual-tile GPUs installed, this would be the second tile in the first GPU. In FLAT mode, the numbers use a system-wide-sub-device-number from a flat numbering perspective.
+Therefore, we could use the second tile in each of four dual-tile GPUs with ZE_AFFINITY_MASK=1,3,5,7.
+
+Additional examples to illustrate this are in the detailed documentation for ZE_AFFINITY_MASK.


Do you mean the detailed documentation in Level Zero Specification? Should we combine that sentence with the next one then to provide a link?

antonwolfy · 2025-09-01T10:41:05Z

docs/doc_sources/user_guides/environment_variables.rst

+Variable ``ZE_AFFINITY_MASK``
+-------------------------------
+Allows users to mask specific devices from being used by SYCL applications.
+If we have ZE_FLAT_DEVICE_HIERARCHY set to COMPOSITE, we can have an AFFINITY of “1” for our application to only see device #1 - making system devices 0, and 2+, invisible.


Suggested change

If we have ZE_FLAT_DEVICE_HIERARCHY set to COMPOSITE, we can have an AFFINITY of “1” for our application to only see device #1 - making system devices 0, and 2+, invisible.

If we have ``ZE_FLAT_DEVICE_HIERARCHY`` set to ``COMPOSITE``, we can have an AFFINITY of “1” for our application to only see device #1 - making system devices 0, and 2+, invisible.

and the same below

jharlow-intel self-assigned this Aug 19, 2025

jharlow-intel requested review from ndgrigorian, antonwolfy, vlad-perevezentsev and vtavana as code owners August 19, 2025 22:10

ndgrigorian reviewed Aug 19, 2025

View reviewed changes

docs/doc_sources/user_guides/environment_variables.rst Show resolved Hide resolved

ndgrigorian reviewed Aug 19, 2025

View reviewed changes

docs/doc_sources/user_guides/environment_variables.rst Outdated Show resolved Hide resolved

ndgrigorian reviewed Aug 19, 2025

View reviewed changes

docs/doc_sources/user_guides/environment_variables.rst Outdated Show resolved Hide resolved

ndgrigorian reviewed Aug 19, 2025

View reviewed changes

docs/doc_sources/user_guides/environment_variables.rst Outdated Show resolved Hide resolved

jharlow-intel force-pushed the task/SAT-7838 branch 2 times, most recently from e346c8f to 6ad1f1b Compare August 20, 2025 15:21

jharlow-intel force-pushed the task/SAT-7838 branch from 6ad1f1b to 48ba91b Compare August 20, 2025 15:43

jharlow-intel force-pushed the task/SAT-7838 branch from 48ba91b to 9037fa2 Compare August 20, 2025 17:13

jharlow-intel force-pushed the task/SAT-7838 branch from 9037fa2 to 5084a64 Compare August 20, 2025 17:57

jharlow-intel requested a review from ndgrigorian August 20, 2025 18:43

ndgrigorian reviewed Aug 26, 2025

View reviewed changes

task: add more env var documentation

a25c161

jharlow-intel force-pushed the task/SAT-7838 branch from 5084a64 to a25c161 Compare August 27, 2025 18:31

jharlow-intel requested a review from ndgrigorian August 27, 2025 18:31

antonwolfy reviewed Sep 1, 2025

View reviewed changes

	Read more `about device hierarchy here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#device-hierarchy>`_ and `here <https://www.intel.com/content/www/us/en/developer/articles/technical/flattening-gpu-tile-hierarchy.html>`_.
	Read more about device hierarchy in `Level Zero Specification <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#device-hierarchy>`_ and `Intel GPU article <https://www.intel.com/content/www/us/en/developer/articles/technical/flattening-gpu-tile-hierarchy.html>`_.


		Additional examples to illustrate this are in the detailed documentation for ZE_AFFINITY_MASK.

		Read more about `affinity masks here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask>`_.

	Read more about `affinity masks here <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask>`_.
	Read more about affinity masks in `Level Zero Specification <https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/PROG.html#affinity-mask>`_.

	If we have ZE_FLAT_DEVICE_HIERARCHY set to COMPOSITE, we can have an AFFINITY of “1” for our application to only see device #1 - making system devices 0, and 2+, invisible.
	If we have ``ZE_FLAT_DEVICE_HIERARCHY`` set to ``COMPOSITE``, we can have an AFFINITY of “1” for our application to only see device #1 - making system devices 0, and 2+, invisible.

task: add more env var documentation #2137

Are you sure you want to change the base?

task: add more env var documentation #2137

Conversation

jharlow-intel commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2025

Uh oh!

github-actions bot commented Aug 19, 2025

Uh oh!

coveralls commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

jharlow-intel commented Aug 20, 2025

Uh oh!

ndgrigorian commented Aug 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jharlow-intel commented Aug 19, 2025 •

edited

Loading

coveralls commented Aug 19, 2025 •

edited

Loading