Skip to content

Commit ac2c357

Browse files
ZzEeKkAaZzEeKkAa
authored
Merge pull request #268 from IntelPython/feature/make_sycl_optional
Make sycl optional
2 parents 56026ce + 66d7a77 commit ac2c357

File tree

11 files changed

+264
-120
lines changed

11 files changed

+264
-120
lines changed

‎.github/workflows/build_and_run.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040
python-version: ${{ matrix.python }}
4141
mamba-version: "*"
4242
activate-environment: dpbench-dev
43-
environment-file: dpbench-env-linux.yml
43+
environment-file: environments/conda-linux-sycl.yml
4444

4545
- name: Conda info
4646
shell: bash -l {0}
@@ -51,7 +51,8 @@ jobs:
5151
- name: Build dpbench
5252
shell: bash -l {0}
5353
run: |
54-
CC=icx CXX=icpx python setup.py develop -- -Dpybind11_DIR=$(python -m pybind11 --cmakedir) -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir)
54+
CC=icx CXX=icpx DPBENCH_SYCL=1 pip install \
55+
--no-index --no-deps --no-build-isolation -e . -v
5556
5657
- name: Run benchmarks
5758
shell: bash -l {0}

‎README.md‎

Lines changed: 117 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -9,100 +9,145 @@ SPDX-License-Identifier: Apache-2.0
99

1010
# DPBench - Benchmarks to evaluate Data-Parallel Extensions for Python
1111

12-
* __*_numba_*.py__ : This file contains Numba implementations of the benchmarks. There are three modes: nopython-mode, nopython-mode-parallel and nopython-mode-parallel-range.
13-
* __*_numba_dpex_*.py__ : This file contains Numba-Dpex implementations of the benchmarks. There are three modes: kernel-mode, numpy-mode and prange-mode.
14-
* __*_dpnp_*.py__ : This file contains dpnp implementations of the benchmarks.
15-
* __*_native_ext/_sycl/__kernel_*.hpp__ : This file contains native dpcpp implementations of the benchmarks.
12+
* **\<benchmark\>\_numba\_\<mode\>.py** : This file contains Numba implementations of the benchmarks. There are three modes: nopython-mode, nopython-mode-parallel and nopython-mode-parallel-range.
13+
* **\<benchmark\>\_numba_dpex\_\<mode\>.py** : This file contains Numba-Dpex implementations of the benchmarks. There are three modes: kernel-mode, numpy-mode and prange-mode.
14+
* **\<benchmark\>\_dpnp\_\<mode\>.py** : This file contains dpnp implementations of the benchmarks.
15+
* **\<benchmark\>\_native_ext/\<benchmark\>\_sycl/_\<benchmark\>_kernel.hpp** : This file contains native dpcpp implementations of the benchmarks.
16+
* **\<benchmark\>\_numpy.py** : This file contains numpy implementations of the benchmarks. It should take benefits of numpy arrays and should avoid loops over arrays.
17+
* **\<benchmark\>\_python.py** : This file contains naive python implementations of the benchmarks. Should be run only for small presets, otherwise it will take long execution time.
18+
* **\<benchmark\>\_numba_mlir\_\<mode\>.py** : This file contains Numba-MLIR implementations of the benchmarks. There are three modes: kernel-mode, numpy-mode and prange-mode. Experimental.
1619

1720
## Examples of setting up and running the benchmarks
18-
1. Setting up conda environment and installing dependencies:
19-
20-
$ conda create -n dpbench-dev
21-
$ conda activate dpbench-dev
22-
$ conda install python
23-
$ conda install -c intel tbb dpcpp_linux-64
24-
$ conda install numpy numba cython cmake ninja scikit-build pandas
25-
$ conda install scipy scikit-learn pybind11 tomli
26-
# do not miss the quotes!
27-
$ conda install -c pkgs/main libgcc-ng">=11.2.0" libstdcxx-ng">=11.2.0" libgomp">=11.2.0"
28-
$ conda install -c dppy/label/dev -c intel -c main dpctl numba-dpex dpnp
29-
$ pip install alembic
30-
31-
2. Build and run DPBench
32-
- To build:
33-
```bash
34-
$ CC=icx CXX=icpx python setup.py develop -- -Dpybind11_DIR=$(python -m pybind11 --cmakedir) -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir)
35-
```
36-
- To run, taking black_scholes for example:
37-
```bash
38-
$ dpbench -b black_scholes run
39-
```
40-
- Similarly, to run all the cases in DPBench:
41-
```bash
42-
$ dpbench -a run
43-
```
21+
1. Clone the repository
4422

45-
3. Device Customization
46-
47-
If a framework is SYCL based, an extra configuration option `sycl_device` may be set in the
48-
framework JSON file to control what device the framework uses for execution. The `sycl_device`
49-
value should be a legal
50-
[SYCL device filter ](https://intel.github.io/llvm-docs/EnvironmentVariables.html#sycl_device_filter)
51-
string. The dpcpp, dpnp, and numba_dpex frameworks support the sycl_device option.
23+
```bash
24+
git clone https://github.com/IntelPython/dpbench
25+
cd dpbench
26+
```
5227

53-
Here is an example:
28+
2. Setting up conda environment and installing dependencies:
5429

55-
```json
56-
{
57-
"framework": {
58-
"simple_name": "dpcpp",
59-
"full_name": "dpcpp",
60-
"prefix": "dp",
61-
"postfix": "dpcpp",
62-
"class": "DpcppFramework",
63-
"arch": "gpu",
64-
"sycl_device": "level_zero:gpu:0"
65-
}
66-
}
30+
```bash
31+
conda env create -n dpbench -f ./environments/conda.yml
6732
```
6833

69-
> **_NOTE:_** The `arch` option is deprecated and not used by dpbench.
34+
- SYCL
7035

71-
To run with customized framework JSON file, pass it as an argument to the `run_benchmark` or
72-
`run_benchmarks` functions.
36+
If you want to build sycl benchmarks as well:
37+
```bash
38+
conda env create -n dpbench -f ./environments/conda-linux-sycl.yml
39+
CC=icx CXX=icpx DPBENCH_SYCL=1 pip install --no-index --no-deps --no-build-isolation -e . -v
40+
```
7341

74-
TODO: current way not working anymore.
42+
3. Build DPBench
7543

7644
```bash
77-
$ python -c "import dpbench; dpbench.run_benchmark(\"black_scholes\", "<absolute path to json file>")"
45+
pip install --no-index --no-deps --no-build-isolation -e . -v
7846
```
7947

80-
## Running numba-mlir benchmarks
81-
1. Setting up conda environment and installing dependencies:
48+
Alternatively you can build it with `setup.py`, but pip version is preferable:
8249

83-
Use same instructions as for usual dpbench setup.
84-
85-
Install latest `numba-mlir` dev package:
50+
```bash
51+
python setup.py develop
52+
```
8653

87-
$ conda install numba-mlir -c dppy/label/dev -c intel
54+
- SYCL
8855

89-
2. Build and run DPBench
56+
For sycl build use:
57+
```bash
58+
CC=icx CXX=icpx DPBENCH_SYCL=1 pip install --no-index --no-deps --no-build-isolation -e . -v
59+
```
9060

91-
Use same commands to setup and run dpbench:
61+
or
9262

9363
```bash
94-
$ dpbench -b black_scholes run
64+
CC=icx CXX=icpx DPBENCH_SYCL=1 python setup.py develop
9565
```
9666

97-
or, to run specific version:
67+
4. Run specific benchmark, e.g. black_scholes
9868

69+
```bash
70+
dpbench -b black_scholes run
71+
```
72+
5. Run all benchmarks
9973

100-
```bash
101-
$ dpbench -b black_scholes -i numba_mlir_k run
102-
```
74+
```bash
75+
dpbench -a run
76+
```
10377

104-
to run all `numba-mlir` benchmarks:
78+
6. Device Customization
10579

106-
```bash
107-
$ dpbench -b black_scholes -i numba_mlir_k,numba_mlir_n,numba_mlir_p run
108-
```
80+
If a framework is SYCL based, an extra configuration option `sycl_device` may be set in the
81+
framework config file or by passing `--sycl-device` argument to `dpbench run` to control what device the framework uses for execution. The `sycl_device`
82+
value should be a legal
83+
[SYCL device filter ](https://intel.github.io/llvm-docs/EnvironmentVariables.html#sycl_device_filter)
84+
string. The dpcpp, dpnp, and numba_dpex frameworks support the sycl_device option.
85+
86+
Here is an example:
87+
88+
```json
89+
dpbench -b black_scholes -i dpnp run --sycl-device=level_zero:gpu:0
90+
```
91+
92+
7. All available options are available using `dpbench --help` and `dpbench <command> --help`:
93+
94+
```
95+
usage: dpbench [-h] [-b [BENCHMARKS]] [-i [IMPLEMENTATIONS]] [-a | --all-implementations | --no-all-implementations] [--version] [-r [RUN_ID]] [--last-run | --no-last-run]
96+
[-d [RESULTS_DB]] [--log-level [{critical,fatal,error,warning,info,debug}]]
97+
{run,report,config} ...
98+
99+
positional arguments:
100+
{run,report,config}
101+
102+
options:
103+
-h, --help show this help message and exit
104+
-b [BENCHMARKS], --benchmarks [BENCHMARKS]
105+
Comma separated list of benchmarks. Leave empty to load all benchmarks.
106+
-i [IMPLEMENTATIONS], --implementations [IMPLEMENTATIONS]
107+
Comma separated list of implementations. Use --all-implementations to load all available implementations.
108+
-a, --all-implementations, --no-all-implementations
109+
If set, all available implementations will be loaded. (default: False)
110+
--version show program's version number and exit
111+
-r [RUN_ID], --run-id [RUN_ID]
112+
run_id to perform actions on. Use --last-run to use latest available run, or leave empty to create new one.
113+
--last-run, --no-last-run
114+
Sets run_id to the latest run_id from the database. (default: False)
115+
-d [RESULTS_DB], --results-db [RESULTS_DB]
116+
Path to a database to store results.
117+
--log-level [{critical,fatal,error,warning,info,debug}]
118+
Log level.
119+
```
120+
121+
```
122+
usage: dpbench run [-h] [-p [{S,M,L}]] [-s | --validate | --no-validate] [--dpbench | --no-dpbench] [--npbench | --no-npbench] [--polybench | --no-polybench] [-r [REPEAT]] [-t [TIMEOUT]]
123+
[--precision [{single,double}]] [--print-results | --no-print-results] [--save | --no-save] [--sycl-device [SYCL_DEVICE]]
124+
[--skip-expected-failures | --no-skip-expected-failures]
125+
126+
Subcommand to run benchmark executions.
127+
128+
options:
129+
-h, --help show this help message and exit
130+
-p [{S,M,L}], --preset [{S,M,L}]
131+
Preset to use for benchmark execution.
132+
-s, --validate, --no-validate
133+
Set if the validation will be run for each benchmark. (default: True)
134+
--dpbench, --no-dpbench
135+
Set if run dpbench benchmarks. (default: True)
136+
--npbench, --no-npbench
137+
Set if run npbench benchmarks. (default: False)
138+
--polybench, --no-polybench
139+
Set if run polybench benchmarks. (default: False)
140+
-r [REPEAT], --repeat [REPEAT]
141+
Number of repeats for each benchmark.
142+
-t [TIMEOUT], --timeout [TIMEOUT]
143+
Timeout time in seconds for each benchmark execution.
144+
--precision [{single,double}]
145+
Data precision to use for array initialization.
146+
--print-results, --no-print-results
147+
Show the result summary or not (default: True)
148+
--save, --no-save Either to save execution into database. (default: True)
149+
--sycl-device [SYCL_DEVICE]
150+
Sycl device to overwrite for framework configurations.
151+
--skip-expected-failures, --no-skip-expected-failures
152+
Either to save execution into database. (default: True)
153+
```

‎dpbench/config/reader.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -356,7 +356,11 @@ def read_benchmark_implementations(
356356
if hasattr(impl_mod, func):
357357
func_name = func
358358
break
359-
except Exception:
359+
except Exception as e:
360+
logging.warn(f"Could not import module: {e}")
361+
import traceback
362+
363+
traceback.print_exc()
360364
continue
361365

362366
config.implementations.append(

‎dpbench/console/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# SPDX-FileCopyrightText: 2022 - 2023 Intel Corporation
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
"""Console tools."""

‎dpbench/console/config.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
"""Report subcommand package."""
66

77
import argparse
8+
import logging
89

910
from ._namespace import Namespace
1011

@@ -33,6 +34,8 @@ def execute_config(args: Namespace):
3334
args: object with all input arguments.
3435
conn: database connection.
3536
"""
37+
import importlib
38+
3639
import dpbench.config as cfg
3740

3841
cfg.GLOBAL = cfg.read_configs(
@@ -42,7 +45,15 @@ def execute_config(args: Namespace):
4245
with_polybench=True,
4346
)
4447

45-
if args.color:
48+
color_output = args.color
49+
50+
if color_output and not importlib.util.find_spec("textwrap"):
51+
logging.warn(
52+
"pygments not found. If you want color output - install it using pip/conda etc"
53+
)
54+
color_output = False
55+
56+
if color_output:
4657
from pprint import pformat
4758

4859
from pygments import highlight

‎dpbench/console/run.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -136,9 +136,9 @@ def execute_run(args: Namespace, conn: sqlalchemy.Engine):
136136
)
137137

138138
if args.all_implementations:
139-
args.implementations = {
139+
args.implementations = [
140140
impl.postfix for impl in cfg.GLOBAL.implementations
141-
}
141+
]
142142

143143
if args.sycl_device:
144144
for framework in cfg.GLOBAL.frameworks:

‎dpbench/migrations/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# SPDX-FileCopyrightText: 2022 - 2023 Intel Corporation
2+
#
3+
# SPDX-License-Identifier: Apache-2.0

‎environments/conda-linux-sycl.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# SPDX-FileCopyrightText: 2022 - 2023 Intel Corporation
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
name: dpbench-dev
6+
channels:
7+
- dppy/label/dev
8+
- intel
9+
- conda-forge
10+
- nodefaults
11+
dependencies:
12+
- conda-forge::pip
13+
- conda-forge::tomli
14+
- conda-forge::alembic>=1.11.0
15+
- conda-forge::sqlalchemy>=2.0.0
16+
- conda-forge::scipy
17+
- conda-forge::scikit-learn
18+
- conda-forge::pandas
19+
- numpy
20+
- numba
21+
- dpctl
22+
- dpnp
23+
- numba-dpex
24+
- numba-mlir
25+
- dpcpp_linux-64
26+
- cython
27+
- cmake>=3.22
28+
- ninja
29+
- scikit-build
30+
# https://github.com/scikit-build/scikit-build/issues/981
31+
- setuptools>=42,<64
32+
- pybind11
33+
- conda-forge::libgcc-ng
34+
- conda-forge::libstdcxx-ng
35+
- conda-forge::libgomp

dpbench-env-linux.yml renamed to environments/conda.yml

Lines changed: 9 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -7,27 +7,18 @@ channels:
77
- dppy/label/dev
88
- intel
99
- conda-forge
10+
- nodefaults
1011
dependencies:
11-
- tbb
12-
- dpcpp_linux-64
12+
- conda-forge::pip
13+
- conda-forge::tomli
14+
- conda-forge::alembic>=1.11.0
15+
- conda-forge::sqlalchemy>=2.0.0
16+
- conda-forge::scipy
17+
- conda-forge::scikit-learn
18+
- conda-forge::pandas
1319
- numpy
1420
- numba
15-
- cython
16-
- cmake>=3.22
17-
- ninja
18-
- scikit-build
19-
- pandas
20-
- scipy
21-
- scikit-learn
22-
- pybind11
23-
- tomli
2421
- dpctl
25-
- numba-dpex
2622
- dpnp
23+
- numba-dpex
2724
- numba-mlir
28-
- libgcc-ng>=11.2.0
29-
- libstdcxx-ng>=11.2.0
30-
- libgomp>=11.2.0
31-
32-
- pip:
33-
- alembic

0 commit comments

Comments
 (0)