Merge remote-tracking branch 'origin/master' into adjacent-neighborhood-change

Author: stijnh

.gitignore

@@ -2,6 +2,8 @@
 poetry.lock
 noxenv.txt
 noxsettings.toml
+hyperparamtuning*/*
+*.prof
 
 ### Python ###
 *.pyc
@@ -16,13 +18,17 @@ push_to_pypi.sh
 .nfs*
 *.log
 *.json
+!kernel_tuner/schema/T1/1.0.0/input-schema.json
+!test/test_T1_input.json
+!test_cache_file*.json
 *.csv
 .cache
 *.ipynb_checkpoints
 examples/cuda/output
 deploy_key
 *.mod
 temp_*.*
+.DS_Store
 .python-version
 .nox
 
@@ -37,4 +43,4 @@ temp_*.*
 .LSOverride
 
 .vscode
-.idea
+.idea

CHANGELOG.md

@@ -3,13 +3,17 @@ All notable changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/).
 
 ## Unreleased
+<!-- ## [1.1.0] - 2025 ?? -->
+- Additional improvements to search space construction
 - changed HIP python bindings from pyhip-interface to the official hip-python
+- Added Python 3.13 and experimental 3.14 support
+- Dropped Python 3.8 and 3.9 support (due to incompatibility with newer scipy versions)
 
 ## [1.0.0] - 2024-04-04
 - HIP backend to support tuning HIP kernels on AMD GPUs
 - Experimental features for mixed-precision and accuracy tuning
 - Experimental features for OpenACC tuning
-- Major speedup due to new parser and using revamped python-constraint for searchspace building
+- Major speedup due to new parser and using revamped python-constraint for search space construction
 - Implemented ability to use `PySMT` and `ATF` for searchspace building
 - Added Poetry for dependency and build management
 - Switched from `setup.py` and `setup.cfg` to `pyproject.toml` for centralized metadata, added relevant tests

INSTALL.rst

@@ -20,7 +20,7 @@ Linux users could type the following to download and install Python 3 using Mini
     wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
     bash Miniconda3-latest-Linux-x86_64.sh
 
-You are of course also free to use your own Python installation, and the Kernel Tuner is developed to be fully compatible with Python 3.9 and newer.
+You are of course also free to use your own Python installation, and the Kernel Tuner is developed to be fully compatible with Python 3.10 and newer.
 
 Installing Python Packages
 --------------------------

README.md

@@ -16,7 +16,7 @@
 
 
 Create optimized GPU applications in any mainstream GPU 
-programming language (CUDA, HIP, OpenCL, OpenACC).
+programming language (CUDA, HIP, OpenCL, OpenACC, OpenMP).
 
 What Kernel Tuner does:
 

doc/requirements.txt

@@ -57,7 +57,7 @@ pytz==2025.1 ; python_version >= "3.9" and python_version < "3.15"
 pywin32==308 ; sys_platform == "win32" and platform_python_implementation != "PyPy" and python_version >= "3.9" and python_version < "3.15"
 pyzmq==26.2.1 ; python_version >= "3.9" and python_version < "3.15"
 referencing==0.36.2 ; python_version >= "3.9" and python_version < "3.15"
-requests==2.32.3 ; python_version >= "3.9" and python_version < "3.15"
+requests==2.32.4 ; python_version >= "3.9" and python_version < "3.15"
 rpds-py==0.23.1 ; python_version >= "3.9" and python_version < "3.15"
 scikit-learn==1.6.1 ; python_version >= "3.9" and python_version < "3.15"
 scipy==1.13.1 ; python_version >= "3.9" and python_version < "3.15"
@@ -78,11 +78,11 @@ stack-data==0.6.3 ; python_version >= "3.9" and python_version < "3.15"
 threadpoolctl==3.5.0 ; python_version >= "3.9" and python_version < "3.15"
 tinycss2==1.4.0 ; python_version >= "3.9" and python_version < "3.15"
 tomli==2.2.1 ; python_version >= "3.9" and python_version < "3.15"
-tornado==6.4.2 ; python_version >= "3.9" and python_version < "3.15"
+tornado==6.5.1 ; python_version >= "3.9" and python_version < "3.15"
 traitlets==5.14.3 ; python_version >= "3.9" and python_version < "3.15"
 typing-extensions==4.12.2 ; python_version >= "3.9" and python_version < "3.15"
 tzdata==2025.1 ; python_version >= "3.9" and python_version < "3.15"
-urllib3==2.3.0 ; python_version >= "3.9" and python_version < "3.15"
+urllib3==2.5.0 ; python_version >= "3.9" and python_version < "3.15"
 wcwidth==0.2.13 ; python_version >= "3.9" and python_version < "3.15"
 webencodings==0.5.1 ; python_version >= "3.9" and python_version < "3.15"
 xmltodict==0.14.2 ; python_version >= "3.9" and python_version < "3.15"

doc/requirements_test.txt

@@ -29,7 +29,7 @@
 # import data from pyproject.toml using https://github.com/sphinx-toolbox/sphinx-pyproject
 # additional data can be added with `[tool.sphinx-pyproject]` and retrieved with `config['']`.
 config = SphinxConfig(
-    "../../pyproject.toml", style="poetry"
+    "../../pyproject.toml",
 )  # add `, globalns=globals()` to directly insert in namespace
 year = time.strftime("%Y")
 startyear = "2016"

doc/source/conf.py

@@ -27,8 +27,8 @@ Steps with :bash:`sudo` access (e.g. on a local device):
     * After installation, restart your shell. 
 #. Install the required Python versions: 
     * On some systems, additional packages may be needed to build Python versions. For example on Ubuntu: :bash:`sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev libsqlite3-dev wget libbz2-dev liblzma-dev lzma`.
-    * Install the Python versions with: :bash:`pyenv install 3.9 3.10 3.11 3.12`. The reason we're installing all these versions as opposed to just one, is so we can test against all supported Python versions.
-#. Set the Python versions so they can be found: :bash:`pyenv local 3.9 3.10 3.11 3.12` (replace :bash:`local` with :bash:`global` when not using the virtualenv).
+    * Install the Python versions with: :bash:`pyenv install 3.9 3.10 3.11 3.12 3.13`. The reason we're installing all these versions as opposed to just one, is so we can test against all supported Python versions.
+#. Set the Python versions so they can be found: :bash:`pyenv local 3.9 3.10 3.11 3.12 3.13` (replace :bash:`local` with :bash:`global` when not using the virtualenv).
 #. Setup a local virtual environment in the folder: :bash:`pyenv virtualenv 3.11 kerneltuner` (or whatever environment name and Python version you prefer).
 #. `Install Poetry <https://python-poetry.org/docs/#installing-with-the-official-installer>`__. 
     * Use :bash:`curl -sSL https://install.python-poetry.org | python3 -` to install Poetry.
@@ -78,7 +78,7 @@ Steps without :bash:`sudo` access (e.g. on a cluster):
     * Verify that your development environment has no missing installs or updates with :bash:`poetry install --sync --dry-run --with test`. 
 #. Check if the environment is setup correctly by running :bash:`pytest`. All tests should pass, except if you're not on a GPU node, or one or more extras has been left out in the previous step, then these tests will skip gracefully.
 #. Set Nox to use the correct backend and location:
-    * Run :bash:`conda -- create-settings-file` to automatically create a settings file. 
+    * Run :bash:`nox -- create-settings-file` to automatically create a settings file. 
     * In this settings file :bash:`noxsettings.toml`, change the :bash:`venvbackend`:
         * If you used Mamba in step 2, to :bash:`mamba`.
         * If you used Miniconda or Anaconda in step 2, to :bash:`conda`.

doc/source/dev-environment.rst

@@ -46,6 +46,12 @@ cache files, serving a value from the cache for the first time in the run also c
 Only unique function evaluations are counted, so the second time a parameter configuration is selected by the strategy it is served from the 
 cache, but not counted as a unique function evaluation.
 
+All optimization algorithms, except for brute_force, random_sample, and bayes_opt, allow the user to specify an initial guess or 
+starting point for the optimization, called ``x0``. This can be passed to the strategy using the ``strategy_options=`` dictionary with ``"x0"`` as key and
+a list of values for each parameter in tune_params to note the starting point. For example, for a kernel that has parameters ``block_size_x`` (64, 128, 256)
+and ``tile_size_x`` (1,2,3), one could pass ``strategy_options=dict(x0=[128,2])`` to ``tune_kernel()`` to make sure the strategy starts from
+the configuration with ``block_size_x=128, tile_size_x=2``. The order in the ``x0`` list should match the order in the tunable parameters dictionary.
+
 Below all the strategies are listed with their strategy-specific options that can be passed in a dictionary to the ``strategy_options=`` argument
 of ``tune_kernel()``.
 

doc/source/optimization.rst

@@ -26,7 +26,7 @@
 }
 """
 
-size = 72*1024*1024
+size = 72 * 1024 * 1024
 
 a = numpy.random.randn(size).astype(numpy.float32)
 b = numpy.random.randn(size).astype(numpy.float32)
@@ -39,7 +39,6 @@
 tune_params["nthreads"] = [1, 2, 3, 4, 8, 12, 16, 24, 32]
 tune_params["vecsize"] = [1, 2, 4, 8, 16]
 
-answer = [a+b, None, None, None]
+answer = [a + b, None, None, None]
 
-tune_kernel("vector_add", kernel_string, size, args, tune_params,
-    answer=answer, compiler_options=['-O3'])
+tune_kernel("vector_add", kernel_string, size, args, tune_params, answer=answer, compiler_options=["-fopenmp", "-O3"])