|
| 1 | +cff-version: 1.2.0 |
| 2 | +title: Autotuning Methodology |
| 3 | +message: >- |
| 4 | + If you use this software, please cite both the article from preferred-citation and the software itself. |
| 5 | +type: software |
| 6 | +authors: |
| 7 | + - given-names: Floris-Jan |
| 8 | + family-names: Willemsen |
| 9 | + |
| 10 | + affiliation: "Leiden University, Netherlands eScience Center" |
| 11 | + orcid: "https://orcid.org/0000-0003-2295-8263" |
| 12 | +identifiers: |
| 13 | + - type: doi |
| 14 | + value: 10.5281/zenodo.11207516 |
| 15 | + description: Zenodo DOI |
| 16 | +repository-code: >- |
| 17 | + https://github.com/AutoTuningAssociation/autotuning_methodology |
| 18 | +url: >- |
| 19 | + https://autotuningassociation.github.io/autotuning_methodology/ |
| 20 | +abstract: >- |
| 21 | + This software package accompanies the paper "A Methodology |
| 22 | + for Comparing Auto-Tuning Optimization Algorithms", making |
| 23 | + the guidelines in the methodology easy to apply. |
| 24 | +keywords: |
| 25 | + - Auto-tuning |
| 26 | + - Methodology |
| 27 | + - Optimization Algorithms |
| 28 | + - Performance Comparison |
| 29 | + - Performance Metrics |
| 30 | + - Performance Optimization |
| 31 | +license: MIT |
| 32 | +commit: 1c096e7ceabed2ffed53838a052c7188ac93124d |
| 33 | +version: 1.0.0b3 |
| 34 | +date-released: "2024-05-17" |
| 35 | +preferred-citation: |
| 36 | + type: article |
| 37 | + title: A Methodology for Comparing Optimization Algorithms for Auto-Tuning |
| 38 | + journal: "Future Generation Computer Systems" |
| 39 | + year: 2024 |
| 40 | + abstract: >- |
| 41 | + Adapting applications to optimally utilize available hardware is no mean feat: the plethora of choices for optimization techniques are infeasible to maximize manually. |
| 42 | + To this end, auto-tuning frameworksauto-tuning frameworks are used to automate this task, which in turn use optimization algorithms to efficiently search the vast search spaces. |
| 43 | + However, there is a lack of comparability in studies presenting advances in auto-tuning frameworks and the optimization algorithms incorporated. |
| 44 | + As each publication varies in the way experiments are conducted, metrics used, and results reported, comparing the performance of optimization algorithms among publications is infeasible. |
| 45 | + The auto-tuning community identified this as a key challenge at the 2022 Lorentz Center workshop on auto-tuning. |
| 46 | + The examination of the current state of the practice in this paper further underlines this. |
| 47 | + We propose a community-driven methodology composed of four steps regarding experimental setup, tuning budget, dealing with stochasticity, and quantifying performance. |
| 48 | + This methodology builds upon similar methodologies in other fields while taking into account the constraints and specific characteristics of the auto-tuning field, resulting in novel techniques. |
| 49 | + The methodology is demonstrated in a simple case study that compares the performance of several optimization algorithms used to auto-tune CUDA kernels on a set of modern GPUs. |
| 50 | + We provide a software tool to make the application of the methodology easy for authors, and simplifies reproducibility of results. |
| 51 | + authors: |
| 52 | + - given-names: Floris-Jan |
| 53 | + family-names: Willemsen |
| 54 | + |
| 55 | + affiliation: "Leiden University, Netherlands eScience Center" |
| 56 | + orcid: "https://orcid.org/0000-0003-2295-8263" |
| 57 | + - given-names: Richard |
| 58 | + family-names: Schoonhoven |
| 59 | + affiliation: Centrum Wiskunde & Informatica |
| 60 | + orcid: "https://orcid.org/0000-0003-3659-929X" |
| 61 | + - orcid: "https://orcid.org/0000-0002-5703-9673" |
| 62 | + given-names: Jiří |
| 63 | + family-names: Filipovič |
| 64 | + affiliation: Masaryk University |
| 65 | + - given-names: Jacob Odgård |
| 66 | + family-names: Tørring |
| 67 | + orcid: "https://orcid.org/0000-0002-9385-7948" |
| 68 | + affiliation: Norwegian University of Science and Technology |
| 69 | + - given-names: Rob |
| 70 | + name-particle: van |
| 71 | + family-names: Nieuwpoort |
| 72 | + affiliation: Leiden University |
| 73 | + orcid: "https://orcid.org/0000-0002-2947-9444" |
| 74 | + - given-names: Ban |
| 75 | + name-particle: van |
| 76 | + family-names: Werkhoven |
| 77 | + orcid: "https://orcid.org/0000-0002-7508-3272" |
| 78 | + affiliation: "Leiden University, Netherlands eScience Center" |
0 commit comments