Merge pull request #531 from powerapi-ng/docs/joss-paper

docs: Add sources of the `joss` paper

Author: gfieni

docs/.gitignore

@@ -0,0 +1,4 @@
+# Ignore inara output files.
+# https://github.com/openjournals/inara
+jats/*
+paper.pdf

docs/Makefile

@@ -0,0 +1,3 @@
+# PowerAPI paper for the [Journal of Open Source Software](https://joss.theoj.org/)
+This folder contains the source of the [PowerAPI paper submitted to the JOSS journal](https://joss.theoj.org/papers/10.21105/joss.06670).  
+You can [follow the steps on the official documentation](https://joss.readthedocs.io/en/latest/paper.html#checking-that-your-paper-compiles) to build the pdf of the paper.

docs/generate.sh

@@ -0,0 +1,144 @@
+@misc{hwpc-sensor,
+  author = {G. Fieni and R. Rouvoy},
+  title = {Hardware Performance Counters (HwPC) Sensor},
+  year = {2024},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  url = {https://github.com/powerapi-ng/hwpc-sensor}
+}
+
+@misc{powerapi-core,
+  author = {G. Fieni and D. Romero and R. Rouvoy},
+  title = {PowerAPI Core},
+  year = {2024},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  url = {https://github.com/powerapi-ng/powerapi}
+}
+
+@misc{smartwatts,
+  author = {G. Fieni and D.Romero and R. Rouvoy},
+  title = {SmartWatts Formula},
+  year = {2024},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  url = {https://github.com/powerapi-ng/smartwatts-formula}
+}
+
+@inproceedings{fieni:2020,
+  TITLE = {Smart{W}atts: Self-Calibrating Software-Defined Power Meter for Containers},
+  AUTHOR = {Fieni, Guillaume and Rouvoy, Romain and Seinturier, Lionel},
+  URL = {https://inria.hal.science/hal-02470128},
+  BOOKTITLE = {CCGRID 2020 - 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing},
+  ADDRESS = {Melbourne, Australia},
+  YEAR = {2020},
+  MONTH = May,
+  DOI = {10.1109/CCGrid49817.2020.00-45},
+  KEYWORDS = {powerapi ; containers ; Power model ; energy},
+  PDF = {https://inria.hal.science/hal-02470128/file/smartwatts-paper.pdf},
+  HAL_ID = {hal-02470128},
+  HAL_VERSION = {v1},
+  DOI = {10.1109/CCGrid49817.2020.00-45},
+}
+
+@article{pedregosa:2011,
+  title={Scikit-learn: Machine learning in {P}ython},
+  author={Pedregosa, Fabian and Varoquaux, Ga{\"e}l and Gramfort, Alexandre and Michel, Vincent and Thirion, Bertrand and Grisel, Olivier and Blondel, Mathieu and Prettenhofer, Peter and Weiss, Ron and Dubourg, Vincent and others},
+  journal={Journal of machine learning research},
+  volume={12},
+  number={Oct},
+  pages={2825--2830},
+  year={2011},
+}
+
+@manual{intel:2023,
+  author = {Intel Corporation},
+  keywords = {Architectures Manual},
+  month = {December},
+  organization = {Intel Corporation},
+  title = {Intel 64 and IA-32 Architectures Software Developer's Manual - Combined Volumes: 1, 2A, 2B, 2C, 2D, 3A, 3B, 3C, 3D, and 4},
+  year = 2023
+}
+
+@inproceedings{hewitt:1973,
+author = {Hewitt, Carl and Bishop, Peter and Steiger, Richard},
+title = {A universal modular ACTOR formalism for artificial intelligence},
+year = {1973},
+publisher = {Morgan Kaufmann Publishers Inc.},
+address = {San Francisco, CA, USA},
+abstract = {This paper proposes a modular ACTOR architecture and definitional method for artificial intelligence that is conceptually based on a single kind of object: actors [or, if you will, virtual processors, activation frames, or streams]. The formalism makes no presuppositions about the representation of primitive data structures and control structures. Such structures can be programmed, micro-coded, or hard wired in a uniform modular fashion. In fact it is impossible to determine whether a given object is "really" represented as a list, a vector, a hash table, a function, or a process. The architecture will efficiently run the coming generation of PLANNER-like artificial intelligence languages including those requiring a high degree of parallelism. The efficiency is gained without loss of programming generality because it only makes certain actors more efficient; it does not change their behavioral characteristics. The architecture is general with respect to control structure and does not have or need goto, interrupt, or semaphore primitives. The formalism achieves the goals that the disallowed constructs are intended to achieve by other more structured methods.},
+booktitle = {Proceedings of the 3rd International Joint Conference on Artificial Intelligence},
+pages = {235–245},
+numpages = {11},
+location = {Stanford, USA},
+series = {IJCAI'73},
+}
+
+@book{agha:1986,
+author = {Agha, Gul},
+title = {Actors: a model of concurrent computation in distributed systems},
+year = {1986},
+isbn = {0262010925},
+publisher = {MIT Press},
+address = {Cambridge, MA, USA}
+}
+
+@inproceedings{fieni:2021,
+  TITLE = {SELFWATTS: On-the-fly Selection of Performance Events to Optimize Software-defined Power Meters},
+  AUTHOR = {Fieni, Guillaume and Rouvoy, Romain and Seinturier, Lionel},
+  URL = {https://inria.hal.science/hal-03173410},
+  BOOKTITLE = {CCGRID 2021 - 21th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing},
+  ADDRESS = {Melbourne, Australia},
+  YEAR = {2021},
+  MONTH = May,
+  KEYWORDS = {Powerapi ; containers ; Virtual machines ; power model ; energy},
+  PDF = {https://inria.hal.science/hal-03173410/file/selfwatts-paper.pdf},
+  HAL_ID = {hal-03173410},
+  HAL_VERSION = {v1},
+  DOI = {10.1109/ccgrid51090.2021.00042},
+}
+
+@article{colmant:2018,
+  TITLE = {The Next 700 {CPU} Power Models},
+  AUTHOR = {Colmant, Maxime and Rouvoy, Romain and Kurpicz, Mascha and Sobe, Anita and Felber, Pascal and Seinturier, Lionel},
+  URL = {https://inria.hal.science/hal-01827132},
+  JOURNAL = {Journal of Systems and Software},
+  PUBLISHER = {{Elsevier}},
+  VOLUME = {144},
+  PAGES = {382-396},
+  YEAR = {2018},
+  MONTH = Jul,
+  DOI = {10.1016/j.jss.2018.07.001},
+  KEYWORDS = {software toolkit ; energy monitoring ; power models ; software-defined power meters ; open testbed},
+  PDF = {https://inria.hal.science/hal-01827132v2/file/colmant-jss.pdf},
+  HAL_ID = {hal-01827132},
+  HAL_VERSION = {v2},
+}
+
+@inproceedings{colmant:2017,
+  TITLE = {Watts{K}it: Software-Defined Power Monitoring of Distributed Systems},
+  AUTHOR = {Colmant, Maxime and Felber, Pascal and Rouvoy, Romain and Seinturier, Lionel},
+  URL = {https://inria.hal.science/hal-01439889},
+  BOOKTITLE = {17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing ({CCGrid})},
+  ADDRESS = {Madrid, Spain},
+  EDITOR = {Franck Capello and Geoffrey Fox and Javier Garcia-Blas},
+  PUBLISHER = {{IEEE}},
+  SERIES = {Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid)},
+  PAGES = {10},
+  YEAR = {2017},
+  MONTH = May,
+  KEYWORDS = {energy ;  cluster ; power ; model ; middleware ; power meter},
+  PDF = {https://inria.hal.science/hal-01439889/file/colmant-ccgrid17-camera-ready.pdf},
+  HAL_ID = {hal-01439889},
+  HAL_VERSION = {v1},
+  DOI = {10.1109/ccgrid.2017.27},
+}
+
+@inproceedings{lebeane:2015,
+  author={LeBeane, Michael and Ryoo, Jee Ho and Panda, Reena and John, Lizy Kurian},
+  booktitle={2015 27th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)},
+  title={Watt Watcher: Fine-Grained Power Estimation for Emerging Workloads},
+  year={2015},
+  pages={106-113},
+  keywords={Radiation detectors;Program processors;Power measurement;Microarchitecture;Hardware;Power demand;Monitoring},
+  doi={10.1109/SBAC-PAD.2015.26}}

docs/joss-paper/.gitignore

@@ -0,0 +1,88 @@
+---
+title: 'PowerAPI: A Python framework for building software-defined power meters'
+tags:
+  - Python
+  - C
+  - energy comsuption
+  - green computing
+authors:
+  - name: Guillaume Fieni
+    orcid: 0000-0002-0165-6824
+    affiliation: 1
+  - name: Daniel Romero Acero
+    orcid: 0000-0002-5317-2610
+    affiliation: 1
+  - name: Pierre Rust
+    orcid: 0000-0002-2761-4702
+    affiliation: 3
+  - name: Romain Rouvoy
+    orcid: 0000-0003-1771-8791
+    affiliation: "1,2"
+affiliations:
+ - name: Inria, France
+   index: 1
+ - name: University of Lille, France
+   index: 2
+ - name: Orange Labs, France
+   index: 3
+date: 26 March 2024
+bibliography: paper.bib
+---
+
+# Summary
+
+Software that we use daily for accessing digital services from connected devices has a negative impact on the environment as it consumes energy. These digital services, hosted by physical machines around the world, also contribute to planetary pollution. Unfortunately, providers of these online services mostly focus on hardware efficiency to reduce the environmental impact without considering the software they host. For this reason, we propose [`PowerAPI`](https://powerapi.org/) [@powerapi-core], a software-defined solution that delivers real-time estimations of software power consumption to spot opportunities to reduce it and therefore to limit their impact on the planet beyond hardware improvements.
+
+
+# Statement of need
+
+State-of-the-art power meters only measure the global consumption of power of the machines hosting the software. To deal with this limitation, several software-defined power meters have been proposed. However, they are statically designed and coupled to a specific processor family or they are mainly based on power models that need a lot of data and time to be trained to predict power consumption accurately [@colmant:2018; @lebeane:2015]. Power models designed in this way are only suitable for environmental conditions (i.e., memory usage, CPU usage, kinds of running applications) similar to those when the data for the training was collected. If these conditions change, power models become deprecated and have to be trained again, which make them unsuitable for production environments of Cloud providers.
+
+To deal with this constraint, we developed [`PowerAPI`](https://powerapi.org/) [@powerapi-core], a software toolkit for assembling software power meters, enabling developers/IT administrators to monitor power consumption of software that they write/deploy. Software-defined power meters created with `PowerAPI` enable power consumption estimations at different granularity levels: process, thread, container, virtual machine, etc. Furthermore, power models used by `PowerAPI` are continuously self-calibrated to consider current execution conditions of the machine hosting the software.  
+
+\autoref{fig:powerapi-overview} present the general idea behind a Software Power Meter in `PowerAPI`:
+
+1. A Sensor collects raw metrics related to a machine hosting applications/software to be monitored in terms of energy consumption.
+2. These metrics are stored in a database to be consumed by a software power model.
+3. This power model uses machine learning techniques to estimate energy consumption of applications/software with the raw metrics and it calibrates itself when required.
+4. Estimations produced by the power model are stored in another database.
+5. The stored estimations are used to optimize concerned applications/software.      
+
+![Power consumption estimation in PowerAPI.\label{fig:powerapi-overview}](powerapi-overview.png){ width=80%}
+
+Currently, `PowerAPI` offers a software-defined power meter composed by `Hardware Performance Counter` (`HwPC`) Sensor [@hwpc-sensor] and `SmartWatts` Formula [@fieni:2020; @smartwatts]. `HwPC` Sensor is written in `C` and is based on the _Running Average Power Limit_ (RAPL) interface [@intel:2023], available on Intel and AMD processors,  and the `perf` tool and `cgroup` from the Linux kernel. `SmartWatts` Formula, developed in Python, defines a power model based on a linear regression from the [sckit-learn](https://scikit-learn.org/stable/) library [@pedregosa:2011], which is self-calibrated by using suitable performance counters and an error threshold provided by the Power Meter user. It is possible to use a `CSV` files, `MongoDB`, or a `HTTP` Socket to store metrics from Sensors. For storing estimations, `PowerAPI` offers `CSV` files, `MongoDB`, `InfluxDB`, and `Prometheus`.
+
+`PowerAPI` is based on the actor model [@hewitt:1973; @agha:1986], which means that there is a low coupling between different architectural elements in a software-defined power meter. This fosters the creation of new `Formulas` and/or `Sensors` actors to tune software-defined power meters according to requirements.
+
+# Tool Demonstration
+
+A demonstrator of `PowerAPI` is made available to illustrate the deployment of a software-defined power meter on a Xeon CPU E5-2407 Intel processor (Sandy Bridge Family) running Debian 11. The selected configuration includes `MongoDB` for storing metrics and `InfluxDB` for the estimations.
+In this demonstration, we monitor, in real-time, the power consumption of two web applications based on a microservice architecture and hosted by a bare-metal server: a social network from [DeathStarBench](https://github.com/delimitrou/DeathStarBench) and [tea store online](https://github.com/DescartesResearch/TeaStore) developed by the [Descartes Research group](https://se.informatik.uni-wuerzburg.de/). \autoref{fig:server-consumption-sn} and \autoref{fig:server-consumption-ts} depicts the [grafana](https://grafana.com/) dashboard for these web applications.
+
+![Social Network Power consumption, server side.\label{fig:server-consumption-sn}](server-consumption-sn.png){width=80%}
+
+![Tea Store Power consumption, server side.\label{fig:server-consumption-ts}](server-consumption-ts.png){width=80%}
+
+In the tool demonstration, we also monitor, in real-time, a client laptop accessing the above-mentioned web applications from Firefox and Chromium. The laptop uses an Intel i5-6300U (Skylake Family) processor and runs Ubuntu 20.04.5 LTS. \autoref{fig:client-consumption-sn} and \autoref{fig:client-consumption-ts} show the grafana dashboard for the client running 2 instances of each browser.
+
+![Social Network Power consumption, client side.\label{fig:client-consumption-sn}](client-consumption-sn.png){width=80%}
+
+![Tea Store Power consumption, client side.\label{fig:client-consumption-ts}](client-consumption-ts.png){width=80%}
+
+
+
+# Publications and Projects
+The software toolkit has results from contributions described in several publications [@fieni:2021; @fieni:2020; @colmant:2018; @colmant:2017] and is already exploited in several research projects:
+
+  - [Distiller ANR project](https://distiller.cloud) searches how to reduce energy consumption of Cloud applications.
+  - [Défi FrugalCloud](https://www.inria.fr/fr/inria-ovhcloud) includes the optimization of the energy footprint of cloud infrastructures operated by [OVHcloud](https://www.ovhcloud.com/).
+  - [Défi Pulse](https://www.inria.fr/fr/pulse-defi-qarnot-computing-ademe-calcul-intensif-hpc-environnement) studies how to valorize emissions from High Performance Computing (HPC) using as use case [Qarnot Computing's](https://qarnot.com/) offers.
+
+
+# Acknowledgements
+
+Currently, the maintenance of the toolkit is funded by [Inria](https://www.inria.fr/en), [Orange Labs](https://lelab.orange.fr/), [OVH Cloud](https://www.ovhcloud.com/en/), [Davidson Consulting](https://www.davidson.fr/) and [La Poste Groupe](https://www.lapostegroupe.com/en).
+
+This work received support from the French government through the _Agence Nationale de la Recherche_ (ANR) under the France 2030 program, including partial funding from the [CARECloud](https://carecloud.irisa.fr) (`ANR-23-PECL-0003`), [DISTILLER](https://anr.fr/Projet-ANR-21-CE25-0022) (`ANR-21-CE25-0022`), and [GreenAct](https://anr.fr/Projet-ANR-21-HDF1-0006) (`ANR-21-HDF1-0006`) projects.
+
+# References