Comparing the energy efficiency of different LLMs

In this project, we get the energy efficiency of different Large Language Models (LLMs), namely Qwen, DeepSeek, Mistral and Codellama. We use EnergiBridge for measuring how much energy each LLM uses when given a programming tasks, taken from the HumanEval dataset.

To run the project, the instructions are as follows (for Mac):

1. Clone the repository.
2. Download the models from HuggingFace. You can use the download_models.sh script.
3. Install uv following its installation instructions.
4. Create a virtual environment with the dependencies from pyproject.toml:

   uv venv
   uv sync
   source .venv/bin/activate
   

5. Run the file run_mac.sh. The entrypoint of the application is run_completion_samples.py.

The results for each model can be found under outputs. The scripts to generate the plots together with resulting .jsonl and .csv files can be found under analysis. The plots of these outputs may be found under plots.