Tulipa Learning Community - Data handling & processing (TLC#1)

[IsaiMaganTNO]

Hi all,

This discussion thread serves to collaborate on the data handling & processing discussed during the Tulipa Learning Community session on April 18th 2025. We aim to develop a 'how to'/script to process TYNDP scenario data into TulipaEnergyModel input data format.

[IsaiMaganTNO]

Session 2 - 23/4/2025

Hi Sander and Nihal, since both of you had other meetings I will use this moment to describe some following up steps for our goal in the TLC session. If I recall correctly from last time, we would like to document (in the form of a How-To) how data from the TYNDP scenario study publication can be transformed from a raw format to Tulipa-input ready format. Specifically, we will be looking at electricity generation capacity across Europe.

For the next session logical follow-up steps would be:

• Have a look at the TYNDP2024 scenario study and find the raw scenario output data files
• What is the format of these files? Which ones do we need for processing?
• What is the scope of the data, which countries/bidding zones/subregions are considered? What type of capacities are considered?

Once we know this we can proceed with making an outline for data conversion. I will not be present during session 3 on the 30th of April, but I invite you to brainstorm together if you know the answers to these questions how to proceed. See you in session 4!

Best,
Isaï

[NihalChehber]

Hi,

I went into the [downloads section of the TYNDP 2024 Scenarios website] (https://2024.entsos-tyndp-scenarios.eu/download/) to look for detailed electricity generation capacity data that we can use as Tulipa input.

At first, I checked the obvious files like the Supply Tool and the Scenario Data Figures, but they only had energy balances (TWh) and EU27 totals, not the detailed capacities per country. In the file Scenario Report Data Figures, sheet / Fig 25 gives the EU27 total installed generation capacity per technology (Solar, Wind, Gas, Nuclear, etc.), in gigawatts (GW), for different years (2030, 2040, 2050).

Then I noticed a ZIP file called Investment Datasets. Inside that ZIP, there an Excel file named "TRAJECTORY.xlsx". It has installed capacities per country/bidding zone, per technology (like Solar, Wind, Nuclear), per year (2030, 2040, 2050), and for different scenarios (Distributed Energy and Global Ambition), which I think is more what we are looking for.

Do you think this is a good file to start with?

I will also not be present next week as I am on vacation.

Best,
Nihal

[IsaiMaganTNO]

Hi Nihal,

Thanks for the detailed write-up! It seems you already have found a lot of interesting material to work with. The investment dataset is an interesting one to consider, although whether or not it is suitable for us depends on our proposed model. If I am correct in my interpretation, I believe that it provides the boundary of investment capacity, see for example below:

This data is then taken by the TYNDP2024 as inputs for their modelling suite, where a multi-model optimization eventually provides us with the results that are then derived from these assumptions. These outputs can be found under their "Outputs" section, which also provides information for the different years and scenarios assumed (and climate years, in most cases).

In short: if we take the investment data we can take over the assumptions made by TYNDP and use these for our own investment module in Tulipa, if we take their output files we get the exact prognosis of capacity that the TYNDP derives from their own assumptions following their optimization methods.

I think it will be best if we quickly discuss this tomorrow during/after the TLC so we can decide on something, and then we can proceed with the data processing. Thanks for your work! See you tomorrow.

Best,
Isaï

[NihalChehber]

Hi all,

Thanks Isa, indeed I think it is useful to look at the scenario outputs. I’ve taken a first look at the ENTSO-E scenario output zipfile "[Distributed Energy 2050 – Climate Year 2009] and then specifically the excel file "MMStandardOutputFile_DE2050_Plexos_CY2009_v11_SoS.xlsb"

Source: https://2024.entsos-tyndp-scenarios.eu/download/

The sheet "Yearly Outputs" contains capacity and generation data across technologies and nodes. Here’s a quick breakdown of the structure:

• Column A contains the Output type. To retrieve installed capacities, we should focus on rows where this column is "Installed Capacity [MW]".

• Column B lists the technologies, such as solar, offshore wind, gas, nuclear, electrolyzers, etc.

• Column C onwards shows data for nodes, each column representing a specific node (e.g. DE50, UK00, etc.).

To move forward, here are a few points we can discuss:

• Which nodes/regions do we want to include? Should we keep the node-level granularity, or aggregate capacities?

• Target year and climate year selection: this file is for 2050 and climate year 2009. The file is for 2050 under climate year 2009. If needed, we can retrieve similar files for 2035 or 2040, and for other climate years (1995, 2008).

• Scenario choice: Do we want to extract data for both Distributed Energy and Global Ambition scenarios? If so, we can extract the relevant rows and columns for each scenario separately and then merge them into a unified dataset for comparison.

• Does the finale dataset with installed capacities need to have a specific format to be used directly in Tulipa?

Best,
Nihal

[IsaiMaganTNO]

Hi Nihal,

Thanks for the detailed description of the dataset! I will make sure to get started on the processing steps necessary for this information and show you what I have next week.

Best,
Isaï

[IsaiMaganTNO]

Hi @NihalChehber and Sander, our contribution in its current form can be found in the following pull request: #1246

It still needs some adjusting before it can be implemented but feel free to suggest changes and/or improvements!