Skip to content

Merge master april#74

Closed
lindnemi wants to merge 56 commits intomainfrom
merge-master-april
Closed

Merge master april#74
lindnemi wants to merge 56 commits intomainfrom
merge-master-april

Conversation

@lindnemi
Copy link
Collaborator

Before asking for a review for this PR make sure to complete the following checklist:

  • Workflow with target rule ariadne_all completes without errors
  • The logic of export_ariadne_variables has been adapted to the changes
  • One or several figures that validate the changes in the PR have been posted as a comment
  • A brief description of the changes has been added to Changelog.md
  • The latest main has been merged into the PR
  • The config has a new prefix of the format YYYYMMDDdescriptive_title

pworschischek-aggmag and others added 30 commits March 14, 2025 16:34
* introduce etpr for TES using stores

* add constraints

* copleted implementation of constraints for TES stores

* updated release notes

* adjusted mock snakemake default

* library change

* update release notes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete .DS_Store

* change retrieval of etpr values in etpr_constraint

* changed TES constraints check based on the configs

* constraint checks

* enhance error handling for the tes constraints

* small changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* changed mock snakemake back to default settings

* feat: add marginal cost for water tank chargers

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: update add_retrofit_gas_boiler_constraint docs

* docs: update return type annotation for get_heat_system_type method

* feat: add heat vents for decentral and rural heating

* feat: remove unnecessary central_heat_vent config

* feat: pass heat vent marginal cost to prepare_sector_network

* refactor: rename marginal_cost_storage to marginal_cost_home_battery_storage

* docs: update configtables

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: update release notes

* add suggestions to ensure that charger and discharger align for charger ratio constraints and energy power constraints

* add lifetime to PTES and TTES charger/discharger, so that they are built with correct baseyear

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: add small penalty on ptes charging

* feat: update fixed cost to capital cost str

* feat: remove unnecessary passing of obsolete config

* feat: remove unneeded config passing

* feat: remove unneeded config passing

* feat: remove unneeded config passing

* Update scripts/solve_network.py

* Update solve_network.py

* feat: re-add TES necessary parameters

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refactor string contains checks

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Amos Schledorn <60692940+amos-schledorn@users.noreply.github.com>
Co-authored-by: Amos Schledorn <a.schledorn@tu-berlin.de>
Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com>
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.9.10 → v0.11.0](astral-sh/ruff-pre-commit@v0.9.10...v0.11.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* handle pipeline entrypoints for reduced country coverage

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add filegraph rule

* Integrates both rules as one

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add release note

* Apply suggestions from code review

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

* Revert "Apply suggestions from code review"

This reverts commit 64021a83014efae7691056142fa418ac372a7bdd.

* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"

This reverts commit eb18d37396f0adafe29098b6cd0329f930bea37e.

* Revert "Integrates both rules as one"

This reverts commit 27c421bbd7d336ec57c15dd9e86d0417f93b4310.

* Update file names to be consistent

* Update release note

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* add rule plot_heatmap_timeseries

* Add check for filename before saving plot in plot_heatmap function
* add rule plot_balance_timeseries

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add warning for missing bus carriers in plot_balance_timeseries

* Update scripts/plot_balance_timeseries.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* handle pipeline entrypoints for reduced country coverage

* add simplified green energy import options

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add tech_colors for import options

* extra_functionality: add energy import limit

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* refac: tighten constraint on usage of global vars

* tighten pylint check in GH CI

* feat: document biomass to methanol options and improve logging

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* inital commit including Fabians, Bobbys and Philipps plot scripts

* add default plotting yaml
improve plotting script
adjust plotting parameters in plotting yaml

* plot_balance_map: make region colors show even if they all have the same price

* fix collect rule for plots

* plot_balance_map: consider vmin and vmax values from config if given

* add release note

* move plotting part from config.default.yaml to plotting.defaul.yaml

* add configuration settings to documentation and fix bugs in solving.csv table which was previously not shown

* Add new color mappings for water pits and geothermal heat pumps in plotting configuration

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Simplify and refactor plot_balance_map

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove default plotting configuration file from copy_default_files function

* use dynamic projection and handle semicircle correction

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update geomap parameter to boolean in plot_balance_map.py

* Add carrier sanitization in plot_balance_map.py

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Refine charger and discharger matching logic to exclude last element in comparison

* fix: improve error string matching of charger and discharger in TES extra functionality

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: correct charger string comparison in checkup of TES energy constraints

* docs: update release notes

---------

Co-authored-by: bobbyxng <bobbyxng@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Amos Schledorn <a.schledorn@tu-berlin.de>
* refactor: Consolidate cost loading functions

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* separate make_summary into global and individual part

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add separate rule for cumulative costs (myopic)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* make_cumulative_costs: add comment

* make_summary: make assign_locations more robust

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* make_summary: clean and refactor to use n.statistics

* handle in/out, release notes, adjust plot_summary

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* handle make_summary_perfect

* handle run wildcard in rule all

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix from previous merge conflict resolution

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* assign_locations: unify across scripts and handle EU loc with NUTS

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add rule to process raw CO2Stop data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: correct casing in data path for CO2JRC_OpenFormats

* Update scripts/build_co2_sequestration_potentials.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor: remove unused rule for geological CO2 storage potential retrieval

* fix: remove duplicate entries in GeoDataFrame during map merge

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…607)

* Move shipping sector into separate function and config

* Move aviation sector into separate function and config

* fix: remove shipping demand file from add_industry
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.11.0 → v0.11.2](astral-sh/ruff-pre-commit@v0.11.0...v0.11.2)
- [github.com/snakemake/snakefmt: v0.10.2 → v0.11.0](snakemake/snakefmt@v0.10.2...v0.11.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…609)

- When solving electricity-only systems, TES is enabled (default value) but heating bus not defined
- If network contains no links, if clause throws error
* docs: update templates and help references

* fix

* fix

* harmonize to `.yaml`

* Add mailing list as knowledge database
* introduce etpr for TES using stores

* add constraints

* copleted implementation of constraints for TES stores

* updated release notes

* adjusted mock snakemake default

* library change

* update release notes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete .DS_Store

* change retrieval of etpr values in etpr_constraint

* changed TES constraints check based on the configs

* constraint checks

* enhance error handling for the tes constraints

* small changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* changed mock snakemake back to default settings

* feat: add marginal cost for water tank chargers

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: update add_retrofit_gas_boiler_constraint docs

* docs: update return type annotation for get_heat_system_type method

* feat: add heat vents for decentral and rural heating

* feat: remove unnecessary central_heat_vent config

* feat: pass heat vent marginal cost to prepare_sector_network

* refactor: rename marginal_cost_storage to marginal_cost_home_battery_storage

* docs: update configtables

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: update release notes

* add suggestions to ensure that charger and discharger align for charger ratio constraints and energy power constraints

* add lifetime to PTES and TTES charger/discharger, so that they are built with correct baseyear

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: add small penalty on ptes charging

* feat: update fixed cost to capital cost str

* feat: remove unnecessary passing of obsolete config

* feat: remove unneeded config passing

* feat: remove unneeded config passing

* feat: remove unneeded config passing

* Update scripts/solve_network.py

* Update solve_network.py

* feat: re-add TES necessary parameters

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* normalize by max temperature spread

* add district heating temperatures as input for prepare_sector_network

* adjust reference deltaT

* address review and allow for myopic foresight

* added docs and release note

* remove unused function arguments and snakemake inputs, add filepath to prevent call of snakemake object within function, revert mock_snakemake arguments

* address final review

* adjust .rst syntax

---------

Co-authored-by: Tom Kaehler <tom_khlr@icloud.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: TomKae00 <144129436+TomKae00@users.noreply.github.com>
Co-authored-by: Amos Schledorn <60692940+amos-schledorn@users.noreply.github.com>
Co-authored-by: Amos Schledorn <a.schledorn@tu-berlin.de>
Co-authored-by: cpschau <caspar.schauss@ieg.fraunhofer.de>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Co-authored-by: cpschau <caspar.schauss@ieg.fraunhofer.de>
* fix: Update tutorial paths to match config directory structure

Fix directory path inconsistency between tutorial commands and actual
file locations. The config sets `run.name: "test-elec"` and
`run.shared_resources.policy: "test"`, which affects where files are stored.

- Update all example commands to include correct paths (resources/test/...)
- Fix analysis example to use correct results path (results/test-elec/...)
- Add explanatory section about how config settings affect directory structure

Fixes [#1614](PyPSA/pypsa-eur#1614)

* Update doc/tutorial.rst

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* pass model_kwargs from config to optimize.create_model

* update default config for docs

* add items to configtables

* move config items to solving.options

* move model_kwargs to solving.options and add keep_files to kwargs

* move model_kwargs and keep_files to solving.options in docs
@lindnemi lindnemi marked this pull request as ready for review April 15, 2025 09:46
@lindnemi
Copy link
Collaborator Author

@lkstrp how to deactivate the automatic review requests to codeowners for the pypsa-de repository?

@lkstrp
Copy link
Member

lkstrp commented Apr 15, 2025

@lkstrp how to deactivate the automatic review requests to codeowners for the pypsa-de repository?

Remove the CODEOWNERS.md file. In the PyPSA-Eur master it is removed already. We kicked the codeowner concept again

Copy link
Contributor

@cpschau cpschau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested the branch and just ran into an issue in add_hydro_profile due to incomplete retrieval of files of the zenodo data bundle. This is PyPSA-Eur related, so I fixed it there:

PyPSA/pypsa-eur#1648

Rest of the workflow ran smoothly and results look plausible from district heating perspective.

lindnemi and others added 2 commits April 17, 2025 10:37
Co-authored-by: cpschau <caspar.schauss@ieg.fraunhofer.de>
@lindnemi
Copy link
Collaborator Author

superseded by #78

@lindnemi lindnemi closed this Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.