Closed
Conversation
* Update locked environment files for all platforms * Add trigger section to pixi.lock * Remove Trigger section from pixi.lock --------- Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me>
* Ensure cop_heat_pump always Pandas DataFrame * Reduce dataframe back to scalar after clipping. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add release note. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* initial dsm implementation for the heating sector * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * assume thermal losses in buildings and remove heating dsm from default * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * improve heat dsr documentation * fix for heat_dsm_profile for leap year (PyPSA#1534) * fix for heat_dsm_profile for leap year * add fneum code suggestion * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * CI related * working workflow * working workflow * adding documentation * Fixing CI and adding impact assessment * addressing comments * deleting report * Fixing merge and addressing comments * support different dsm potentials for different planning horizons * config settings * improve and adjust documentation * Larger example * higher temporal resolution * comparing heat dsm vs no heat dsm * Fix docs: heat DSM restriction_time is timezone-adjusted * code: Restructure feature Rename config and restructure config. Add directionality option. * Fix colours for plotting * code: Consistent labelling of heat dsm * Update tests * minor typo * addressing comments * doc: Fix time description * doc: Update UTC/local time confusion * code: Fix linting complaints * pre-commit * Fix test config --------- Co-authored-by: martacki <martha.frysztacki@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com> Co-authored-by: euronion <johannes.hampp@openenergytransition.org> Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com>
* code: Versioning for hotmaps dataset * code: Fix CLI not working for foreign owned records * code: versioning for co2stop dataset * code: versioning for nitrogen statistics datset * code: fix wrong file name for REUSE.toml * code: versioning for NUTS 2013 and NUTS 2021 * code: Add limited debug mode for CLI script * code: Fix issues with datasets not being detected * code: Fix minor bugs on CLI and outputs * code: Versioning for eurostat energy balances * code: Remove obsolete retrieve script, now part of retrieve.smk * code: Versioning for eurostat household energy balances * code: Fix missing replacements for eurostat energy balances * code: Remove prompt for API key * code: Fix typo for enspreso biomass dataset * code: Fix issue with OSM dataset versioning * code: Versioning for WDPA and WDPA marine * code: Implement versioning for TYNDP base network * code: Fix bug missing osm_version in base_network.py * code: Properly fix the bug this time * code: Implement versioning for powerplants / powerplantsmatching * code/docs: Implement versioning for costs * code: Implement versioning for cutouts, move to data/ , deprecate shared_cutouts * Fix: Wrong URL for powerplants * code: Implement versioning for HDD and runoff per country (retrieval only) * code: Add build options for HDD and runoff per country (untested) * code: Add option for monthly retrieval of cutouts while building * code: Restore build_natura_raster.py script * code: Implement versioning for natura raster + reintegrate build natura raster script * data: Change URB POP dataset to Zenodo link * code: Adjust link for GEM GSPT * code: Versioning for GEM Europe Gas Tracker * doc: Correct license attributions in REUSE.toml for data/ * code: Implement versioning for BFS vehicle data * code: Implement versioning for BFS GDP and POP data * doc: Update data_inventory.csv for all the datasets added * code: Fix versions for HDD and runoff per country * code: Fix rules for creating HDD and runoff per country * code: Add special cutout config for HDD/runoff * code: Fix monthly_requests for cutouts * code: Add logic for building and versioning of mobility profiles from BASt * data: Update data for mobility profiles from BASt * code: Windows compatability for cutouts * code: Deprecate automatic cutout bounds * pre-commit formatting * code: Remove cutout dir configuration option * code: Fix option for archived mobility_profiles * code: Cleanup unused stub * config: Remove deprecated config option from configfiles * code: Remove prompt for API key * LUISA land cover database - data versioning * JRC-IDEES dataset versioning implementation * Remove obsolete script for retrieving jrc idees data * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scigrid gas network - versioning implementation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * doc: scigrid dataset updates to versions.csv * Synthetic electricity demand versioning * Copernicus Global land service versioning * changing not-supported to might-work flags for some datasets * Ship raster - versioning implementation * EEZ dataset versioning implementation * NUTS3 population dataset versioning * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * GDP per capita dataset versioning * WorldPop population count dataset versioning * EEA ghg emissions data versioning * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * GEBCO versioning * Attributed Ports data versioning * Updating URLs * CORINE dataset versioning * Emobility dataset versioning * Common function to unzip folders * H2 salt caverns data versioning * Update primary link for GHG emissions * Primary link update for population count dataset * Lau regions data versioning * Data versioning for Aquifer data BGR * Updating archive link for lau regions * Retrieving corine dataset from primary datasource * OSM boundaries primary data versioning * OSM boundaries archive link update * rename function * code: Fix enspreso_biomass dataset name * code: Improve WDPA dataset handling * code: Simplify zip file handling * code: Simplify eurostat balances access * code: Centrally configure http storage provider * code: Fix typo * code: Enable config of prepare_kwargs * Bug fix - remove duplicate input entries for rule build_shapes * Removing check for retrieve flag * Moving corine Apikey from file to config parameter * Polishing the code * Code polishing for aquifer data set * code: Remove unnecessary code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * code: simplify worldbank retrieval rule * change indexing method for rules * Remove code block for retrieving databundle * modifying the way to unzip folders * Code modifications * doc updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Script license * License correction * Doc formatting * code: Cleanup GEBCO retrieval * code: Always copy storage(...) retrieved data, manage keep_local centrally * code: remove params from retrieve_jrc_idees * code: remove params from retrieve_gas_infrastructure * code: remove params for retrieve_eez * code: Remove params for retrieve_osm_boundaries * code: Deprecate enable::retrieve, enable::retrieve_databundle, enable::retrieve_cost_data * code: addendum to centrally managing `keep_local` setting for storage(...) * code: formatting * doc: Update configtables for data:: config options * code: Cleanup old comments * doc: remove duplicate dataset * doc: Clarify `url` entry in data/versions.csv * Code cleanup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding comments * Versioning for dh_areas * Data versioning for geothermal heat utilisation potentials * Update links for archive versions of dh_areas and geothermal heat utilisation potentials * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * doc: Configuration entries for dh_areas and geothermal_heat_utilisation_potentials * code: Remove obsolete databundle retrieval * doc: Update retrieve.rst * doc: Remove outdated information and sites * doc: Update PR template with hint to instruction on data versioning * code: Fix config["enable"]["retrieve"] artifacts * code: Update paths for accessing retrieved gas network * code: Remove obsolete script * code: Remove validate_checksum as it keeps failing from too many requests * code: Fix links for web archive versions * doc: Update formatting * doc: try different table formatting * code: Increase retries on retrieve * code: Fix in biomass potentials * code: Modify code to reference variable instead of index * code: Bug fix * code: correction to PyPSA set_option * remove conda in snakemake rule * code: Set max_requests_per_second * code: Adding versioning for JRC Ardeco dataset * code: JRC Ardeco URL updates * code: plot balance map bug fix * Update pypsa version to 0.33.2 * code: Cleanup ARDECO retrieval * code: Fix ARDECO retrieval rules * code: Adjust http storage limits to avoid Zenodo rate limit * debug: Extract information about Zenodo cert * Switch to primary sources for BFS data to avoid web archive retry limits * code: See if we can extract the Zenodo rate limit * code: Only request head * code: Align retrieve_corine with rest of code * code: Remove superflous parts * code: Add modified snakemake storage plugin * code: Fix typo * wip * WIP stuff * code: Bugfix workflow * nuclear approach to rate limiting errors * code: Fix EEZ retrieval from archive * feat: Add snakemake storage plugin for downloading zenodo files with ratelimit backoff, concurrent downloading and progress bars. * Update retrieve.smk to rely on zenodo storage plugin * Make pre-commit happy * fix url substring sanitization * Use tqdm-loggable to avoid flooding github logs * Add retries option to zenodo storage provider * fix: merge conflict to accomodate new rule build_dh_areas * fix lau_regions zenodo url * Re-add zenodo checksums (retrieved from api) and pull retrying into storage plugin * Replace aiohttp with httpx better connection pooling and timeout handling. * Release the stream a tiny bit earlier * fix: do cache data_versions for lru_cache to work the method needs to be at the toplevel. * Allow deactivating the cache and skipping all remote checks Use these settings for ci, since we already cache the resulting data folder. * Derive from settings from SettingsBase and clean up some docs * Revert "debug: Extract information about Zenodo cert" This reverts commit c1edb28. * fix: address unfounded fears * improve docstrings * feat: improve checksumming by allowing any digest * Update README.md * Expand user in `cache` directory * fix: missed fixes * cleanup * refactor: move storage plugin to own repo * refactor: rename `version_latest` to `version` * feat: allow passing corine secret via envvar * refactor: move zenodo cli to dedicated `utils` dir * Skip storage plugin cache in CI * feat: extract aquifer_data onaccess saves space. * fix: use vsizip to transparently decompress nuts3 2021 * fix: save synthetic electricity demand as csv.gz * fix: clean conda caches after update * Revert "fix: save synthetic electricity demand as csv.gz" This reverts commit 0ebc02c. * Revert "fix: use vsizip to transparently decompress nuts3 2021" This reverts commit 9e6cd7f. * Revert "feat: extract aquifer_data onaccess" This reverts commit 6de88a2. * remove tenacity and old checksum/ internet helpers * Adjust file paths to work on Windows Minor changes: - set paths to always use posix format * Fix file paths on windows nicely (#71) * fix: use string paths with / even on windows * feat: Add pixi toml and lock * fix: fix retrieve_worldbank_urban_population * Update rules/build_sector.smk * Revert "feat: Add pixi toml and lock" This reverts commit 9c9bf97. * code: Remove quotation marks in config * Update comment * Update storage plugin config * code: Update pixi dependencies * code: Fix wildcard * Update rules/build_electricity.smk Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * Update doc/release_notes.rst Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * update `pixi.lock` * remove snakemake-storage-plugin-cached-http install from pypi * Revert "update `pixi.lock`" This reverts commit 6cecd76. * revert updated lock file * fix pre-commit --------- Co-authored-by: Sermisha <sermisha.narayana@openenergytransition.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sermisha <156903227+SermishaNarayana@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me> Co-authored-by: Jonas Hoersch <jonas.hoersch@openenergytransition.org> Co-authored-by: Jonas Hörsch <coroa@posteo.de> Co-authored-by: Eric Nitschke <eric.nitschke@tuhh.de> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* Fix BFS Swiss transport parsing * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add release notes --------- Co-authored-by: Danele Lerede <danelelerede@Martins-MacBook-Air.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix 2.0 build_energy_totals for CH. * Updated release notes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Set mock_snakemake to rule defaults. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…25-12-08 - fixed merge conflict in PR template by keeping OET-specific checkbox but replacing data source checkbox - fixed RST URL ref in PR template markdown file - fixed merge conflict in REUSE.toml by keeping both OET-specific entry and all new upstream entries
8 tasks
Member
Author
|
@LukasFrankenQ any idea why your updates to biomass existing capacity would be failing in our CI? |
Member
Hi, @brynpickering. It might be related to having additional functionality in |
Member
Author
|
Thanks @yerbol-akhmetov , I hadn't realised we had diverged so much in |
3 tasks
Member
|
Contributed the PR template fix upstream, see PyPSA#1957 |
Member
|
Closed by #80. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merging PR to replace #79 (taking a different approach so I have write privileges to handle merge conflicts).
Merge conflicts faced
data/versions.csv(new file but ignored in tracking by gitignore)Changes proposed in this Pull Request (from upstream)
Checklist
pixi.toml(usingpixi add <dependency-name>).config/config.default.yaml.doc/configtables/*.csv.these instructions <https://pypsa-eur.readthedocs.io/en/latest/data_sources.html>_ have been followed.doc/release_notes.rstis added.