Nrel atb usa costs (#160)

* code: re-locate imports

* data: add ATB electricity inputs for 2022 and 2024

* code: introduce new script compile_cost_assumptions_nrel.py

* code: fix imports in _helpers.py

* code: filter columns and core_metric_parameter

* code: new changes dropping duplicates

* code: add additional OCC as core_metric_parameter

* code: %-yr

* code: newest changes to nrel

* code: filter and rename technologies

* code-config: update config and compile_cost_assumptions_nrel.py

* code: rename columns

* code: add unit tests read_atb_input_file

* code: re-factor

* code: unit test calculate_fom_percentage

* code: new progress

* code: add unit tests for repeat_values

* code: add unit tests for fom_percentage

* code: modify unit test for fom percentage

* code: update documentation

* code: modify code

* code: re-work code

* code: remove unnecessary print from unit test file

* data: add technical data for CCS-equipped power plants and update coal efficiency

* data: coal and gas retrofits

* data: pypsa battery changed to 'Utility-Scale_Battery_Storage_6Hr_Battery_Storage'

* data: updated technology names to displayname ATB, added discount rates

* code: remove WACC_Real

* code: update scripts/compile_cost_assumptions_nrel.py

* code: updated NREL ATB nomenclature according to 'displayname'

* data: removed unnecessary data for future years

* data: added hydrogen production technologies

* code: update on including atb costs into existing costs

* code: atb costs included in existing costs

* data: updated header column 'financial_case'

* code: change in query

* data: updated nomenclature 'further description' in 'discount_rates_usa.csv'

* code: include discount rate

* code: separate add discount rate as external function

* code: remove unnecessary outputs

* data: add output US

* doc: update release_notes

* config: remove unnecessary configs

* code: remove pypsa dependency

* code: add get_query_string

* doc: solve typo

* code: change comment

* docu: update documentation

* data: correct mistake in manual_input.csv

* doc: replace PyPSA-Eur-sec with PyPSA-Earth in README.md

* code: substitute from pypsa.descriptors import Dict with Dict=dict in _helpers.py

* data: added missing currency year for hydrogen production technologies

* code: remove index from NREL output files

* code: add units for CF and modify unit for FOM

* code: modify from $ and € to USD and EUR

* data: modified € to EUR

* data: add efficiency for Coal-IGCC-90%-CCS and modify efficiency for Coal-IGCC

* data: add ICCT data for hydrogen production & storage, FT and DAC

* doc: update release notes

* code: fix bug in queries

* data: add US-specific fuel costs for coal, gas and oil

* code: remove tax_credit_case

* code: update __init__

* code: Nuclear - AP1000 to Nuclear - AP100

* code: match technology names from ATB2022 to those in ATB2024

* data: add US-specific discount rate for 2020

* data: remove scenario and financial_case from fuel_costs_usa.csv

* data: simplify names for hydrogen production technologies

* code: remove unnecessary technologies from atb_e_2022

* data: modify structure of discount rates and fuel costs files

* code: add fuel costs

* data: include newest values from fuel_costs

* data: correct source for gas fuel cost

* docstring: add docstring to the functions

* code: revert back to x instead of row

* doc: update doc strings

* doc: updated documentation

* code: update unit tests

* code: final touches

* data: fix currency years

* data: remove duplicates from discount_rates_usa

* code: remove uncessary file and spaces

* code: change fixture scope

* data: add outputs folder

* remove unnecessary file

* code: remove duplicated before concat with discount_rates and fuel_costs. Plus rename file

* code: modify .gitignore

* remove .DS_Store

* code: cast value to float

* code: remove unnecessary white-spaces

* env: add pytest to environment.yaml

* add licenses

* data: update calculation coal cost

* data: update reference for coal costs

* data: correct unit for USA oil and gas costs

* config: remove fastparquet

* remove hash

* code: fix typo in get_conversion_dictionary

* code: changes to convertion --> conversion and fix unit tests

* comment to PR

* code: explicitly mention the origin of NREL ATB input files

* update output cost files

* code: fix issues in variables naming

---------

Co-authored-by: Daniele Lerede <[email protected]>
Co-authored-by: Fabrizio Finozzi <[email protected]>

Author: 3 people

.gitignore

@@ -2,6 +2,7 @@
 #
 # SPDX-License-Identifier: GPL-3.0-only
 
+.DS_Store
 .snakemake
 _build
 .idea

README.md

@@ -21,8 +21,7 @@ standardized cost years, technology names, units and source information. For fur
 
 
 The outputs are used in
-[PyPSA-Eur](https://github.com/PyPSA/pypsa-eur) and
-[PyPSA-Eur-Sec](https://github.com/PyPSA/pypsa-eur-sec).
+[PyPSA-Eur](https://github.com/PyPSA/pypsa-eur) and [PyPSA-Earth](https://github.com/pypsa-meets-earth/pypsa-earth).
 
 
 ## Licence

Snakefile

@@ -36,6 +36,26 @@ rule compile_cost_assumptions:
         "scripts/compile_cost_assumptions.py"
 
 
+rule compile_cost_assumptions_usa:
+    input:
+        cost_files_to_modify=expand("outputs/costs_{year}.csv", year=config["years"]),
+        nrel_atb_input_files=expand(
+            "inputs/atb_e_{year}.parquet",
+            year=config["nrel_atb"]["nrel_atb_input_years"],
+        ),
+        nrel_atb_input_discount_rate="inputs/discount_rates_usa.csv",
+        nrel_atb_input_fuel_costs="inputs/fuel_costs_usa.csv",
+    output:
+        expand("outputs/US/costs_{year}.csv", year=config["years"]),
+    threads: 1
+    resources:
+        mem=500,
+    conda:
+        "environment.yaml"
+    script:
+        "scripts/compile_cost_assumptions_usa.py"
+
+
 # rule convert_fraunhofer:
 #     input:
 #         fraunhofer = "docu/Anhang-Studie-Wege-zu-einem-klimaneutralen-Energiesystem.pdf"

config.yaml

@@ -7,6 +7,13 @@ version: 0.9.2
 # considered years for output data
 years: [2020, 2025, 2030, 2035, 2040, 2045, 2050]
 
+nrel_atb:
+  nrel_atb_input_years: [2022, 2024]
+  nrel_atb_columns_to_keep: ["atb_year", "core_metric_parameter", "core_metric_case", "core_metric_variable", "technology", "technology_alias", "display_name", "scenario", "units", "value"]
+  nrel_atb_core_metric_parameter_to_keep: ["CAPEX", "CF", "Fixed O&M", "Variable O&M", "Fuel", "Additional OCC"]
+  nrel_atb_technology_to_remove: ["Coal-CCS-95% -> Transformational Tech", "Coal-Max-CCS -> Transformational Tech", "Coal-new -> Transformational Tech", "NG combined cycle 95% CCS (F-frame basis -> Transformational Tech)", "NG combined cycle 95% CCS (H-frame basis -> Transformational Tech)", "NG combined cycle Max CCS (F-frame basis -> Transformational Tech)", "NG combined cycle Max CCS (H-frame basis -> Transformational Tech)", "AEO"]
+  nrel_atb_source_link: "NREL/ATB-https://data.openei.org/s3_viewer?bucket=oedi-data-lake&prefix=ATB%2Felectricity%2Fcsv%2F2022%2F"
+
 expectation: ""  # tech data uncertainty, possible options [None, "optimist", "pessimist"]
 
 #year for EUR outputs

docs/release_notes.rst

@@ -23,7 +23,7 @@ Release Notes
 
 * fixed DEA technology data sheet name for central water tank storage to point to actual PTES data
 
-* Fix minor issues in the code
+* added geothermal district heating as `central geothermal-sourced heat pump` based on DEA technology catalogue ("45.1.a Geothermal DH, 1200m, E")
 
 * fix minor issues in the code
 
@@ -35,6 +35,13 @@ Release Notes
 
 * add pre-commit
 
+* include NREL/ATB costs for electricity generation technologies in a dedicated US cost outputs
+
+* include ICCT techno-economic parameters for electrolyzers, hydrogen storage, Fischer-Tropsch, Direct air capture
+
+* include JRC data for fossil- and biomass-based hydrogen production technologies (with and without CCS)
+
+
 Technology-Data 0.9.2 (30 August 2024)
 ======================================
 

environment.yaml

@@ -13,12 +13,14 @@ dependencies:
 - snakemake-minimal>=8.5
 - pandas>=2.1
 - pypsa
+- pyarrow
 - numpy
 - beautifulsoup4
 - xlrd
 - scipy
 - openpyxl>=3.1.2
 - packaging
+- pytest
 
 # Development dependencies
 - pre-commit