|
| 1 | +================= |
| 2 | +General Structure |
| 3 | +================= |
| 4 | + |
| 5 | +The dataset combines the data of all the data sources listed in |
| 6 | +`Data-Sources <#Data-Sources>`__ and provides the following information: |
| 7 | + |
| 8 | +- **Power plant name** - claim of each database |
| 9 | +- **Fueltype** - {Solid Biomass, Biogas, Geothermal, Hard Coal, Hydro, Lignite, |
| 10 | + Nuclear, Natural Gas, Oil, Solar, Wind, Other} |
| 11 | +- **Technology** - {CCGT, OCGT, Steam Turbine, Combustion Engine, |
| 12 | + Run-Of-River, Pumped Storage, Reservoir} |
| 13 | +- **Set** - {Power Plant (PP), Combined Heat and Power (CHP), Storages |
| 14 | + (Stores)} |
| 15 | +- **Capacity** - [MW] |
| 16 | +- **Duration** - Maximum state of charge capacity in terms of hours at |
| 17 | + full output capacity |
| 18 | +- **Dam Information** - Dam volume [Mm^3] and Dam Height [m] |
| 19 | +- **Geo-position** - Latitude, Longitude |
| 20 | +- **Country** - EU-27 + CH + NO (+ UK) minus Cyprus and Malta |
| 21 | +- **YearCommissioned** - Commmisioning year of the powerplant |
| 22 | +- **RetroFit** - Year of last retrofit |
| 23 | +- **projectID** - Immutable identifier of the power plant |
| 24 | + |
| 25 | + |
| 26 | +All data files of the package will be stored in the folder given by |
| 27 | +``pm.core.package_config['data_dir']`` |
| 28 | + |
| 29 | + |
| 30 | +Data Sources |
| 31 | +------------ |
| 32 | + |
| 33 | +- OPSD - `Open Power System |
| 34 | + Data <http://data.open-power-system-data.org/>`__ publish their |
| 35 | + `data <http://data.open-power-system-data.org/conventional_power_plants/>`__ |
| 36 | + under a free license |
| 37 | +- GEO - `Global Energy |
| 38 | + Observatory <http://globalenergyobservatory.org/>`__, the data is not |
| 39 | + directly available on the website, but can be obtained from an |
| 40 | + `sqlite |
| 41 | + scraper <https://morph.io/coroa/global_energy_observatory_power_plants>`__ |
| 42 | +- GPD - `Global Power Plant |
| 43 | + Database <http://datasets.wri.org/dataset/globalpowerplantdatabase>`__ |
| 44 | + provide their data under a free license |
| 45 | +- GBPT - `Global Bioenergy Powerplant Tracker by Global Energy Monitor<https://globalenergymonitor.org/projects/global-bioenergy-power-tracker/>`__ |
| 46 | +- GCPT - `Global Coal Powerplant Tracker by Global Energy Monitor <https://globalenergymonitor.org/projects/global-coal-plant-tracker/>`__ |
| 47 | +- GGPT - `Global Gas Powerplant Tracker by Global Energy Monitor <https://globalenergymonitor.org/projects/global-gas-plant-tracker/>`__ |
| 48 | +- GGTPT - `Global Geothermal Powerplant Tracker by Global Energy Monitor <https://globalenergymonitor.org/projects/global-geothermal-power-tracker/>`__ |
| 49 | +- GNPT - `Global Nuclear Powerplant Tracker by Global Energy Monitor <https://globalenergymonitor.org/projects/global-nuclear-power-tracker/>`__ |
| 50 | +- GSPT - `Global Solar Powerplant Tracker by Global Energy Monitor <https://globalenergymonitor.org/projects/global-solar-power-tracker/>`__ |
| 51 | +- GWPT - `Global Wind Powerplant Tracker by Global Energy Monitor <https://globalenergymonitor.org/projects/global-wind-power-tracker/>`__ |
| 52 | +- CARMA - `Carbon Monitoring for Action <http://carma.org/plant>`__ |
| 53 | +- ENTSOe - `European Network of Transmission System Operators for |
| 54 | + Electricity <http://entsoe.eu/>`__, annually provides statistics |
| 55 | + about aggregated power plant capacities. Their data can be used as a |
| 56 | + validation reference. We further use their `annual energy generation |
| 57 | + report from |
| 58 | + 2010 <https://www.entsoe.eu/db-query/miscellaneous/net-generating-capacity>`__ |
| 59 | + as an input for the hydro power plant classification. The `power |
| 60 | + plant |
| 61 | + dataset <https://transparency.entsoe.eu/generation/r2/installedCapacityPerProductionUnit/show>`__ |
| 62 | + on the ENTSO-E transparency website is downloaded using the `ENTSO-E |
| 63 | + Transparency |
| 64 | + API <https://transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html>`__. |
| 65 | +- JRC - `Joint Research Centre Hydro-power plants |
| 66 | + database <https://github.com/energy-modelling-toolkit/hydro-power-database>`__ |
| 67 | +- IRENA - `International Renewable Energy |
| 68 | + Agency <http://resourceirena.irena.org/gateway/dashboard/>`__ open |
| 69 | + available statistics on power plant capacities. |
| 70 | +- BNETZA - |
| 71 | + `Bundesnetzagentur <https://www.bundesnetzagentur.de/EN/Areas/Energy/Companies/SecurityOfSupply/GeneratingCapacity/PowerPlantList/PubliPowerPlantList_node.html>`__ |
| 72 | + open available data source for Germany’s power plants |
| 73 | +- UBA (Umwelt Bundesamt Datenbank “Kraftwerke in Deutschland) |
| 74 | + |
| 75 | +Not available but supported sources: |
| 76 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 77 | + |
| 78 | +- IWPDCY (International Water Power & Dam Country Yearbook) |
| 79 | +- WEPP (Platts, World Elecrtric Power Plants Database) |
| 80 | + |
| 81 | + |
| 82 | +Reliabilty Score |
| 83 | +---------------- |
| 84 | + |
| 85 | +When the matched power plant entries from different sources are combined, the resulting value per column is determined by the most reliable source. The corresponding reliability scores |
| 86 | +are: |
| 87 | + |
| 88 | +======= ================ |
| 89 | +Dataset Reliabilty score |
| 90 | +======= ================ |
| 91 | +JRC 6 |
| 92 | +ESE 6 |
| 93 | +UBA 5 |
| 94 | +OPSD 5 |
| 95 | +OPSD_EU 5 |
| 96 | +OPSD_DE 5 |
| 97 | +WEPP 4 |
| 98 | +ENTSOE 4 |
| 99 | +IWPDCY 3 |
| 100 | +GPD 3 |
| 101 | +GEO 3 |
| 102 | +BNETZA 3 |
| 103 | +CARMA 1 |
| 104 | +======= ================ |
| 105 | + |
| 106 | + |
| 107 | + |
| 108 | +How it works |
| 109 | +------------ |
| 110 | + |
| 111 | +Whereas single databases as the CARMA, GEO or the OPSD database provide |
| 112 | +non standardized and incomplete information, the datasets can complement |
| 113 | +each other and improve their reliability. In a first step, |
| 114 | +powerplantmatching converts all powerplant dataset into a standardized |
| 115 | +format with a defined set of columns and values. The second part |
| 116 | +consists of aggregating power plant blocks together into units. Since |
| 117 | +some of the datasources provide their powerplant records on unit level, |
| 118 | +without detailed information about lower-level blocks, comparing with |
| 119 | +other sources is only possible on unit level. In the third and |
| 120 | +name-giving step the tool combines (or matches)different, standardized |
| 121 | +and aggregated input sources keeping only powerplants units which appear |
| 122 | +in more than one source. The matched data afterwards is complemented by |
| 123 | +data entries of reliable sources which have not matched. |
| 124 | + |
| 125 | +The aggregation and matching process heavily relies on |
| 126 | +`DUKE <https://github.com/larsga/Duke>`__, a java application |
| 127 | +specialized for deduplicating and linking data. It provides many |
| 128 | +built-in comparators such as numerical, string or geoposition |
| 129 | +comparators. The engine does a detailed comparison for each single |
| 130 | +argument (power plant name, fuel-type etc.) using adjusted comparators |
| 131 | +and weights. From the individual scores for each column it computes a |
| 132 | +compound score for the likeliness that the two powerplant records refer |
| 133 | +to the same powerplant. If the score exceeds a given threshold, the two |
| 134 | +records of the power plant are linked and merged into one data set. |
| 135 | + |
| 136 | +Let’s make that a bit more concrete by giving a quick example. Consider |
| 137 | +the following two data sets |
| 138 | + |
| 139 | +Dataset 1: |
| 140 | +~~~~~~~~~~ |
| 141 | + |
| 142 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 143 | +| | Name | Fueltype | Classification | Country | Capacity | lat | lon | File | |
| 144 | ++===+==========+==========+================+=============+==========+=========+============+======+ |
| 145 | +| 0 | Aarberg | Hydro | nan | Switzerland | 14.609 | 47.0444 | 7.27578 | nan | |
| 146 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 147 | +| 1 | Abbey | Oil | nan | United | 6.4 | 51.687 | -0.0042057 | nan | |
| 148 | +| | mills | | | Kingdom | | | | | |
| 149 | +| | pumping | | | | | | | | |
| 150 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 151 | +| 2 | Abertay | Other | nan | United | 8 | 57.1785 | -2.18679 | nan | |
| 152 | +| | | | | Kingdom | | | | | |
| 153 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 154 | +| 3 | Aberthaw | Coal | nan | United | 1552.5 | 51.3875 | -3.40675 | nan | |
| 155 | +| | | | | Kingdom | | | | | |
| 156 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 157 | +| 4 | Ablass | Wind | nan | Germany | 18 | 51.2333 | 12.95 | nan | |
| 158 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 159 | +| 5 | Abono | Coal | nan | Spain | 921.7 | 43.5588 | -5.72287 | nan | |
| 160 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 161 | + |
| 162 | +and |
| 163 | + |
| 164 | +Dataset 2: |
| 165 | +~~~~~~~~~~ |
| 166 | + |
| 167 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 168 | +| | Name | Fueltype | Classification | Country | Capacity | lat | lon | File | |
| 169 | ++===+==========+==========+================+=============+==========+=========+============+======+ |
| 170 | +| 0 | Aarberg | Hydro | nan | Switzerland | 15.5 | 47.0378 | 7.272 | nan | |
| 171 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 172 | +| 1 | Aberthaw | Coal | Thermal | United | 1500 | 51.3873 | -3.4049 | nan | |
| 173 | +| | | | | Kingdom | | | | | |
| 174 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 175 | +| 2 | Abono | Coal | Thermal | Spain | 921.7 | 43.5528 | -5.7231 | nan | |
| 176 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 177 | +| 3 | Abwinden | Hydro | nan | Austria | 168 | 48.248 | 14.4305 | nan | |
| 178 | +| | asten | | | | | | | | |
| 179 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 180 | +| 4 | Aceca | Oil | CHP | Spain | 629 | 39.941 | -3.8569 | nan | |
| 181 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 182 | +| 5 | Aceca | Natural | CCGT | Spain | 400 | 39.9427 | -3.8548 | nan | |
| 183 | +| | fenosa | gas | | | | | | | |
| 184 | ++---+----------+----------+----------------+-------------+----------+---------+------------+------+ |
| 185 | + |
| 186 | +where Dataset 2 has the higher reliability score. Apparently entries 0, |
| 187 | +3 and 5 of Dataset 1 relate to the same power plants as the entries 0, 1 |
| 188 | +and 2 of Dataset 2. The toolset detects those similarities and combines |
| 189 | +them into the following set, but prioritising the values of Dataset 2: |
| 190 | + |
| 191 | ++---+----------+----------------+----------+----------------+----------+---------+---------+------+ |
| 192 | +| | Name | Country | Fueltype | Classification | Capacity | lat | lon | File | |
| 193 | ++===+==========+================+==========+================+==========+=========+=========+======+ |
| 194 | +| 0 | Aarberg | Switzerland | Hydro | nan | 15.5 | 47.0378 | 7.272 | nan | |
| 195 | ++---+----------+----------------+----------+----------------+----------+---------+---------+------+ |
| 196 | +| 1 | Aberthaw | United Kingdom | Coal | Thermal | 1500 | 51.3873 | -3.4049 | nan | |
| 197 | ++---+----------+----------------+----------+----------------+----------+---------+---------+------+ |
| 198 | +| 2 | Abono | Spain | Coal | Thermal | 921.7 | 43.5528 | -5.7231 | nan | |
| 199 | ++---+----------+----------------+----------+----------------+----------+---------+---------+------+ |
0 commit comments