Adding PVGIS horizon data retrieval method (#1395)

* added horizon model first draft

* added get horizon function

* fixing stickler items & updates

* stickler whitespace

* finishing up stickler

* finishing up stickler, 2

* added optional deps, changed argument of horizon_map from path to array

* stickler

* Update pvlib/iotools/pvgis.py

Co-authored-by: Karel De Brabandere <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/horizon.py

Co-authored-by: Cliff Hansen <[email protected]>

* updates

* cleaning up stickler

* cleaning up stickler 2

* cleaning up stickler 3

* updated math for angle calculation

* removed old comment

* updated angle maximization method

* one of these days I'll remember stickler before pushing

* changed variable names to make clear that we're working in pixels

* added module to download SRTM DEM data

* spelling/spacing

* stickler

* stickler3

* fixed f string formatting

stickler doesn't like it but it doesn't work with spaces

* added horizon retrival, functions, dem retrival

* stickler

* stickler2

* stickler3

* Revert "merge2"

This reverts commit 6f9ad5e, reversing
changes made to 43951a2.

* fix formatting

* fix formatting, 2

* removing horizon.py

* tests added

* updated test

* fixed stickler

* minor formatting/stickler

* Update v0.9.6.rst

* Update iotools.rst

* Apply suggestions from code review

Co-authored-by: Kevin Anderson <[email protected]>

* Change assert to use np builtin

* correct docstring for hidden function

* Chnage proxy settings to passing through kwargs

* Changed proxy settings to pass kwargs

* Change assertion to use assert_series_equal from conftest

* Added horizon profile contributions

* Update v0.9.6.rst

* Delete cgiar.py

* Delete test_cgiar.py

* Update shading.py

* Update setup.py

* Update __init__.py

* Update iotools.rst

* Update test_pvgis.py

* return only two columns and makde bytes->utf-8 implicit

* Use JSON to streamline processing

* Update pvgis.py

* added url default info

* remove np dependency

* Apply suggestions from code review

Co-authored-by: Adam R. Jensen <[email protected]>

* Moved ref for consistency

* Updated name of horizon data variable

* Add CSV for horizon tests

* Test whole dataframe

* Change coordinate system to pvlib

* fix test to read in csv properly

* Apply suggestions from code review

Co-authored-by: Adam R. Jensen <[email protected]>

* Changed names for consistency

* Match column names in test dataframe

* stickler

* Set horizon_azimuth to the index

* Added invalid coords test

* update test to reflect change to index

* forgot xfail flag

* Update test_read_pvgis_horzion_invalid_coords

* return metadata object

* Update pvgis.py

* fix tests for metadata update

* Remove index column from test file

* Convert output to pd.Series

* Update tests to assert pd.Series

* Remove duplicate north value (0, 360)

---------

Co-authored-by: Ben <[email protected]>
Co-authored-by: Karel De Brabandere <[email protected]>
Co-authored-by: Cliff Hansen <[email protected]>
Co-authored-by: Pierce <[email protected]>
Co-authored-by: Kevin Anderson <[email protected]>
Co-authored-by: Adam R. Jensen <[email protected]>

Author: 7 people

docs/sphinx/source/reference/iotools.rst

@@ -31,6 +31,7 @@ of sources and file formats relevant to solar energy modeling.
    iotools.read_pvgis_tmy
    iotools.get_pvgis_hourly
    iotools.read_pvgis_hourly
+   iotools.get_pvgis_horizon
    iotools.get_bsrn
    iotools.read_bsrn
    iotools.parse_bsrn

docs/sphinx/source/whatsnew/v0.9.6.rst

@@ -21,6 +21,8 @@ Deprecations
 
 Enhancements
 ~~~~~~~~~~~~
+* Added function to retrieve horizon data from PVGIS 
+  :py:func:`pvlib.iotools.get_pvgis_horizon`. (:issue:`1290`, :pull:`1395`)
 * Added ``map_variables`` argument to the :py:func:`pvlib.iotools.read_tmy3` in
   order to offer the option of mapping column names to standard pvlib names.
   (:issue:`1517`, :pull:`1623`)
@@ -31,7 +33,6 @@ Enhancements
 * :py:func:`pvlib.iotools.get_psm3` now uses the new NSRDB 3.2.2 endpoint for
   hourly and half-hourly single-year datasets. (:issue:`1591`, :pull:`1736`)
 
-
 Bug fixes
 ~~~~~~~~~
 * `data` can no longer be left unspecified in
@@ -59,9 +60,17 @@ Contributors
 ~~~~~~~~~~~~
 * Lakshya Garg (:ghuser:`Lakshyadevelops`)
 * Adam R. Jensen (:ghuser:`adamrjensen`)
+* Ben Pierce (:ghuser:`bgpierc`)
+* Joseph Palakapilly (:ghuser:`JPalakapillyKWH`)
+* Cliff Hansen (:ghuser:`cwhanse`)
+* Anton Driesse (:ghuser:`adriesse`)
+* Will Holmgren (:ghuser:`wholmgren`)
+* Mark Mikofski (:ghuser:`mikofski`)
+* Karel De Brabandere (:ghuser:`kdebrab`)
+* Josh Stein (:ghuser:`jsstein`)
+* Kevin Anderson (:ghuser:`kandersolar`)
 * Siddharth Kaul (:ghuser:`k10blogger`)
 * Kshitiz Gupta (:ghuser:`kshitiz305`)
 * Stefan de Lange (:ghuser:`langestefan`)
 * :ghuser:`ooprathamm`
 * Kevin Anderson (:ghuser:`kandersolar`)
-

pvlib/data/test_read_pvgis_horizon.csv

@@ -0,0 +1,49 @@
+horizon_azimuth,horizon_elevation
+0,9.9
+7.5,13
+15,14.5
+22.5,15.7
+30,14.9
+37.5,15.3
+45,15.7
+52.5,15.7
+60,13
+67.5,11.5
+75,11.1
+82.5,11.5
+90,10.3
+97.5,11.5
+105,10.3
+112.5,9.5
+120,10.7
+127.5,11.8
+135,11.8
+142.5,8.8
+150,8.4
+157.5,7.3
+165,5.7
+172.5,5.7
+180,4.6
+187.5,3.4
+195,0.8
+202.5,0
+210,0
+217.5,0
+225,0
+232.5,0
+240,0
+247.5,0
+255,0
+262.5,0
+270,0
+277.5,0
+285,0
+292.5,0
+300,0
+307.5,0
+315,1.1
+322.5,1.9
+330,3.8
+337.5,5
+345,6.5
+352.5,9.2

pvlib/iotools/__init__.py

@@ -15,6 +15,7 @@
 from pvlib.iotools.pvgis import get_pvgis_tmy, read_pvgis_tmy  # noqa: F401
 from pvlib.iotools.pvgis import read_pvgis_hourly  # noqa: F401
 from pvlib.iotools.pvgis import get_pvgis_hourly  # noqa: F401
+from pvlib.iotools.pvgis import get_pvgis_horizon  # noqa: F401
 from pvlib.iotools.bsrn import get_bsrn  # noqa: F401
 from pvlib.iotools.bsrn import read_bsrn  # noqa: F401
 from pvlib.iotools.bsrn import parse_bsrn  # noqa: F401

pvlib/iotools/pvgis.py

@@ -665,3 +665,57 @@ def read_pvgis_tmy(filename, pvgis_format=None, map_variables=None):
         data = data.rename(columns=VARIABLE_MAP)
 
     return data, months_selected, inputs, meta
+
+
+def get_pvgis_horizon(latitude, longitude, url=URL, **kwargs):
+    """Get horizon data from PVGIS.
+
+    Parameters
+    ----------
+    latitude : float
+        Latitude in degrees north
+    longitude : float
+        Longitude in degrees east
+    url: str, default: :const:`pvlib.iotools.pvgis.URL`
+        Base URL for PVGIS
+    kwargs:
+        Passed to requests.get
+
+    Returns
+    -------
+    data : pd.Series
+        Pandas Series of the retrived horizon elevation angles. Index is the
+        corresponding horizon azimuth angles.
+    metadata : dict
+        Metadata returned by PVGIS.
+
+    Notes
+    -----
+    The horizon azimuths are specified clockwise from north, e.g., south=180.
+    This is the standard pvlib convention, although the PVGIS website specifies
+    south=0.
+
+    References
+    ----------
+    .. [1] `PVGIS horizon profile tool
+       <https://ec.europa.eu/jrc/en/PVGIS/tools/horizon>`_
+    """
+    params = {'lat': latitude, 'lon': longitude, 'outputformat': 'json'}
+    res = requests.get(url + 'printhorizon', params=params, **kwargs)
+    if not res.ok:
+        try:
+            err_msg = res.json()
+        except Exception:
+            res.raise_for_status()
+        else:
+            raise requests.HTTPError(err_msg['message'])
+    json_output = res.json()
+    metadata = json_output['meta']
+    data = pd.DataFrame(json_output['outputs']['horizon_profile'])
+    data.columns = ['horizon_azimuth', 'horizon_elevation']
+    # Convert azimuth to pvlib convention (north=0, south=180)
+    data['horizon_azimuth'] += 180
+    data.set_index('horizon_azimuth', inplace=True)
+    data = data['horizon_elevation']  # convert to pd.Series
+    data = data[data.index < 360]  # remove duplicate north point (0 and 360)
+    return data, metadata

pvlib/tests/iotools/test_pvgis.py

@@ -9,8 +9,9 @@
 import requests
 from pvlib.iotools import get_pvgis_tmy, read_pvgis_tmy
 from pvlib.iotools import get_pvgis_hourly, read_pvgis_hourly
+from pvlib.iotools import get_pvgis_horizon
 from ..conftest import (DATA_DIR, RERUNS, RERUNS_DELAY, assert_frame_equal,
-                        fail_on_pvlib_version)
+                        fail_on_pvlib_version, assert_series_equal)
 from pvlib._deprecation import pvlibDeprecationWarning
 
 
@@ -509,6 +510,23 @@ def test_get_pvgis_map_variables(pvgis_tmy_mapped_columns):
     assert all([c in pvgis_tmy_mapped_columns for c in actual.columns])
 
 
+@pytest.mark.remote_data
+@pytest.mark.flaky(reruns=RERUNS, reruns_delay=RERUNS_DELAY)
+def test_read_pvgis_horizon():
+    pvgis_data, _ = get_pvgis_horizon(35.171051, -106.465158)
+    horizon_data = pd.read_csv(DATA_DIR / 'test_read_pvgis_horizon.csv',
+                               index_col=0)
+    horizon_data = horizon_data['horizon_elevation']
+    assert_series_equal(pvgis_data, horizon_data)
+
+
+@pytest.mark.remote_data
+@pytest.mark.flaky(reruns=RERUNS, reruns_delay=RERUNS_DELAY)
+def test_read_pvgis_horizon_invalid_coords():
+    with pytest.raises(requests.HTTPError, match='lat: Incorrect value'):
+        _, _ = get_pvgis_horizon(100, 50)  # unfeasible latitude
+
+
 def test_read_pvgis_tmy_map_variables(pvgis_tmy_mapped_columns):
     fn = DATA_DIR / 'tmy_45.000_8.000_2005_2016.json'
     actual, _, _, _ = read_pvgis_tmy(fn, map_variables=True)