Skip to content

Commit 491eb5c

Browse files
committed
remove usage of qwdata
1 parent 4b3a3e8 commit 491eb5c

File tree

3 files changed

+21
-135
lines changed

3 files changed

+21
-135
lines changed

README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44
![Conda Version](https://img.shields.io/conda/v/conda-forge/dataretrieval)
55
![Downloads](https://static.pepy.tech/badge/dataretrieval)
66

7-
:warning: USGS data availability and format are changing on Water Quality Portal (WQP). Since March 2024, data obtained from WQP legacy profiles will not include new USGS data or recent updates to existing data.
7+
:warning: USGS data availability and format have changed on Water Quality Portal (WQP). Since March 2024, data obtained from WQP legacy profiles will not include new USGS data or recent updates to existing data. All USGS data (up to and beyond March 2024) are available using the new WQP beta services. You can access the beta services by setting `legacy=False` in the functions in the `wqp` module.
8+
89
To view the status of changes in data availability and code functionality, visit: https://doi-usgs.github.io/dataRetrieval/articles/Status.html
910

1011
:mega: **09/03/2024:** The groundwater levels service has switched endpoints, and `dataretrieval` was updated accordingly in [`v1.0.10`](https://github.com/DOI-USGS/dataretrieval-python/releases/tag/v1.0.10). Older versions using the discontinued endpoint will return 503 errors for `nwis.get_gwlevels` or the `service='gwlevels'` argument. Visit [Water Data For the Nation](https://waterdata.usgs.gov/blog/wdfn-waterservices-2024/) for more information.
@@ -34,15 +35,11 @@ import dataretrieval.nwis as nwis
3435
# specify the USGS site code for which we want data.
3536
site = '03339000'
3637

37-
3838
# get instantaneous values (iv)
3939
df = nwis.get_record(sites=site, service='iv', start='2017-12-31', end='2018-01-01')
4040

41-
# get water quality samples (qwdata)
42-
df2 = nwis.get_record(sites=site, service='qwdata', start='2017-12-31', end='2018-01-01')
43-
4441
# get basic info about the site
45-
df3 = nwis.get_record(sites=site, service='site')
42+
df2 = nwis.get_record(sites=site, service='site')
4643
```
4744
Services available from NWIS include:
4845
- instantaneous values (iv)
@@ -51,7 +48,10 @@ Services available from NWIS include:
5148
- site info (site)
5249
- discharge peaks (peaks)
5350
- discharge measurements (measurements)
54-
* water quality samples (qwdata)
51+
52+
Water quality data are available from:
53+
- [Samples](https://waterdata.usgs.gov/download-samples/#dataProfile=site) - Discrete USGS water quality data only
54+
- [Water Quality Portal](https://www.waterqualitydata.us/) - Discrete water quality data from USGS and EPA. Older data are available in the legacy WQX version 2 format; all data are available in the beta WQX3.0 format.
5555

5656
To access the full functionality available from NWIS web services, nwis.get record appends any additional kwargs into the REST request. For example
5757
```python

dataretrieval/nwis.py

Lines changed: 6 additions & 118 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,6 @@
3636

3737
WATERSERVICES_SERVICES = ["dv", "iv", "site", "stat"]
3838
WATERDATA_SERVICES = [
39-
"qwdata",
4039
"gwlevels",
4140
"measurements",
4241
"peaks",
@@ -135,125 +134,14 @@ def get_qwdata(
135134
**kwargs,
136135
) -> Tuple[pd.DataFrame, BaseMetadata]:
137136
"""
138-
Get water sample data from qwdata service.
139-
140-
.. warning::
141-
142-
WARNING: Beginning in March 2024 the NWIS qw data endpoint will
143-
not deliver new data or updates to existing data.
144-
Eventually the endpoint will be retired. For updated information visit:
145-
https://waterdata.usgs.gov.nwis/qwdata
146-
For additional details, see the R package vignette:
147-
https://doi-usgs.github.io/dataRetrieval/articles/Status.html
148-
If you have additional questions about the qw data service,
149-
email CompTools@usgs.gov.
150-
151-
Parameters
152-
----------
153-
sites: string or list of strings, optional, default is None
154-
If the qwdata parameter site_no is supplied, it will overwrite the
155-
sites parameter
156-
start: string, optional, default is None
157-
If the qwdata parameter begin_date is supplied, it will overwrite the
158-
start parameter (YYYY-MM-DD)
159-
end: string, optional, default is None
160-
If the qwdata parameter end_date is supplied, it will overwrite the
161-
end parameter (YYYY-MM-DD)
162-
multi_index: bool, optional
163-
If False, a dataframe with a single-level index (datetime) is returned,
164-
default is True
165-
wide_format : bool, optional
166-
If True, return data in wide format with multiple samples per row and
167-
one row per time, default is True
168-
datetime_index : bool, optional
169-
If True, create a datetime index, default is True
170-
ssl_check: bool, optional
171-
If True, check SSL certificates, if False, do not check SSL,
172-
default is True
173-
**kwargs: optional
174-
If supplied, will be used as query parameters
175-
176-
Returns
177-
-------
178-
df: ``pandas.DataFrame``
179-
Times series data from the NWIS JSON
180-
md: :obj:`dataretrieval.utils.Metadata`
181-
A custom metadata object
182-
183-
Examples
184-
--------
185-
.. doctest::
186-
187-
>>> # get water sample information for site 11447650
188-
>>> df, md = dataretrieval.nwis.get_qwdata(
189-
... sites="11447650", start="2010-01-01", end="2010-02-01"
190-
... )
137+
Get water sample data from qwdata service - deprecated, use `get_usgs_samples()`
138+
in the samples module.
191139
192140
"""
193-
warnings.warn(
194-
(
195-
"WARNING: Starting in March 2024, the NWIS qw data endpoint is "
196-
"retiring and no longer receives updates. For more information, "
197-
"refer to https://waterdata.usgs.gov.nwis/qwdata and "
198-
"https://doi-usgs.github.io/dataRetrieval/articles/Status.html "
199-
"or email CompTools@usgs.gov."
200-
)
201-
)
202-
203-
_check_sites_value_types(sites)
204-
205-
kwargs["site_no"] = kwargs.pop("site_no", sites)
206-
kwargs["begin_date"] = kwargs.pop("begin_date", start)
207-
kwargs["end_date"] = kwargs.pop("end_date", end)
208-
kwargs["multi_index"] = multi_index
209-
if wide_format:
210-
kwargs["qw_sample_wide"] = "qw_sample_wide"
211-
212-
payload = {
213-
"agency_cd": "USGS",
214-
"format": "rdb",
215-
"pm_cd_compare": "Greater than",
216-
"inventory_output": "0",
217-
"rdb_inventory_output": "file",
218-
"TZoutput": "0",
219-
"rdb_qw_attributes": "expanded",
220-
"date_format": "YYYY-MM-DD",
221-
"rdb_compression": "value",
222-
"submitted_form": "brief_list",
223-
}
224-
225-
# check for parameter codes, and reformat query args
226-
qwdata_parameter_code_field = "parameterCd"
227-
if kwargs.get(qwdata_parameter_code_field):
228-
parameter_codes = kwargs.pop(qwdata_parameter_code_field)
229-
parameter_codes = to_str(parameter_codes)
230-
kwargs["multiple_parameter_cds"] = parameter_codes
231-
kwargs["param_cd_operator"] = "OR"
232-
233-
search_criteria = kwargs.get("list_of_search_criteria")
234-
if search_criteria:
235-
kwargs["list_of_search_criteria"] = "{},{}".format(
236-
search_criteria, "multiple_parameter_cds"
237-
)
238-
else:
239-
kwargs["list_of_search_criteria"] = "multiple_parameter_cds"
240-
241-
kwargs.update(payload)
242-
243-
warnings.warn(
244-
"NWIS qw web services are being retired. "
245-
+ "See this note from the R package for more: "
246-
+ "https://doi-usgs.github.io/dataRetrieval/articles/qwdata_changes.html",
247-
category=DeprecationWarning,
248-
)
249-
response = query_waterdata("qwdata", ssl_check=ssl_check, **kwargs)
250-
251-
df = _read_rdb(response.text)
252-
253-
if datetime_index is True:
254-
df = format_datetime(df, "sample_dt", "sample_tm", "sample_start_time_datum_cd")
255-
256-
return format_response(df, **kwargs), NWIS_Metadata(response, **kwargs)
141+
return print("This function is deprecated and has been " \
142+
"replaced with `get_usgs_samples() in the " \
143+
"samples module. If you have questions, " \
144+
"please reach out to comptools@usgs.gov")
257145

258146

259147
def get_discharge_measurements(

demos/R Python Vignette equivalents.ipynb

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
"outputs": [],
1515
"source": [
1616
"from dataretrieval import nwis\n",
17+
"from dataretrieval import samples\n",
1718
"from dataretrieval import wqp"
1819
]
1920
},
@@ -45,8 +46,8 @@
4546
"\n",
4647
"# Sample data Nitrate:\n",
4748
"parameterCd <- \"00618\"\n",
48-
"qwData <- readNWISqw(siteNumber,parameterCd,\n",
49-
" \"1980-01-01\",\"2010-01-01\")\n",
49+
"qwData <- read_USGS_samples(monitoringLocationIdentifier=sprintf(\"USGS-%s\", siteNumber),usgsPCode=parameterCd,\n",
50+
" activityStartDateLower=\"1980-01-01\", activityStartDateUpper=\"2010-01-01\")\n",
5051
"\n",
5152
"pCode <- readNWISpCode(parameterCd)\n",
5253
"'''\n",
@@ -61,7 +62,7 @@
6162
"\n",
6263
"# sample data Nitrate:\n",
6364
"parameterCd = \"00618\"\n",
64-
"qwData, md = nwis.get_qwdata(sites=siteNumber, parameterCd=parameterCd, start=\"1980-01-01\", end=\"2010-01-01\")\n",
65+
"usgs_samples_data, md = samples.get_usgs_samples(monitoringLocationIdentifier=f\"USGS-{siteNumber}\", usgsPCode=parameterCd, activityStartDateLower=\"1980-01-01\", activityStartDateUpper=\"2010-01-01\")\n",
6566
"\n",
6667
"pCode, md = nwis.get_pmcodes(parameterCd=parameterCd)"
6768
]
@@ -199,18 +200,15 @@
199200
"parameterCd <- c(\"00618\",\"71851\")\n",
200201
"startDate <- \"1985-10-01\"\n",
201202
"endDate <- \"2012-09-30\"\n",
202-
"dfLong <- readNWISqw(siteNumber, parameterCd, \n",
203-
" startDate, endDate)\n",
204-
"# Or the wide return:\n",
205-
"dfWide <- readNWISqw(siteNumber, parameterCd,\n",
206-
" startDate, endDate, reshape=TRUE)\n",
203+
"dfLong <- read_USGS_samples(monitoringLocationIdentifier=sprintf(\"USGS-%s\", siteNumber), usgsPCode=parameterCd, \n",
204+
" activityStartDateLower=startDate, activityStartDateUpper=endDate)\n",
207205
"'''\n",
208206
"siteNumber = \"01491000\"\n",
209207
"parameterCd = [\"00618\",\"71851\"]\n",
210208
"startDate = \"1985-10-01\"\n",
211209
"endDate = \"2012-09-30\"\n",
212-
"dfLong, md = nwis.get_qwdata(sites=siteNumber, parameterCd=parameterCd,\n",
213-
" start=startDate, end=endDate)"
210+
"dfLong, md = samples.get_usgs_samples(monitoringLocationIdentifier=f\"USGS-{siteNumber}\", usgsPCode=parameterCd,\n",
211+
" activityStartDateLower=startDate, activityStartDateUpper=endDate)"
214212
]
215213
},
216214
{

0 commit comments

Comments
 (0)