Skip to content

Commit 3ea572d

Browse files
sampsonjsampsonj
authored
Data Query Endpoint v0.9.0 (#71)
1 parent f0f4db5 commit 3ea572d

File tree

139 files changed

+4321
-3296
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

139 files changed

+4321
-3296
lines changed

‎.dockerignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,5 @@ packages/**/*/build
1212
**/*.env.development.local
1313
**/*.env.test.local
1414
**/*.env.production.local
15+
16+
.venv

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,5 @@
3232
**/*yarn-error.log*
3333

3434
**/*.eslintcache
35+
36+
.venv

‎Dockerfile‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#-------------------------------------------------------------------------------
22
# Builder container for reproducible build environment
33

4-
FROM python:3.11-alpine AS builder
4+
FROM python:3.12-alpine AS builder
55

66
WORKDIR /usr/src/app
77
COPY ./docs /usr/src/app/build-temp/api-docs/docs
@@ -23,7 +23,7 @@ RUN cd /usr/src/app/build-temp/api-docs \
2323
#-------------------------------------------------------------------------------
2424
# Builder container for reproducible build environment
2525

26-
FROM golang:1.21-alpine AS go-builder
26+
FROM golang:1.23-alpine AS go-builder
2727

2828
WORKDIR /go/src/app
2929

@@ -39,7 +39,7 @@ RUN go mod verify \
3939
#-------------------------------------------------------------------------------
4040
# Build production container with only necessary artifacts
4141

42-
FROM alpine:3.18
42+
FROM alpine:3.20
4343

4444
# Traefik auto discover labels
4545
LABEL \

‎docs/content/endpoints/data-query.md‎

Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
# Data Query Endpoint
2+
3+
The `/data/query` endpoint provides the ability to query for available data
4+
files by product, one or more sites, a specified date range, and optionally
5+
by release, package type, and whether or not the query results should include
6+
provisional data. By default, provisional data are excluded from results and
7+
the latest available release data are included.
8+
9+
<a name="data-releases"></a>
10+
## **Data Releases**
11+
12+
The `/data/query` endpoints allow filtering the response based on data available within
13+
a particular release by adding a `release` query parameter to GET requests or by
14+
specifying the `release` property within the request body object to POST requests. For example:
15+
16+
### GET Request
17+
```
18+
/data/query?productCode={productCode}&siteCode={siteCode}&startDateMonth={startDateMonth}&endDateMonth={endDateMonth}&release={releaseTag}
19+
```
20+
21+
### POST Request
22+
```
23+
/data/query
24+
```
25+
``` JSON
26+
{
27+
"productCode": "DP1.00001.001",
28+
"siteCodes": ["ABBY"],
29+
"startDateMonth": "2023-01",
30+
"endDateMonth": "2023-12",
31+
"release": "RELEASE-2024"
32+
}
33+
```
34+
35+
The learn more about data releases, see: <a href="#" onclick="Router.jumpToReleasePage()">releases</a>
36+
37+
38+
<a name="paths"></a>
39+
## **Paths**
40+
41+
<a name="get_data_query-availability"></a>
42+
### GET `/data/query`
43+
44+
#### **Description**
45+
Get files available for a given product, one or more sites, date range, release, package combination
46+
47+
48+
#### **Parameters**
49+
50+
|Type|Name|Description|Schema|
51+
|---|---|---|---|
52+
|**Query**|**productCode** <br>*required*|Product code to query for files|string|
53+
|**Query**|**siteCode** <br>*required*|One or more four letter site codes to query for files|string|
54+
|**Query**|**startDateMonth** <br>*required*|YYYY-MM start date month to query for files|string|
55+
|**Query**|**endDateMonth** <br>*required*|YYYY-MM end date month to query for files|string|
56+
|**Query**|**release** <br>*optional*|The name of the release to query for|string|
57+
|**Query**|**package** <br>*optional*|Package type to query, basic or expanded|enum (basic, expanded)|
58+
|**Query**|**includeProvisional** <br>*optional*|Optionally include provisional data in the query|boolean|
59+
60+
61+
#### **Responses**
62+
63+
|HTTP Code|Description|Schema|
64+
|---|---|---|
65+
|**200**|Available files|[Response 200](#get_data_query_availability_response_200)|
66+
|**400**|One or more invalid paramaters specified|[Response 400](#get_data_query_availability_response_400)|
67+
|**default**|General error|[error](#error)|
68+
69+
<h5 id="get_data_query_availability_response_200">Response 200</h5>
70+
71+
|Name|Schema|
72+
|---|---|
73+
|**data**|[data](#data-query-response-data)|
74+
75+
76+
<h5 id="get_data_query_availability_response_400">Response 400</h5>
77+
78+
|Name|Schema|
79+
|---|---|
80+
|**errors** <br>*optional*|[[error](#error)]|
81+
82+
83+
#### **Produces**
84+
85+
* `application/json`
86+
87+
88+
#### **Tags**
89+
90+
* Data Query
91+
92+
93+
<a name="post_data_query-availability"></a>
94+
### POST `/data/query`
95+
96+
#### **Description**
97+
Get files available for a given product, one or more sites, date range, release, package combination
98+
99+
100+
#### **Request Body**
101+
102+
|Name|Description|Schema|
103+
|---|---|---|
104+
|**productCode** <br>*required*|Product code to query for files|string|
105+
|**siteCodes** <br>*required*|One or more four letter site codes to query for files|[string]|
106+
|**startDateMonth** <br>*required*|YYYY-MM start date month to query for files|string|
107+
|**endDateMonth** <br>*required*|YYYY-MM end date month to query for files|string|
108+
|**release** <br>*optional*|The name of the release to query for|string|
109+
|**package** <br>*optional*|Package type to query, basic or expanded|enum (basic, expanded)|
110+
|**includeProvisional** <br>*optional*|Optionally include provisional data in the query|boolean|
111+
112+
Example JSON Request Body
113+
114+
``` JSON
115+
{
116+
"productCode": "DP1.00001.001",
117+
"siteCodes": ["ABBY"],
118+
"startDateMonth": "2023-01",
119+
"endDateMonth": "2023-12",
120+
"release": "RELEASE-2024",
121+
"package": "basic",
122+
"includeProvisional": false
123+
}
124+
```
125+
126+
#### **Responses**
127+
128+
|HTTP Code|Description|Schema|
129+
|---|---|---|
130+
|**200**|Available files|[Response 200](#get_data_query_availability_response_200)|
131+
|**400**|One or more invalid paramaters specified|[Response 400](#get_data_query_availability_response_400)|
132+
|**default**|General error|[error](#error)|
133+
134+
<h5 id="get_data_query_availability_response_200">Response 200</h5>
135+
136+
|Name|Schema|
137+
|---|---|
138+
|**data**|[data](#data-query-response-data)|
139+
140+
141+
<h5 id="get_data_query_availability_response_400">Response 400</h5>
142+
143+
|Name|Schema|
144+
|---|---|
145+
|**errors** <br>*optional*|[[error](#error)]|
146+
147+
148+
#### **Produces**
149+
150+
* `application/json`
151+
152+
153+
#### **Tags**
154+
155+
* Data Query
156+
157+
158+
<a name="definitions"></a>
159+
## **Definitions**
160+
161+
<a name="data-query-response-data"></a>
162+
### **data**
163+
164+
|Name|Description|Schema|
165+
|---|---|---|
166+
|**productCode**|Revisioned, shortened code for the data product (DP1.00001.001, DP1.10072.001, etc)|string|
167+
|**siteCodes**|List of four character site codes available|[string]|
168+
|**startDate**|The start date of the query interval|string (date-time)|
169+
|**endDate**|The end date of the query interval|string (date-time)|
170+
|**packageType**|The type of package, basic or expanded|string|
171+
|**releases**|The list of available releases for the query|[[releases](#data-response-releases)]|
172+
173+
174+
<a name="data-response-releases"></a>
175+
### **releases**
176+
Describes a release package
177+
178+
|Name|Description|Schema|
179+
|---|---|---|
180+
|**release**|The name of the release|string|
181+
|**generationDate**|The generation date of the release|string (date-time)|
182+
|**packages**|The list of available packages within the release for the query|[[packages](#data-packages)]|
183+
184+
185+
<a name="data-packages"></a>
186+
### **packages**
187+
Describes a data package
188+
189+
|Name|Description|Schema|
190+
|---|---|---|
191+
|**domainCode**|Three character domain abbreviation (D01, D02, etc) for the domain this package is in|string|
192+
|**siteCode**|Four character code for the site|string|
193+
|**month**|The available month, formatted as YYYY-MM|string|
194+
|**packageType**|The type of package, basic or expanded|string|
195+
|**generationDate**|The generation date of the package|string (date-time)|
196+
|**files**|The list of available files within the package for the query|[[file](#file)]|
197+
198+
199+
<a name="file"></a>
200+
### **file**
201+
202+
<div class="md-typeset__scrollwrap">
203+
<div class="md-typeset__table">
204+
<table>
205+
<thead>
206+
<tr>
207+
<th>Name</th>
208+
<th>Description</th>
209+
<th>Schema</th>
210+
</tr>
211+
</thead>
212+
<tbody>
213+
<tr>
214+
<td><strong>name</strong></td>
215+
<td>Filename</td>
216+
<td>string</td>
217+
</tr>
218+
<tr>
219+
<td><strong>size</strong></td>
220+
<td>File size in bytes</td>
221+
<td>number (int)</td>
222+
</tr>
223+
<tr>
224+
<td><strong>md5</strong></td>
225+
<td>MD5 value in hex</td>
226+
<td>string</td>
227+
</tr>
228+
<tr>
229+
<td><strong>crc32c</strong></td>
230+
<td>CRC32C value in hex</td>
231+
<td>string</td>
232+
</tr>
233+
<tr>
234+
<td><strong>url</strong></td>
235+
<td>Download URL</td>
236+
<td>string</td>
237+
</tr>
238+
</tbody>
239+
</table>
240+
</div>
241+
</div>
242+
243+
244+
<a name="error"></a>
245+
### **error**
246+
247+
|Name|Schema|
248+
|---|---|
249+
|**detail** <br>*optional*|string|
250+
|**status** <br>*optional*|number (int)|
251+
252+
253+
<br />

‎docs/content/endpoints/data.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,18 +47,18 @@ Get files available for a given product/site/month combination
4747

4848
|HTTP Code|Description|Schema|
4949
|---|---|---|
50-
|**200**|Available files|[Response 200](#get_data_availability_resposne_200)|
51-
|**400**|Product, site not found or invalid date range specified|[Response 400](#get_data_availability_resposne_400)|
50+
|**200**|Available files|[Response 200](#get_data_availability_response_200)|
51+
|**400**|Product, site not found or invalid date range specified|[Response 400](#get_data_availability_response_400)|
5252
|**default**|General error|[error](#error)|
5353

54-
<h5 id="get_data_availability_resposne_200">Response 200</h5>
54+
<h5 id="get_data_availability_response_200">Response 200</h5>
5555

5656
|Name|Schema|
5757
|---|---|
5858
|**data**|[data](#data-productcode-sitecode-year-month-get-data)|
5959

6060

61-
<h5 id="get_data_availability_resposne_400">Response 400</h5>
61+
<h5 id="get_data_availability_response_400">Response 400</h5>
6262

6363
|Name|Schema|
6464
|---|---|
@@ -252,7 +252,7 @@ Get a data package for a given product/site/month and package combination
252252
|---|---|---|
253253
|**name**|The name of the external data object|string|
254254
|**type**|The type of external data link provided.|string|
255-
|**url**|URL to external data.|number (int)|
255+
|**url**|URL to external data.|string|
256256

257257
<a name="error"></a>
258258
### **error**

‎docs/content/endpoints/prototype-datasets.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,7 @@ Type definition for prototype dataset data
212212
|---|---|---|
213213
|**datasetUuid** |The dataset UUID|string|
214214
|**datasetProjectTitle** |The title of the project|string|
215-
|**files** |List of data files for the dataset|[prototypeDatasetDataFile](#prototypedatasetdatafile) array|
215+
|**files** |List of data files for the dataset|[prototypeDatasetDataFile](#prototypedatafile) array|
216216
|**dataLocations** |List of data locations for the dataset|[prototypeDataLocation](#prototypedatalocation) array|
217217

218218
<a name="prototypedatafile"></a>

‎docs/content/endpoints/releases.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ including availability by product, site, and month (also see the
55
[data product catalog](https://data.neonscience.org/data-products/explore)
66
to filter products and availability by release). The `/releases/{releaseTag}`
77
endpoint also supplies path based access for a specific release for the
8-
`/products`, `/sites`, and `/data` endpoints.
8+
`/products`, `/sites`, `/data`, and `/data/query` endpoints.
99

1010
A data release is a set of data files that is static (unchanging),
1111
always available to end users, and citable.

‎docs/content/explorer/build/asset-manifest.json‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
{
22
"files": {
3-
"main.css": "./static/css/main.bc25d16a.css",
4-
"main.js": "./static/js/main.9538d8e4.js",
3+
"main.css": "./static/css/main.ab959d79.css",
4+
"main.js": "./static/js/main.240f2395.js",
55
"index.html": "./index.html",
6-
"main.bc25d16a.css.map": "./static/css/main.bc25d16a.css.map",
7-
"main.9538d8e4.js.map": "./static/js/main.9538d8e4.js.map"
6+
"main.ab959d79.css.map": "./static/css/main.ab959d79.css.map",
7+
"main.240f2395.js.map": "./static/js/main.240f2395.js.map"
88
},
99
"entrypoints": [
10-
"static/css/main.bc25d16a.css",
11-
"static/js/main.9538d8e4.js"
10+
"static/css/main.ab959d79.css",
11+
"static/js/main.240f2395.js"
1212
]
1313
}

‎docs/content/explorer/build/index.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="NEON Data API REST Explorer"/><link rel="manifest" href="./manifest.json"/><link href="https://fonts.gstatic.com" rel="preconnect" crossorigin=""><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Inter:300,400,400i,700%7CRoboto+Mono&amp;display=fallback"><title>NEON Data API REST Explorer</title><script defer="defer" src="./static/js/main.9538d8e4.js"></script><link href="./static/css/main.bc25d16a.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
1+
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="NEON Data API REST Explorer"/><link rel="manifest" href="./manifest.json"/><link href="https://fonts.gstatic.com" rel="preconnect" crossorigin=""><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Inter:300,400,400i,700%7CRoboto+Mono&amp;display=fallback"><title>NEON Data API REST Explorer</title><script defer="defer" src="./static/js/main.240f2395.js"></script><link href="./static/css/main.ab959d79.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

docs/content/explorer/build/static/css/main.bc25d16a.css renamed to docs/content/explorer/build/static/css/main.ab959d79.css

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)