Skip to content

Commit 8380375

Browse files
garycentricgarycentric
authored
Merge pull request #110994 from sunasing/subranch
Added How-to-guide to get weather data
2 parents e393d5e + 58f11d5 commit 8380375

File tree

2 files changed

+238
-0
lines changed

2 files changed

+238
-0
lines changed

‎articles/industry/agriculture/get-weather-data-from-weather-partner.md

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
---
2+
title: Get weather data from weather partners
3+
description: This article describes how to get weather data from partners.
4+
author: sunasing
5+
ms.topic: article
6+
ms.date: 03/31/2020
7+
ms.author: sunasing
8+
---
9+
10+
# Get weather data from weather partners
11+
12+
Azure FarmBeats helps you to bring weather data from your weather data provider(s) using a docker-based Connector Framework. Using this framework, weather data providers implement a docker that can be integrated with FarmBeats. Currently the following weather data providers are supported:
13+
14+
[NOAA data from Azure Open Datasets](https://azure.microsoft.com/services/open-datasets/)
15+
16+
The weather data can be used to generate actionable insights and build AI/ML models on FarmBeats.
17+
18+
## Before you start
19+
20+
To get weather data, ensure that you have installed FarmBeats. **Weather integration is supported in version 1.2.11 or higher**. To install Azure FarmBeats, see [Install FarmBeats](https://aka.ms/farmbeatsinstalldocumentation).
21+
22+
## Enable weather integration with FarmBeats
23+
24+
To start getting weather data on your FarmBeats Data hub, follow the steps below:
25+
26+
1. Go to your FarmBeats Data hub swagger (https://yourdatahub.azurewebsites.net/swagger)
27+
28+
2. Navigate to /Partner API and make a POST request with the following input payload:
29+
30+
```json
31+
{ 
32+
33+
"dockerDetails": { 
34+
"credentials": { 
35+
"username": "<credentials to access private docker - not required for public docker>", 
36+
"password": "<credentials to access private docker – not required for public docker>" 
37+
}, 
38+
"imageName" : "<docker image name. Default is azurefarmbeats/fambeats-noaa>",
39+
"imageTag" : "<docker image tag, default:latest>",
40+
"azureBatchVMDetails": { 
41+
"batchVMSKU" : "<VM SKU. Default is standard_d2_v2>", 
42+
"dedicatedComputerNodes" : 1,
43+
"nodeAgentSKUID": "<Node SKU. Default is batch.node.ubuntu 18.04>" 
44+
}
45+
},
46+
"partnerCredentials": { 
47+
"key1": "value1", 
48+
"key2": "value2" 
49+
}, 
50+
"partnerType": "Weather", 
51+
"name": "<Name of the partner>", 
52+
"description": "<Description>",
53+
"properties": {  } 
54+
} 
55+
```
56+
57+
For example, to get weather data from NOAA by Azure Open Datasets, use the payload below. You can modify the name and description as per your preference.
58+
59+
```json
60+
{
61+
62+
"dockerDetails": {
63+
"imageName": "azurefarmbeats/farmbeats-noaa",
64+
"imageTag": "latest",
65+
"azureBatchVMDetails": {
66+
"batchVMSKU": "standard_d2_v2",
67+
"dedicatedComputerNodes": 1,
68+
"nodeAgentSKUID": "batch.node.ubuntu 18.04"
69+
}
70+
},
71+
"partnerType": "Weather",
72+
"name": "ods-noaa",
73+
"description": "NOAA data from Azure Open Datasets registered as a Weather Partner"
74+
}
75+
```
76+
77+
> [!NOTE]
78+
> For more information about the Partner object, see [Appendix](get-weather-data-from-weather-partner.md#appendix)
79+
80+
The preceding step will provision the resources to enable docker to run in the customer's FarmBeats environment.
81+
82+
It takes about 10-15 minutes to provision the above resources.
83+
84+
3. Check the status of the /Partner object that you created in step 2. To do this, make a GET request on /Partner API and check for the **status** of the partner object. Once FarmBeats provisions the partner successfully, the status is set to **Active**.
85+
86+
4. Navigate to /JobType API and make a GET request on the same. Check for the weather jobs that are created as part of the Partner addition process in step 2. The **pipelineName** field in the weather jobs will be of the following format:
87+
"partner-name_partner-type_job-name".
88+
89+
5. Now your FarmBeats instance has an active weather data partner and you can run jobs to request weather data for a particular location (latitude/longitude) and a date range. The JobType(s) will have details on what parameters are required to run weather jobs.
90+
91+
For example, for NOAA data from Azure Open Datasets, following JobType(s) will be created:
92+
93+
- get_weather_data (Get ISD/historical weather data)
94+
- get_weather_forecast_data (Get GFS/forecast weather data)
95+
96+
6. Make a note of the **ID** and the parameters of the JobType(s).
97+
98+
7. Navigate to /Jobs API and make a POST request on /Jobs with the following input payload:
99+
100+
```json
101+
{
102+
"typeId": "<id of the JobType>",
103+
"arguments": {
104+
"additionalProp1": {},
105+
"additionalProp2": {},
106+
"additionalProp3": {}
107+
},
108+
"name": "<name of the job>",
109+
"description": "<description>",
110+
"properties": {}
111+
}
112+
```
113+
114+
For example, to run **get_weather_data**, use the following payload:
115+
116+
```json
117+
{
118+
119+
"typeId": "<id of the JobType>",
120+
"arguments": {
121+
"latitude": 47.620422,
122+
"longitude": -122.349358,
123+
"start_date": "yyyy-mm-dd",
124+
"end_date": "yyyy-mm-dd"
125+
},
126+
"name": "<name of the job>",
127+
"description": "<description>",
128+
"properties": {}
129+
}
130+
```
131+
132+
8. The preceding step will run the weather jobs as defined in the partner docker and ingest weather data into FarmBeats. You can check the status of the job by making a GET request on /Jobs and look for **currentState** in the response. Once complete, the currentState is set to **Succeeded**.
133+
134+
## Query ingested weather data
135+
136+
After the weather jobs are complete, you can query ingested weather data to build models or actionable insights. There are two ways to access and query weather data from FarmBeats:
137+
138+
- API and
139+
- Time Series Insights (TSI).
140+
141+
### Query using REST API
142+
143+
To query weather data using FarmBeats REST API, follow the steps below:
144+
145+
1. In your FarmBeats Data hub swagger (https://yourdatahub.azurewebsites.net/swagger), navigate to /WeatherDataLocation API and make a GET request. The response will have /WeatherDataLocation object(s) created for the location (latitude/longitude) that was specified as part of the job run. Make a note of the **ID** and the **weatherDataModelId** of the object(s).
146+
147+
2. Make a GET/{id} on /WeatherDataModel API for the **weatherDataModelId** as noted in step 1. The "Weather Data Model" has all the metadata and details about the ingested weather data. For example, **Weather Measure** within the **Weather Data Model** object has details about what weather information is supported and in what types and units. For example,
148+
149+
```json
150+
{
151+
"name": "Temperature <name of the weather measure - this is what we will receive as part of the queried weather data>",
152+
"dataType": "Double <Data Type - eg. Double, Enum>",
153+
"type": "AmbientTemperature <Type of measure eg. AmbientTemperature, Wind speed etc.>",
154+
"unit": "Celsius <Unit of measure eg. Celsius, Percentage etc.>",
155+
"aggregationType": "None <either of None, Average, Maximum, Minimum, StandardDeviation, Sum, Total>",
156+
"description": "<Description of the measure>"
157+
}
158+
```
159+
160+
Make a note of the response from the GET/{id} call for the Weather Data Model.
161+
162+
3. Navigate to **Telemetry** API and make a POST request with the following input payload:
163+
164+
```json
165+
{
166+
"weatherDataLocationId": "<id from step 1 above>",
167+
"searchSpan": {
168+
"from": "2020-XX-XXT07:30:00Z",
169+
"to": "2020-XX-XXT07:45:00Z"
170+
}
171+
}
172+
```
173+
174+
4. The response containing weather data that is available for the specified time range will look like this:
175+
176+
```json
177+
{
178+
"timestamps": [
179+
"2020-XX-XXT07:30:00Z",
180+
"2020-XX-XXT07:45:00Z"
181+
],
182+
"properties": [
183+
{
184+
"values": [
185+
"<id of the weatherDataLocation>",
186+
"<id of the weatherDataLocation>"
187+
],
188+
"name": "Id",
189+
"type": "String"
190+
},
191+
{
192+
"values": [
193+
29.1,
194+
30.2
195+
],
196+
"name": "Temperature <name of the WeatherMeasure as defined in the WeatherDataModel object>",
197+
"type": "Double <Data Type of the value - eg. Double>"
198+
}
199+
]
200+
}
201+
```
202+
203+
In the preceding example, the response has data for two timestamps along with the measure name ("Temperature") and values of the reported weather data in the two timestamps. You will need to refer to the associated Weather Data Model (as described in step 2 above) to interpret the type and unit of the reported values.
204+
205+
### Query using Azure Time Series Insights (TSI)
206+
207+
FarmBeats uses [Azure Time Series Insights (TSI)](https://azure.microsoft.com/services/time-series-insights/) to ingest, store, query, and visualize data at IoT scale--data that's highly contextualized and optimized for time series.
208+
209+
Weather data is received on an EventHub and then pushed to a TSI environment within FarmBeats resource group. Data can then be directly queried from the TSI. For more information, see [TSI documentation](https://docs.microsoft.com/azure/time-series-insights/time-series-insights-explorer).
210+
211+
Follow the steps to visualize data on TSI:
212+
213+
1. Go to **Azure portal** > **FarmBeats DataHub resource group** > select **Time Series Insights** environment (tsi-xxxx) > **Data Access Policies**. Add user with Reader or Contributor access.
214+
215+
2. Go to the **Overview** page of **Time Series Insights** environment (tsi-xxxx) and select the **Time Series Insights Explorer URL**. You can now visualize the ingested weather data.
216+
217+
Apart from storing, querying and visualization of weather data, TSI also enables integration to a Power BI dashboard. For more information, see [Visualize data from Time Series Insights in Power BI](https://docs.microsoft.com/azure/time-series-insights/how-to-connect-power-bi).
218+
219+
## Appendix
220+
221+
| Partner | Details |
222+
| ------- | ------- |
223+
| DockerDetails - imageName | Docker image name. For example, docker.io/azurefarmbeats/farmbeats-noaa (image in hub.docker.com) OR myazureacr.azurecr.io/mydockerimage (image in Azure Container Registry) and so on. If no registry is provided, default is hub.docker.com |
224+
| DockerDetails - imageTag | Tag name of the docker image. Default is "latest" |
225+
| DockerDetails - credentials | Credentials to access the private docker. This will be provided by the partner to the customer |
226+
| DockerDetails - azureBatchVMDetails - batchVMSKU | Azure Batch VM SKU. See [here](https://docs.microsoft.com/azure/virtual-machines/linux/sizes?toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json) for all Linux virtual machines available. Valid values are: "Small', 'ExtraLarge', 'Large', 'A8', 'A9', 'Medium', 'A5', 'A6', 'A7', 'STANDARD_D1', 'STANDARD_D2', 'STANDARD_D3', 'STANDARD_D4', 'STANDARD_D11', 'STANDARD_D12', 'STANDARD_D13', 'STANDARD_D14', 'A10', 'A11', 'STANDARD_D1_V2', 'STANDARD_D2_V2', 'STANDARD_D3_V2', 'STANDARD_D4_V2', 'STANDARD_D11_V2', 'STANDARD_D12_V2', 'STANDARD_D13_V2', 'STANDARD_D14_V2', 'STANDARD_G1', 'STANDARD_G2', 'STANDARD_G3', 'STANDARD_G4', 'STANDARD_G5', 'STANDARD_D5_V2', 'BASIC_A1', 'BASIC_A2', 'BASIC_A3', 'BASIC_A4', 'STANDARD_A1', 'STANDARD_A2', 'STANDARD_A3', 'STANDARD_A4', 'STANDARD_A5', 'STANDARD_A6', 'STANDARD_A7', 'STANDARD_A8', 'STANDARD_A9', 'STANDARD_A10', 'STANDARD_A11', 'STANDARD_D15_V2', 'STANDARD_F1', 'STANDARD_F2', 'STANDARD_F4', 'STANDARD_F8', 'STANDARD_F16', 'STANDARD_NV6', 'STANDARD_NV12', 'STANDARD_NV24', 'STANDARD_NC6', 'STANDARD_NC12', 'STANDARD_NC24', 'STANDARD_NC24r', 'STANDARD_H8', 'STANDARD_H8m', 'STANDARD_H16', 'STANDARD_H16m', 'STANDARD_H16mr', 'STANDARD_H16r', 'STANDARD_A1_V2', 'STANDARD_A2_V2', 'STANDARD_A4_V2', 'STANDARD_A8_V2', 'STANDARD_A2m_V2', 'STANDARD_A4m_V2', 'STANDARD_A8m_V2', 'STANDARD_M64ms', 'STANDARD_M128s', 'STANDARD_D2_V3'. **Default is "standard_d2_v2"** |
227+
| DockerDetails - azureBatchVMDetails - dedicatedComputerNodes | No. of dedicated computer nodes for batch pool. Default value is 1. |
228+
| DockerDetails - azureBatchVMDetails - nodeAgentSKUID | Azure Batch Node Agent SKU ID. Currently only "batch.node.ubuntu 18.04" batch node agent is supported. |
229+
| DockerDetails - partnerCredentials | credentials for calling partner API in docker. The partner needs to give this information to their customers based on the auth mechanism that is supported eg. Username/password or API Keys. |
230+
| partnerType | "Weather" (Other partner types in FarmBeats are "Sensor" and "Imagery") |
231+
| name | Desired name of the partner in FarmBeats system |
232+
| description | Description |
233+
234+
## Next steps
235+
236+
You now have queried sensor data from your Azure FarmBeats instance. Now, learn how to [generate maps](generate-maps-in-azure-farmbeats.md#generate-maps) for your farms.

‎articles/industry/agriculture/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@
2929
href: manage-farms-in-azure-farmbeats.md
3030
- name: Get sensor data
3131
href: get-sensor-data-from-sensor-partner.md
32+
- name: Get weather data
33+
href: get-weather-data-from-weather-partner.md
3234
- name: Generate maps
3335
href: generate-maps-in-azure-farmbeats.md
3436
- name: Configure rules and manage alerts

0 commit comments

Comments
 (0)