You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -20,15 +20,18 @@ Using AOGS, we capture the Aqua broadcast when the satellite is within line of s
20
20
In this tutorial, we will follow these steps to collect and process Aqua data:
21
21
22
22
> [!div class="checklist"]
23
-
> *[Schedule a contact and collect Aqua direct broadcast data using AOGS](#step-1-schedule-a-contact-and-collect-aqua-direct-broadcast-data-using-aogs).
24
-
> *[Process Aqua direct broadcast data using RT-STPS](#step-2-process-aqua-direct-broadcast-data-using-rt-stps).
25
-
> *[Create higher level products using IPOPP](#step-3-create-higher-level-products-using-ipopp).
23
+
> *[Use AOGS to schedule and contact and collect Aqua data](#step-1-use-aogs-to-schedule-a-contact-and-collect-aqua-data).
24
+
> *[Install NASA DRL tools](#step-2-install-nasa-drl-tools).
25
+
> *[Create Level-0 product using RT-STPS](#step-3-create-level-0-product-using-rt-stps).
26
+
> *[Create higher level products using IPOPP](#step-4-create-higher-level-products-using-ipopp).
26
27
27
28
Optional setup steps for capturing the ground station telemetry are included the guide on [receiving real-time telemetry from the ground stations](receive-real-time-telemetry.md).
28
29
29
-
## Step 1: Schedule a contact and collect Aqua direct broadcast data using AOGS
30
+
## Step 1: Use AOGS to schedule a contact and collect Aqua data
30
31
31
-
Follow the steps listed in [Tutorial: Downlink data from NASA's AQUA public satellite](downlink-aqua.md) to schedule a contact with Aqua using AOGS and collect the direct broadcast data on an Azure VM for further processing.
32
+
Execute steps listed in [Tutorial: Downlink data from NASA's AQUA public satellite](downlink-aqua.md)
33
+
34
+
The above tutorial provides a walkthrough for scheduling a contact with Aqua and collecting the direct broadcast data on an Azure VM.
32
35
33
36
> [!NOTE]
34
37
> In the section [Prepare a virtual machine (VM) to receive the downlinked AQUA data](downlink-aqua.md#prepare-your-virtual-machine-vm-and-network-to-receive-aqua-data), use the following values:
@@ -38,80 +41,15 @@ Follow the steps listed in [Tutorial: Downlink data from NASA's AQUA public sate
38
41
> -**Size:** Standard_D8s_v5 or higher
39
42
> -**IP Address:** Ensure that the VM has at least one standard public IP address
40
43
41
-
At the end of this step, you should have the raw direct broadcast saved as ```.bin``` files under the ```~/aquadata``` folder on the receiver-vm.
42
-
43
-
## Step 2: Process Aqua direct broadcast data using RT-STPS
44
-
45
-
The [Real-time Software Telemetry Processing System](https://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=69)(RT-STPS) is a NASA-provided software for processing Aqua direct broadcast data. The steps below cover installation of RT-STPS Verson 6.0 on the receiver-vm, and production of Level-0 Production Data Set(PDS) files from the data collected in the previous step.
46
-
47
-
Register with the [NASA DRL](https://directreadout.sci.gsfc.nasa.gov/) to download the RT-STPS installation package.
48
-
49
-
Transfer the installation binaries to the receiver-vm:
Alternatively, you can upload your installation binaries to a container in Azure Storage and download them to the receiver-vm using [AzCopy](../storage/common/storage-use-azcopy-v10.md)
This command produces Level-0 Production Data Set (```.pds```) files under the ```~/software/rt-stps/data``` directory.
101
-
102
-
## Step 3: Create higher level products using IPOPP
103
-
104
-
[International Planetary Observation Processing Package (IPOPP)](https://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=68) is another NASA-provided software to process Aqua Level-0 data into higher level products.
105
-
In the steps below, you'll process the Level-0 data generated in the previous step using IPOPP.
44
+
At the end of this step, you should have the raw direct broadcast data saved as ```.bin``` files under the ```~/aquadata``` folder on the ```receiver-vm```.
106
45
46
+
## Step 2: Install NASA DRL tools
107
47
> [!NOTE]
108
-
> Due to potential resource contention, DRL recommends installing RT-STPS and IPOPP on separate machines. But for this tutorial, we install both on the our receiver-vm because we don't run them at the same time. For production workloads, please follow sizing and isolation recommendations in the user guides available on the DRL website.
48
+
> Due to potential resource contention, DRL recommends installing RT-STPS and IPOPP on separate machines. But for this tutorial, we install both tools on the ```receiver-vm``` because we don't run them at the same time. For production workloads, please follow sizing and isolation recommendations in the user guides available on the DRL website.
109
49
110
50
### Increase OS disk size on the receiver-vm
111
51
112
-
IPOPP installation and subsequent generation of products requires more disk space than what is available on an Azure VM by default.
113
-
114
-
Follow the steps below to increase the size of the OS disk on the ```receiver-vm``` to 1TB.
52
+
The default disk space allocated to the OS disk of an Azure VM is not sufficient for installing NASA DRL tools. Follow the steps below to increase the size of the OS disk on the ```receiver-vm``` to 1TB.
115
53
116
54
### [Portal](#tab/portal2)
117
55
@@ -143,131 +81,116 @@ sda 0:0:0:0 1T
143
81
```
144
82
145
83
### Install Desktop and VNC Server
146
-
147
-
IPOPP installation requires using a browser to sign on to the DRL website to download the installation script. This script must be run from the same host that it was downloaded to. The subsequent IPOPP configuration also requires a GUI. Therefore, we install a full desktop and a vnc server to enable running GUI applications on the receiver-vm.
148
-
84
+
Using NASA DRL tools requires support for running GUI applications. To enable this, install desktop tools and vncserver on the `receiver-vm`:
149
85
```bash
150
86
sudo yum install tigervnc-server
151
87
sudo yum groups install "GNOME Desktop"
152
88
```
153
-
154
89
Start VNC server:
155
-
156
90
```bash
157
91
vncsever
158
92
```
159
-
Enter a password when prompted.
160
-
161
-
Port forward the vncserver port (5901) over ssh:
93
+
Enter a password when prompted.
162
94
95
+
### Remotely access the VM Desktop
96
+
Port forward the vncserver port (5901) over SSH to your local machine:
163
97
```bash
164
98
ssh -L 5901:localhost:5901 azureuser@receiver-vm
165
99
```
100
+
1. On your local machine, download and install [TightVNC Viewer](https://www.tightvnc.com/download.php).
101
+
1. Start the TightVNC Viewer and connect to ```localhost:5901```.
102
+
1. Enter the vncserver password you entered in the previous step.
103
+
1. You should see the GNOME Desktop that is running on the VM in the VNC viewer window.
166
104
167
-
Download the [TightVNC](https://www.tightvnc.com/download.php) viewer and connect to ```localhost:5901``` and enter the vncserver password entered in the previous step. You should see the GNOME desktop running on the VM.
105
+
### Download RT-STPS and IPOPP installation files
106
+
From the GNOME Desktop, go to **Applications** > **Internet** > **Firefox** to start a browser.
168
107
169
-
Start a new terminal, and start the Firefox browser
108
+
Log on to the [NASA DRL](https://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=325&type=software) website and download the **RT-STPS** installation files and the **IPOPP downloader script** under software downloads. The downloaded files will land under ~/Downloads.
170
109
110
+
Alternatively, you can download the installation files on your local machine first and then upload to a container in Azure Storage. Then use [AzCopy](../storage/common/storage-use-azcopy-v10.md) to download to your ```receiver-vm```.
111
+
112
+
### Install RT-STPS
113
+
```bash
114
+
tar -xvzf ~/Downloads/RT-STPS_7.0.tar.gz --directory ~/
115
+
tar -xvzf ~/Downloads/RT-STPS_7.0_testdata.tar.gz --directory ~/
116
+
cd~/rt-stps
117
+
./install.sh
118
+
```
119
+
Validate your RT-STPS install by processing the test data supplied with the installation:
[Log on the DRL website](https://directreadout.sci.gsfc.nasa.gov/loginDRL.cfm?cid=320&type=software) and download the downloader script.
176
-
124
+
Verify that output files exist in the data folder:
177
125
```bash
178
-
cd~/Downloads
179
-
./downloader_DRL-IPOPP_4.1.sh
126
+
ls -la ~/data/
180
127
```
181
-
182
-
This script will download \~35G and will take 1 hour or more.
183
-
184
-
Alternatively, you can upload your installation binaries to a container in Azure Storage and download them to the receiver-vm using [AzCopy](../storage/common/storage-use-azcopy-v10.md)
128
+
This completes the RT-STPS installation.
185
129
186
130
### Install IPOPP
187
-
131
+
Run the IPOPP downloader script to download the IPOPP installation files.
188
132
```bash
189
133
cd~/Downloads
190
-
tar -xvzf DRL-IPOPP_4.1.tar.gz --directory ~/
134
+
./downloader_DRL-IPOPP_4.1.sh
135
+
tar -xvzf ~/Downloads/DRL-IPOPP_4.1.tar.gz --directory ~/
191
136
cd~/IPOPP
192
137
./install_ipopp.sh
193
138
```
194
139
195
-
### Start IPOPP services
140
+
### Configure and start IPOPP services
141
+
IPOPP services are configured using its Dashboard GUI.
196
142
197
-
```bash
198
-
cd~/drl/tools
199
-
./services.sh start
200
-
```
201
-
### Verify service status
143
+
[Go to the VM Desktop](#remotely-access-the-vm-desktop) and start a new terminal under **Applications** > **Utilities** > **Terminal**
202
144
145
+
Start the IPOPP dashboard from the terminal:
203
146
```bash
204
-
cd~/drl/tools
205
-
./services.sh status
147
+
~/drl/tools/dashboard.sh
206
148
```
149
+
IPOPP starts in the process monitoring mode. Switch to **Configuration Mode** by the using the menu option.
207
150
208
-
### Configure IPOPP services using its dashboard
209
-
210
-
The above steps start IPOPP services with the default configuration. IPOPP services must be configured using the dashboard GUI.
151
+
Enable the following under the **EOS** tab:
152
+
* gbad
153
+
* MODISL1DB l0l1aqua
154
+
* MODISL1DB l1atob
155
+
* IMAPP
211
156
212
-
To start the dashboard, first port forward the vncserver port (5901) over ssh:
157
+
Switch back to **Process Monitoring** mode using the menu option.
213
158
159
+
Start IPOPP services:
214
160
```bash
215
-
ssh -L 5901:localhost:5901 azureuser@receiver-vm
161
+
~/drl/tools/services.sh start
162
+
~/drl/tools/services.sh status
216
163
```
164
+
This completes the IPOPP installation and configuration.
217
165
218
-
Using the TightVNC client, connect to localhost:5901 and enter the vncserver password that you previously set.
219
-
220
-
On the virtual machine desktop, open a new terminal and start the dashboard:
166
+
## Step 3: Create Level-0 product using RT-STPS
221
167
168
+
Run rt-stps in batch mode to process the ```.bin``` file collected in Step 1
1. IPOPP dashboard starts in process monitoring mode. Switch to **Configuration Mode** by using the menu option.
229
-
230
-
1. Aqua related products can be configured from **EOS** tab in configuration mode. Disable all other tabs. We're interested in the MODIS Aerosol L2 (MOD04) product, which is produced by IMAPP SPA. Therefore, enable the following in the **EOS** tab:
231
-
232
-
- gbad
173
+
This command produces Level-0 Production Data Set (```.pds```) files under the ```~/rt-stps/data``` directory.
233
174
234
-
- MODISL1DB l0l1aqua
235
-
236
-
- MODISL1DB l1atob
237
-
238
-
- IMAPP
239
-
240
-
1. After updating the configuration, switch back to **Process Monitoring** mode using the menu. All tiles will be in OFF mode initially.
241
-
242
-
1. When prompted, save changes to the configuration.
243
-
244
-
1. Click **Start Services** in the action menu. Note that **Start Services** is only enabled in process monitoring mode.
245
-
246
-
1. Click **Check IPOPP Services** in action menu to validate.
175
+
## Step 4: Create higher level products using IPOPP
247
176
248
177
## Ingest data for processing
249
178
250
-
Copy the PDS files generated by RT-STPS to the IPOPP ingest directory for further processing.
179
+
Copy the PDS files generated by RT-STPS in the previous step to the IPOPP ingest directory for further processing.
0 commit comments