Skip to content

Commit 5abe455

Browse files
authored
Update RT-STPS version and re-order sections
1 parent efaebcd commit 5abe455

File tree

1 file changed

+69
-146
lines changed

1 file changed

+69
-146
lines changed

articles/orbital/satellite-imagery-with-orbital-ground-station.md

Lines changed: 69 additions & 146 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,18 @@ Using AOGS, we capture the Aqua broadcast when the satellite is within line of s
2020
In this tutorial, we will follow these steps to collect and process Aqua data:
2121

2222
> [!div class="checklist"]
23-
> * [Schedule a contact and collect Aqua direct broadcast data using AOGS](#step-1-schedule-a-contact-and-collect-aqua-direct-broadcast-data-using-aogs).
24-
> * [Process Aqua direct broadcast data using RT-STPS](#step-2-process-aqua-direct-broadcast-data-using-rt-stps).
25-
> * [Create higher level products using IPOPP](#step-3-create-higher-level-products-using-ipopp).
23+
> * [Use AOGS to schedule and contact and collect Aqua data](#step-1-use-aogs-to-schedule-a-contact-and-collect-aqua-data).
24+
> * [Install NASA DRL tools](#step-2-install-nasa-drl-tools).
25+
> * [Create Level-0 product using RT-STPS](#step-3-create-level-0-product-using-rt-stps).
26+
> * [Create higher level products using IPOPP](#step-4-create-higher-level-products-using-ipopp).
2627
2728
Optional setup steps for capturing the ground station telemetry are included the guide on [receiving real-time telemetry from the ground stations](receive-real-time-telemetry.md).
2829

29-
## Step 1: Schedule a contact and collect Aqua direct broadcast data using AOGS
30+
## Step 1: Use AOGS to schedule a contact and collect Aqua data
3031

31-
Follow the steps listed in [Tutorial: Downlink data from NASA's AQUA public satellite](downlink-aqua.md) to schedule a contact with Aqua using AOGS and collect the direct broadcast data on an Azure VM for further processing.
32+
Execute steps listed in [Tutorial: Downlink data from NASA's AQUA public satellite](downlink-aqua.md)
33+
34+
The above tutorial provides a walkthrough for scheduling a contact with Aqua and collecting the direct broadcast data on an Azure VM.
3235

3336
> [!NOTE]
3437
> In the section [Prepare a virtual machine (VM) to receive the downlinked AQUA data](downlink-aqua.md#prepare-your-virtual-machine-vm-and-network-to-receive-aqua-data), use the following values:
@@ -38,80 +41,15 @@ Follow the steps listed in [Tutorial: Downlink data from NASA's AQUA public sate
3841
> - **Size:** Standard_D8s_v5 or higher
3942
> - **IP Address:** Ensure that the VM has at least one standard public IP address
4043
41-
At the end of this step, you should have the raw direct broadcast saved as ```.bin``` files under the ```~/aquadata``` folder on the receiver-vm.
42-
43-
## Step 2: Process Aqua direct broadcast data using RT-STPS
44-
45-
The [Real-time Software Telemetry Processing System](https://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=69)(RT-STPS) is a NASA-provided software for processing Aqua direct broadcast data. The steps below cover installation of RT-STPS Verson 6.0 on the receiver-vm, and production of Level-0 Production Data Set(PDS) files from the data collected in the previous step.
46-
47-
Register with the [NASA DRL](https://directreadout.sci.gsfc.nasa.gov/) to download the RT-STPS installation package.
48-
49-
Transfer the installation binaries to the receiver-vm:
50-
51-
```console
52-
ssh azureuser@receiver-vm 'mkdir -p ~/software'
53-
scp RT-STPS_6.0*.tar.gz azureuser@receiver-vm:~/software/.
54-
```
55-
56-
Alternatively, you can upload your installation binaries to a container in Azure Storage and download them to the receiver-vm using [AzCopy](../storage/common/storage-use-azcopy-v10.md)
57-
58-
### Install rt-stps
59-
60-
```console
61-
sudo yum install java-11-openjdk
62-
cd ~/software
63-
tar -xzvf RT-STPS_6.0.tar.gz
64-
cd ./rt-stps
65-
./install.sh
66-
```
67-
68-
### Install rt-stps patches
69-
70-
```console
71-
cd ~/software
72-
tar -xzvf RT-STPS_6.0_PATCH_1.tar.gz
73-
tar -xzvf RT-STPS_6.0_PATCH_2.tar.gz
74-
tar -xzvf RT-STPS_6.0_PATCH_3.tar.gz
75-
cd ./rt-stps
76-
./install.sh
77-
```
78-
79-
### Validate install
80-
81-
```console
82-
cd ~/software
83-
tar -xzvf RT-STPS_6.0_testdata.tar.gz
84-
cd ~/software/rt-stps
85-
rm ./data/*
86-
./bin/batch.sh config/npp.xml ./testdata/input/rt-stps_npp_testdata.dat
87-
# Verify that files exist
88-
ls -la ./data
89-
```
90-
91-
### Create Level-0 product
92-
93-
Run rt-stps in batch mode to process the ```.bin``` file collected in Step 1
94-
95-
```console
96-
cd ~/software/rt-stps
97-
./bin/batch.sh ./config/aqua.xml ~/aquadata/raw-2022-05-29T0957-0700.bin
98-
```
99-
100-
This command produces Level-0 Production Data Set (```.pds```) files under the ```~/software/rt-stps/data``` directory.
101-
102-
## Step 3: Create higher level products using IPOPP
103-
104-
[International Planetary Observation Processing Package (IPOPP)](https://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=68) is another NASA-provided software to process Aqua Level-0 data into higher level products.
105-
In the steps below, you'll process the Level-0 data generated in the previous step using IPOPP.
44+
At the end of this step, you should have the raw direct broadcast data saved as ```.bin``` files under the ```~/aquadata``` folder on the ```receiver-vm```.
10645

46+
## Step 2: Install NASA DRL tools
10747
> [!NOTE]
108-
> Due to potential resource contention, DRL recommends installing RT-STPS and IPOPP on separate machines. But for this tutorial, we install both on the our receiver-vm because we don't run them at the same time. For production workloads, please follow sizing and isolation recommendations in the user guides available on the DRL website.
48+
> Due to potential resource contention, DRL recommends installing RT-STPS and IPOPP on separate machines. But for this tutorial, we install both tools on the ```receiver-vm``` because we don't run them at the same time. For production workloads, please follow sizing and isolation recommendations in the user guides available on the DRL website.
10949
11050
### Increase OS disk size on the receiver-vm
11151

112-
IPOPP installation and subsequent generation of products requires more disk space than what is available on an Azure VM by default.
113-
114-
Follow the steps below to increase the size of the OS disk on the ```receiver-vm``` to 1TB.
52+
The default disk space allocated to the OS disk of an Azure VM is not sufficient for installing NASA DRL tools. Follow the steps below to increase the size of the OS disk on the ```receiver-vm``` to 1TB.
11553

11654
### [Portal](#tab/portal2)
11755

@@ -143,131 +81,116 @@ sda 0:0:0:0 1T
14381
```
14482

14583
### Install Desktop and VNC Server
146-
147-
IPOPP installation requires using a browser to sign on to the DRL website to download the installation script. This script must be run from the same host that it was downloaded to. The subsequent IPOPP configuration also requires a GUI. Therefore, we install a full desktop and a vnc server to enable running GUI applications on the receiver-vm.
148-
84+
Using NASA DRL tools requires support for running GUI applications. To enable this, install desktop tools and vncserver on the `receiver-vm`:
14985
```bash
15086
sudo yum install tigervnc-server
15187
sudo yum groups install "GNOME Desktop"
15288
```
153-
15489
Start VNC server:
155-
15690
```bash
15791
vncsever
15892
```
159-
Enter a password when prompted.
160-
161-
Port forward the vncserver port (5901) over ssh:
93+
Enter a password when prompted.
16294

95+
### Remotely access the VM Desktop
96+
Port forward the vncserver port (5901) over SSH to your local machine:
16397
```bash
16498
ssh -L 5901:localhost:5901 azureuser@receiver-vm
16599
```
100+
1. On your local machine, download and install [TightVNC Viewer](https://www.tightvnc.com/download.php).
101+
1. Start the TightVNC Viewer and connect to ```localhost:5901```.
102+
1. Enter the vncserver password you entered in the previous step.
103+
1. You should see the GNOME Desktop that is running on the VM in the VNC viewer window.
166104

167-
Download the [TightVNC](https://www.tightvnc.com/download.php) viewer and connect to ```localhost:5901``` and enter the vncserver password entered in the previous step. You should see the GNOME desktop running on the VM.
105+
### Download RT-STPS and IPOPP installation files
106+
From the GNOME Desktop, go to **Applications** > **Internet** > **Firefox** to start a browser.
168107

169-
Start a new terminal, and start the Firefox browser
108+
Log on to the [NASA DRL](https://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=325&type=software) website and download the **RT-STPS** installation files and the **IPOPP downloader script** under software downloads. The downloaded files will land under ~/Downloads.
170109

110+
Alternatively, you can download the installation files on your local machine first and then upload to a container in Azure Storage. Then use [AzCopy](../storage/common/storage-use-azcopy-v10.md) to download to your ```receiver-vm```.
111+
112+
### Install RT-STPS
113+
```bash
114+
tar -xvzf ~/Downloads/RT-STPS_7.0.tar.gz --directory ~/
115+
tar -xvzf ~/Downloads/RT-STPS_7.0_testdata.tar.gz --directory ~/
116+
cd ~/rt-stps
117+
./install.sh
118+
```
119+
Validate your RT-STPS install by processing the test data supplied with the installation:
171120
```bash
172-
firefox
121+
cd ~/rt-stps
122+
./bin/batch.sh config/jpss1.xml ./testdata/input/rt-stps_jpss1_testdata.dat
173123
```
174-
175-
[Log on the DRL website](https://directreadout.sci.gsfc.nasa.gov/loginDRL.cfm?cid=320&type=software) and download the downloader script.
176-
124+
Verify that output files exist in the data folder:
177125
```bash
178-
cd ~/Downloads
179-
./downloader_DRL-IPOPP_4.1.sh
126+
ls -la ~/data/
180127
```
181-
182-
This script will download \~35G and will take 1 hour or more.
183-
184-
Alternatively, you can upload your installation binaries to a container in Azure Storage and download them to the receiver-vm using [AzCopy](../storage/common/storage-use-azcopy-v10.md)
128+
This completes the RT-STPS installation.
185129

186130
### Install IPOPP
187-
131+
Run the IPOPP downloader script to download the IPOPP installation files.
188132
```bash
189133
cd ~/Downloads
190-
tar -xvzf DRL-IPOPP_4.1.tar.gz --directory ~/
134+
./downloader_DRL-IPOPP_4.1.sh
135+
tar -xvzf ~/Downloads/DRL-IPOPP_4.1.tar.gz --directory ~/
191136
cd ~/IPOPP
192137
./install_ipopp.sh
193138
```
194139

195-
### Start IPOPP services
140+
### Configure and start IPOPP services
141+
IPOPP services are configured using its Dashboard GUI.
196142

197-
```bash
198-
cd ~/drl/tools
199-
./services.sh start
200-
```
201-
### Verify service status
143+
[Go to the VM Desktop](#remotely-access-the-vm-desktop) and start a new terminal under **Applications** > **Utilities** > **Terminal**
202144

145+
Start the IPOPP dashboard from the terminal:
203146
```bash
204-
cd ~/drl/tools
205-
./services.sh status
147+
~/drl/tools/dashboard.sh
206148
```
149+
IPOPP starts in the process monitoring mode. Switch to **Configuration Mode** by the using the menu option.
207150

208-
### Configure IPOPP services using its dashboard
209-
210-
The above steps start IPOPP services with the default configuration. IPOPP services must be configured using the dashboard GUI.
151+
Enable the following under the **EOS** tab:
152+
* gbad
153+
* MODISL1DB l0l1aqua
154+
* MODISL1DB l1atob
155+
* IMAPP
211156

212-
To start the dashboard, first port forward the vncserver port (5901) over ssh:
157+
Switch back to **Process Monitoring** mode using the menu option.
213158

159+
Start IPOPP services:
214160
```bash
215-
ssh -L 5901:localhost:5901 azureuser@receiver-vm
161+
~/drl/tools/services.sh start
162+
~/drl/tools/services.sh status
216163
```
164+
This completes the IPOPP installation and configuration.
217165

218-
Using the TightVNC client, connect to localhost:5901 and enter the vncserver password that you previously set.
219-
220-
On the virtual machine desktop, open a new terminal and start the dashboard:
166+
## Step 3: Create Level-0 product using RT-STPS
221167

168+
Run rt-stps in batch mode to process the ```.bin``` file collected in Step 1
222169
```bash
223-
cd ~/drl/tools
224-
./dashboard.sh
170+
cd ~/rt-stps
171+
./bin/batch.sh ./config/aqua.xml ~/aquadata/raw-2022-05-29T0957-0700.bin
225172
```
226-
Configure services:
227-
228-
1. IPOPP dashboard starts in process monitoring mode. Switch to **Configuration Mode** by using the menu option. 
229-
230-
1. Aqua related products can be configured from **EOS** tab in configuration mode. Disable all other tabs. We're interested in the MODIS Aerosol L2 (MOD04) product, which is produced by IMAPP SPA. Therefore, enable the following in the **EOS** tab: 
231-
232-
- gbad 
173+
This command produces Level-0 Production Data Set (```.pds```) files under the ```~/rt-stps/data``` directory.
233174

234-
- MODISL1DB l0l1aqua 
235-
236-
- MODISL1DB l1atob 
237-
238-
- IMAPP 
239-
240-
1. After updating the configuration, switch back to **Process Monitoring** mode using the menu. All tiles will be in OFF mode initially. 
241-
242-
1. When prompted, save changes to the configuration.  
243-
244-
1. Click **Start Services** in the action menu. Note that **Start Services** is only enabled in process monitoring mode.  
245-
246-
1. Click **Check IPOPP Services** in action menu to validate.
175+
## Step 4: Create higher level products using IPOPP
247176

248177
## Ingest data for processing
249178

250-
Copy the PDS files generated by RT-STPS to the IPOPP ingest directory for further processing.
179+
Copy the PDS files generated by RT-STPS in the previous step to the IPOPP ingest directory for further processing.
251180

252-
```console
253-
cp ~/software/rt-stps/data/* ~/drl/data/dsm/ingest/.
181+
```bash
182+
cp ~/rt-stps/data/* ~/drl/data/dsm/ingest/.
254183
```
255-
256184
Run IPOPP ingest to create the products configured in the dashboard. 
257-
258-
```
185+
```bash
259186
~/drl/tools/ingest_ipopp.sh
260187
```
261-
262188
You can watch the progress in the dashboard.
263-
264-
```
189+
```bash
265190
~/drl/tools/dashboard.sh
266191
```
267-
268192
IPOPP will produce output products in the following directory:
269-
270-
```
193+
```bash
271194
cd ~/drl/data/pub/gsfcdata/aqua/modis/
272195
```
273196

0 commit comments

Comments
 (0)