updates v0.5.0 tracking docs

Author: troyraen

version_tracking/v0.5.0/README.md

@@ -1,18 +1,29 @@
 # v0.5.0
 
+- [Results Summary](#results-summary)  
 - [Documentation of Changes](#documentation-of-changes)
 - [Test the Changes](#test-the-changes)
-- [Results Summary](#results-summary)  
+
+---
+
+## Results Summary
+
+Auto-scheduler and cue-response checker appear to be working smoothly.
+
+Documentation is back up on ReadTheDocs.
 
 ---
 
 ## Documentation of Changes
 
 - PR [#62](https://github.com/mwvgroup/Pitt-Google-Broker/pull/62)
-- [schedule-night-conductor.md](schedule-night-conductor.md)
-- [external-connection.md](external-connection.md)
-
+- Auto-scheduler: [schedule-night-conductor.md](schedule-night-conductor.md)
+- Pub/Sub tutorial setup:
+    - [external-connection.md](external-connection.md)  (setup an external GCP project and connect to our Pub/Sub streams)
+    - [stream-looper.md](stream-looper.md) (setup a VM to run a consumer simulator that publishes ZTF alerts to a dedicated PS topic on a slow loop)
 
+Started but didn't finish; TODO in a future version:
+- [supernnova.md](supernnova.md) (implement SuperNNova)
 
 ---
 
@@ -89,52 +100,4 @@ attr=topic_date=20210727
 gcloud pubsub topics publish "$topic" --message="$cue" --attribute="$attr"
 ```
 
-#### consumer sim, tbd
-
-
-start/stop the broker
-```bash
-survey=ztf
-testid=v050
-topic="${survey}-cue_night_conductor-${testid}"
-
-# start the night
-cue=START
-attr=KAFKA_TOPIC=NONE  # leave consumer VM off
-gcloud pubsub topics publish "$topic" --message="$cue" --attribute="$attr"
-
-# end the night
-cue=END
-gcloud pubsub topics publish "$topic" --message="$cue"
-```
-
-```python
-from broker_utils import consumer_sim as bcs
-
-testid = 'v050'
-survey = 'decat'
-sub_id = 'decat-alerts-reservoir-testschema'  # production instance doesn't exist yet
-# survey = 'ztf'
-# sub_id = 'ztf_alerts-reservoir'  # production instance names are not yet updated
-instance = (survey, testid)
-# alert_rate = (100, 'once')
-alert_rate = 'ztf-active-avg'
-runtime = (30, 'min')  # options: 'sec', 'min', 'hr', 'night'(=10 hrs)
-
-bcs.publish_stream(alert_rate, instance, sub_id=sub_id, runtime=runtime)
-```
-```bash
-# end the night
-NIGHT="END"
-gcloud compute instances add-metadata "$nconductVM" --zone="$zone" \
-        --metadata NIGHT="$NIGHT"
-gcloud compute instances start "$nconductVM" --zone "$zone"
-```
-
 <!-- fe Test the Changes -->
-
-
-
----
-
-## Results Summary

version_tracking/v0.5.0/stream-looper.md

@@ -1,6 +1,7 @@
-## Setup stream-looper VM and topic
+# Setup stream-looper VM and topic
+
+## Create the Pub/Sub topic and allow public subscriptions
 
-Create the topic and allow public subscriptions
 ```bash
 PROJECT=$GOOGLE_CLOUD_PROJECT
 TOPIC="ztf-loop"
@@ -15,7 +16,8 @@ gcloud pubsub topics get-iam-policy $topic_path --format yaml > $fnametmp
 gcloud pubsub topics set-iam-policy $topic_path $fname
 ```
 
-Create the VM and setup the consumer simulator
+## Create the VM and setup the consumer simulator
+
 ```bash
 vmname="stream-looper"
 zone="us-central1-a"
@@ -34,21 +36,44 @@ gcloud compute instances create "$vmname" \
 gcloud compute instances add-metadata "$vmname" --zone="$zone" --metadata=startup-script=""
 
 # ssh in
-gcloud compute ssh $vmname
-screen
-ipython3
+# gcloud compute ssh $vmname
 ```
+
+### Set a startup script to run consumer simulator indefinitely
+
+Python script to trigger at startup.
+Save the following code to a root-executable file at
+/home/consumer_sim/run-looper-indefinitely.py:
+
 ```python
+#!/usr/bin/env python3
+# -*- coding: UTF-8 -*-
 from broker_utils import consumer_sim as bcs
 
-# publish 1 alert every 5 seconds
-alert_rate = (12, 'perMin')
+# set args to publish 1 alert every second
+alert_rate = (60, 'perMin')
 kwargs = {
     'instance': None,
-    'runtime': (1, 'hr'),
-    'publish_batch_every': (5, 'sec'),
-    'sub_id': 'ztf-alerts-reservoir-v050',
+    'runtime': (1, 'night'),
+    'publish_batch_every': (1, 'sec'),
+    'sub_id': 'ztf-alerts-reservoir',
     'topic_id': 'ztf-loop',
+    'auto_confirm': True,
 }
-bcs.publish_stream(alert_rate, **kwargs)
+
+# run indefinitely
+while True:
+    bcs.publish_stream(alert_rate, **kwargs)
+```
+
+Set a startup script to execute the python file in a background thread:
+```bash
+startupscript="#! /bin/bash
+dir=/home/consumer_sim/
+fname=run-looper-indefinitely
+nohup python3 \${dir}\${fname}.py >> \${dir}\${fname}.out 2>&1 &
+"
+
+# set the startup script
+gcloud compute instances add-metadata "$vmname" --metadata=startup-script="$startupscript"
 ```