Small fixes: memory tests, alternate binaries, ... (#76)

- Required apps to be installed in $GOPATH/bin
- Changed memory tests to require 2000MB, not 2048MB
- Updated READMEs, removed images for new install process
- Updated default app server destinations from @perrig
- Handled more bwtest errors
- Moved all bwest graph points to use test start, not end
- Fixed camerapp storage issue moving fetch to `webapp/data/images`
- Allows continuous SCMP echo console feedback during continuous test
- Prevents camerapp and sensorapp from using continuous switch
- Memory tests now show size of memory during failure
- SCION run test now ignores default sciond socket on localhost topology
- Adds localhost test script to launch local scionlab apps servers

Author: mwfarb

README.md

@@ -8,7 +8,7 @@ More information on [SCION](https://www.scion-architecture.net/), and [tutorials
 
 To build:
 1. run once the provided script `deps.sh` to set up the dependencies
-2. run `make` to build all projects
+2. run `make install` to build all projects and install into `$GOPATH/bin`
 
 
 ## bat

webapp/README.md

@@ -1,143 +1,42 @@
-SCIONLab Go Static Tester
+Webapp AS Visualization
 =========================
 
-## Webapp Setup
-
-Webapp is a Go application that will serve up a static web portal to make it easy to
-experiment with SCIONLab test apps on a virtual machine.
-
-To install the SCIONLab `webapp`:
-
-```shell
-go get github.com/netsec-ethz/scion-apps/webapp
-```
-
-### Local Infrastructure
+More installation and usage information is available on the [SCION Tutorials web page for webapp](https://netsec-ethz.github.io/scion-tutorials/as_visualization/webapp/).
 
-To run the Go Web UI at default localhost 127.0.0.1:8000 run:
+## Webapp Setup
+Webapp is a Go application that will serve up a static web portal to make it easy to visualize and experiment with SCIONLab test apps on a virtual machine.
 
+### Install
 ```shell
-webapp
+mkdir ~/go/src/github.com/netsec-ethz
+cd ~/go/src/github.com/netsec-ethz
+git clone https://github.com/netsec-ethz/scion-apps.git
 ```
 
-### SCIONLab Virtual Machine
-
-Using vagrant, make sure to edit your `vagrantfile` to provision the additional port
-for the Go web server by adding this line for port 8080 (for example, just choose any forwarding
-port not already in use by vagrant):
-
-```
-config.vm.network "forwarded_port", guest: 8080, host: 8080, protocol: "tcp"
-```
-
-To run the Go Web UI at a specific address (-a) and port (-p) like 0.0.0.0:8080 for a SCIONLabVM use:
-
+### Build
+Install all [SCIONLab apps](https://github.com/netsec-ethz/scion-apps) and dependancies, including `webapp`:
 ```shell
-webapp -a 0.0.0.0 -p 8080 -r .
+cd scion-apps
+./deps.sh
+make install
 ```
 
-Now, open a web browser at http://127.0.0.1:8080, to begin.
+### Run
+!!! warning
+    If the old [scion-viz](https://github.com/netsec-ethz/scion-viz) web server is running on your SCIONLab VM, port 8000 may still be in use. To remedy this, before `vagrant up`, make sure to edit your `vagrantfile` to provision an alternate port for the `webapp` web server. Add this line for a different port, say 8080 (for example, just choose any forwarding port not already in use by vagrant, and use that port everywhere below):
 
-## Development
+    ```
+    config.vm.network "forwarded_port", guest: 8080, host: 8080, protocol: "tcp"
+    ```
 
-For developing the web server go code, since it is annoying to make several changes,
-only to have to start and stop the web server each time, a watcher library
-like `go-watcher` is recommended.
-
-```shell
-go get github.com/canthefason/go-watcher
-go install github.com/canthefason/go-watcher/cmd/watcher
-```
-
-After installation you can `cd` to the `webapp.go` directory and webapp will be rebuilt
-and rerun every time you save your source file changes, with or without command arguments.
-
-```shell
-cd ~/go/src/github.com/netsec-ethz/scion-apps/webapp
-watcher -a 0.0.0.0 -p 8080 -r ..
-```
-or
+To run the Go Web UI at a specific address (-a) and port (-p) like 0.0.0.0:8000 for a SCIONLab VM use:
 ```shell
-cd ~/go/src/github.com/netsec-ethz/scion-apps/webapp
-watcher
+cd webapp
+webapp -a 0.0.0.0 -p 8000
 ```
+Now, open a web browser at [http://127.0.0.1:8000](http://127.0.0.1:8000), to begin.
 
-## Webapp Features
-
-This Go web server wraps several SCION test client apps and provides an interface
-for any text and/or image output received.
-[SCIONLab Apps](http://github.com/netsec-ethz/scion-apps) are on Github.
-
-Two functional server tests are included to test the networks without needing
-specific sensor or camera hardware, `imagetest` and `statstest`.
-
-Supported client applications include `camerapp`, `sensorapp`, and `bwtester`.
-For best results, ensure the desired server-side apps are running and connected to
-the SCION network first. Instructions to setup the servers are
-[here](https://github.com/netsec-ethz/scion-apps/blob/master/README.md).
-The web interface launched above can be used to run the client-side apps.
-
-### File System Browser
-
-The File System Browser button on the front page will allow you to navigate and serve any
-files on the SCIONLab node from the root (-r) directory you specified (if any) when
-starting webapp.go.
-
-### bwtester client
-
-Simply adjust the dials to the desired level, while the icon lock will allow you
-to keep one value constant.
-
-![Webapp Bandwidth Test](static/img/bwtest.png?raw=true "Webapp Bandwidth Test")
-
-
-### camerapp client
-
-The retrieved image will appear scaled down and can be clicked on to open a larger size.
-
-![Webapp Image Test](static/img/imagetest.png?raw=true "Webapp Image Test")
-
-
-### sensorapp client
-
-![Webapp Stats Test](static/img/statstest.png?raw=true "Webapp Stats Test")
-
-
-### statstest server
-
-This hardware-independent test will echo some remote machine stats from the Python script
-`local-stats.py`, which is piped to the server for transmission to clients.
-On your remote SCION server node run (substituting your own address parameters):
-
-```shell
-cd $GOPATH/src/github.com/netsec-ethz/scion-apps/webapp/tests/statstest/statsserver
-python3 local-stats.py | sensorserver -s 1-15,[127.0.0.5]:35555
-```
-
-Now, from your webapp browser interface running on your virtual client SCION node,
-you can enter both client and server addresses and ask the client for remote stats
-by clicking on the `sensorapp` tab.
-
-### imagetest server
-
-This hardware-independent test will generate an image with some remote machine stats from
-the Go app `local-image.go`, which will be saved locally for transmission to clients.
-
-You may need golang.org's image package first:
-
-```shell
-go get golang.org/x/image
-```
-
-On your remote SCION server node run (substituting your own address parameters):
-
-```shell
-cd $GOPATH/src/github.com/netsec-ethz/scion-apps/webapp/tests/imgtest/imgserver
-go build
-./imgserver | imageserver -s 1-18,[127.0.0.8]:38887
-```
-
-Now, from your webapp browser interface running on your virtual client SCION node,
-you can enter both client and server addresses and ask the client for the most
-recently generated remote image by clicking on the `camerapp` tab.
-
+## Related Links
+* [Webapp SCIONLab AS Visualization Tutorials](https://netsec-ethz.github.io/scion-tutorials/as_visualization/webapp/)
+* [Webapp SCIONLab Apps Visualization](https://netsec-ethz.github.io/scion-tutorials/as_visualization/webapp_apps/)
+* [Webapp Development Tips](https://netsec-ethz.github.io/scion-tutorials/as_visualization/webapp_development/)

webapp/config/servers_default.json

@@ -15,7 +15,7 @@
   {
    "name": "17-ffaa:1:13",
    "isdas": "17-ffaa:1:13",
-   "addr": "192.168.1.111",
+   "addr": "192.168.1.79",
    "port": 30100
   },
   {
@@ -67,34 +67,34 @@
   {
    "name": "17-ffaa:1:13",
    "isdas": "17-ffaa:1:13",
-   "addr": "192.168.1.111",
+   "addr": "192.168.1.79",
    "port": 42003
   }
  ],
  "echo": [
   {
    "name": "17-ffaa:0:1107 Attachment Point",
    "isdas": "17-ffaa:0:1107",
-   "addr": "10.0.8.1",
-   "port": 30100
+   "addr": "192.33.93.195",
+   "port": 40002
   },
   {
    "name": "18-ffaa:0:1202 Attachment Point",
    "isdas": "18-ffaa:0:1202",
-   "addr": "10.0.8.1",
-   "port": 30100
+   "addr": "128.105.21.208",
+   "port": 40002
   },
   {
    "name": "19-ffaa:0:1303 Attachment Point",
    "isdas": "19-ffaa:0:1303",
-   "addr": "10.0.8.1",
-   "port": 30100
+   "addr": "141.44.25.144",
+   "port": 40002
   },
   {
    "name": "20-ffaa:0:1404 Attachment Point",
    "isdas": "20-ffaa:0:1404",
-   "addr": "10.0.8.1",
-   "port": 30100
+   "addr": "203.230.60.98",
+   "port": 40002
   }
  ]
 }

webapp/lib/bwcont.go

@@ -31,6 +31,8 @@ var reItMax = `(?i:interarrival time max:\s*)([0-9.-]*)(?:\s*ms)`
 var reErr1 = `(?i:err=*)"(.*?)"`
 var reErr2 = `(?i:crit msg=*)"(.*?)"`
 var reErr3 = `(?i:error:\s*)([\s\S]*)`
+var reErr4 = `(?i:eror:\s*)([\s\S]*)`
+var reErr5 = `(?i:crit:\s*)([\s\S]*)`
 var reUPath = `(?i:using path:)`
 
 // ExtractBwtestRespData will parse cmd line output from bwtester for adding BwTestItem fields.
@@ -98,6 +100,8 @@ func ExtractBwtestRespData(resp string, d *model.BwTestItem, start time.Time) {
 		match1, _ := regexp.MatchString(reErr1, r[i])
 		match2, _ := regexp.MatchString(reErr2, r[i])
 		match3, _ := regexp.MatchString(reErr3, r[i])
+		match4, _ := regexp.MatchString(reErr4, r[i])
+		match5, _ := regexp.MatchString(reErr5, r[i])
 
 		if match1 {
 			re := regexp.MustCompile(reErr1)
@@ -108,6 +112,12 @@ func ExtractBwtestRespData(resp string, d *model.BwTestItem, start time.Time) {
 		} else if match3 {
 			re := regexp.MustCompile(reErr3)
 			err = re.FindStringSubmatch(r[i])[1]
+		} else if match4 {
+			re := regexp.MustCompile(reErr4)
+			err = re.FindStringSubmatch(r[i])[1]
+		} else if match5 {
+			re := regexp.MustCompile(reErr5)
+			err = re.FindStringSubmatch(r[i])[1]
 		} else if err == "" && r[i] != "" {
 			// fallback to first line if err msg needed
 			err = r[i]

webapp/lib/image.go

@@ -73,7 +73,7 @@ func findNewestFileExt(dir, extRegEx string) (imgFilename string, imgTimestamp i
 
 // FindImageInfoHandler locating most recent image and writing text info data about it.
 func FindImageInfoHandler(w http.ResponseWriter, r *http.Request) {
-	filesDir := "."
+	filesDir := path.Join(GOPATH, LABROOT, "webapp/data/images")
 	imgFilename, _ := findNewestFileExt(filesDir, regexImageFiles)
 	if imgFilename == "" {
 		return
@@ -84,7 +84,7 @@ func FindImageInfoHandler(w http.ResponseWriter, r *http.Request) {
 
 // FindImageHandler locating most recent image formatting it for graphic display in response.
 func FindImageHandler(w http.ResponseWriter, r *http.Request, browserAddr string, port int) {
-	filesDir := "."
+	filesDir := path.Join(GOPATH, LABROOT, "webapp/data/images")
 	imgFilename, _ := findNewestFileExt(filesDir, regexImageFiles)
 	if imgFilename == "" {
 		fmt.Fprint(w, "Error: Unable to find image file locally.")
@@ -111,6 +111,6 @@ func FindImageHandler(w http.ResponseWriter, r *http.Request, browserAddr string
 		panic("Unhandled image type!")
 	}
 	CheckError(err)
-	url := fmt.Sprintf("http://%s:%d/%s/%s", browserAddr, port, "images", imgFilename)
+	url := fmt.Sprintf("http://%s:%d/%s/%s", browserAddr, port, "data/images", imgFilename)
 	writeJpegTemplate(w, &rawImage, url)
 }