Home

Welcome to the OptSched wiki!

This is a place for helpful tips and tricks for working with OptSched as a project developer.

• Long-running commands over SSH
  • Task queue with screen
• Run a tool on corresponding files between directory structures

---

Running a long-running command over SSH

Building the benchmark suites take a long time. There are also several long-running tasks which we may need to do. For this, the screen command can be useful.

In it's simplest form, run: screen -q the command to run (e.g. screen -q ninja), or screen -q bash -c '...' for more complex scripts which need redirection or pipes. Ctrl+AD puts the screen into the background, and you can then log out of your ssh session. screen -r resumes.

Building a task queue with screen

More generally, you can have screen work as a task queue and queue up several commands to run after each other. See this SuperUser answer:

startqueue: starts the queuing system.

#!/usr/bin/env bash
screen -d m -S queue

queue: enqueue a command

#!/usr/bin/env bash
screen -S queue -X stuff "$@^M"

Where the ^M is a single special character. In vim in insert mode, produce it by typing Ctrl+V M. You may want to :set list to show the special character.

viewqueue: Look at the screen.

#!/usr/bin/env bash
screen -S queue -r

A more advanced task queue

With a little work, we can make it possible to view what has started/ended in the queue:

startqueue: starts the queuing system.

#/usr/bin/env bash
screen -d -m -S queuerun
screen -d -m -S queueview
screen -S queueview -X stuff "while true; do read; done >/dev/null^M"

queue: enqueue a command

#!/usr/bin/env bash
tmstmp=$(date '+%Y-%m-%d %r-%Z')
screen -S queueview -X stuff "Queue: $tmstmp  $@\n"
screen -S queuerun -X stuff "screen -S queueview -X stuff \"Start:   \$(date '+%Y-%m-%d %r-%Z')  $@\n\""
screen -S queuerun -X stuff "$@^M"
screen -S queuerun -X stuff "screen -S queueview -X stuff \"End:   \$(date '+%Y-%m-%d %r-%Z')  $@\n\""

viewqueue: Look at the queue.

#!/usr/bin/env bash
screen -S queueview -r

viewqueueout: Look at the terminal on which everything is being run.

#!/usr/bin/env bash
screen -S queueview -r

Again, the ^Ms are a single special character.

Run over corresponding files across mirrored directories

We often have multiple differently-configured run results in directories. As an example, consider a/ and b/ both with 0.log and 1.log. How do we run a tool over the corresponding files a/0.log b/0.log and a/1.log b/1.log?

$ paste -d ' \n' <(find a/ -type f) <(find b/ -type f) | xargs -L1 echo
a/0.log b/0.log
a/1.log b/1.log

You could also just paste -d '\n' and use xargs -L2 instead.

paste -d ' \n' takes the two redirected shell results (<(...)) and zips the lines together; with just '\n', the lines are interleaved. xargs uses stdin as the commandline arguments of what follows it, but -L# tells it to stick to # lines per command invocation.