Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions montage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<img align=right src=montage.png width=256></img>

Montage Workflow Example
------------------------

This workflow gives an example of using Makeflow to parallelize
the Montage astronomical image mosaic tool.

If you have not done so already, please clone this example repository like so:
```
git clone https://github.com/cooperative-computing-lab/makeflow-examples.git
cd ./makeflow-examples/montage
```

First, build the Montage binary for your architecture:

```
git clone https://github.com/Caltech-IPAC/Montage ./montage
cp ./mDAG.c ./montage/grid/Pegasus
cd montage
make
cd ..
```

Then, create a working directory and generate a workflow:

```
mkdir ./montage_example
./montage/bin/mDAG 2mass j m51 0.5 0.5 0.00277778 ./montage_example http://montage-web.ipac.caltech.edu/workspace/m51
```

Next, run the file_get.pl script to grab all the necessary images of the sky using the URLs generated by mDAG:

```
cp ./file_get.pl ./montage_example
cd ./montage_example
perl ./file_get.pl ./url.list
```

Finally, execute the workflow using makeflow locally,
or using a batch system like Condor, SGE, or Work Queue:

```
makeflow montage.mf
makeflow -T condor montage.mf
makeflow -T sge montage.mf
makeflow -T wq montage.mf
```
39 changes: 39 additions & 0 deletions montage/file_get.pl
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/perl
#
#Copyright (C) 2016- The University of Notre Dame
#This software is distributed under the GNU General Public License.
#See the file COPYING for details.
#

use strict;
use Scalar::Util qw(looks_like_number);
use Math::Complex;
use POSIX;

my $num_args = $#ARGV + 1;
if($num_args ne 1) {
print "Must specify input data file.\n";
exit 1;
}

my $in = $ARGV[0];
my $result = 0;
my $i = 1;
open(INPUT, $in);

while (my $line = <INPUT>) {
chomp $line;
my @parts = split(" ",$line);
my $final = $parts[0];
my $intermediate = $parts[0] . ".gz";
my $url = $parts[1];
$result = system("curl $url -o $intermediate");
if($result ne 0) { print "Error in curl #$i\n"; exit 1; }
$result = system("gunzip -c $intermediate > $final; rm $intermediate");
if($result ne 0) { print "Error in gunzip and rm #$i\n"; exit 1; }
$i++;
sleep(1);
}

close(INPUT);
exit 0;
Loading