Skip to content

Commit eb8224b

Browse files
authored
Update all files for the migration of CloudSuite 3. (#319)
1 parent 329cac5 commit eb8224b

File tree

12 files changed

+147
-140
lines changed

12 files changed

+147
-140
lines changed

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# CloudSuite 3.0 #
22

3+
**This branch is an archive where all CloudSuite 3.0 benchmarks are stored. All prebuilt images are available at [cloudsuite3][old] at dockerhub. If you're searching for CloudSuite 4.0, please checkout [master][master] branch.**
4+
35
[CloudSuite][csp] is a benchmark suite for cloud services. The third release consists of eight applications that have
46
been selected based on their popularity in today's datacenters. The benchmarks are based on real-world software
57
stacks and represent real-world setups.
@@ -26,3 +28,5 @@ We encourage CloudSuite users to use GitHub issues for requests for enhancements
2628
[csl]: http://cloudsuite.ch/pages/license/ "CloudSuite License"
2729
[csb]: http://cloudsuite.ch/#download "CloudSuite Benchmarks"
2830
[pkb]: https://github.com/GoogleCloudPlatform/PerfKitBenchmarker "Google's PerfKit Benchmarker"
31+
[old]: https://hub.docker.com/orgs/cloudsuite3/repositories "CloudSuite3 on Dockerhub"
32+
[master]: https://github.com/parsa-epfl/cloudsuite "CloudSuite Master"

docs/benchmarks/data-analytics.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ The benchmark consists of running a Naive Bayes classifier on a Wikimedia datase
1212
To obtain the images:
1313

1414
```bash
15-
$ docker pull cloudsuite/hadoop
16-
$ docker pull cloudsuite/data-analytics
15+
$ docker pull cloudsuite3/hadoop
16+
$ docker pull cloudsuite3/data-analytics
1717
```
1818

1919
## Running the benchmark ##
@@ -30,16 +30,16 @@ Start the master with:
3030

3131
```bash
3232
$ docker run -d --net hadoop-net --name master --hostname master \
33-
cloudsuite/data-analytics master
33+
cloudsuite3/data-analytics master
3434
```
3535

3636
Start a number of slaves with:
3737

3838
```bash
3939
$ docker run -d --net hadoop-net --name slave01 --hostname slave01 \
40-
cloudsuite/hadoop slave
40+
cloudsuite3/hadoop slave
4141
$ docker run -d --net hadoop-net --name slave02 --hostname slave02 \
42-
cloudsuite/hadoop slave
42+
cloudsuite3/hadoop slave
4343
...
4444
```
4545

@@ -51,6 +51,6 @@ Run the benchmark with:
5151
$ docker exec master benchmark
5252
```
5353

54-
[dhrepo]: https://hub.docker.com/r/cloudsuite/data-analytics/ "DockerHub Page"
55-
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/data-analytics.svg "Go to DockerHub Page"
56-
[dhstars]: https://img.shields.io/docker/stars/cloudsuite/data-analytics.svg "Go to DockerHub Page"
54+
[dhrepo]: https://hub.docker.com/r/cloudsuite3/data-analytics/ "DockerHub Page"
55+
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite3/data-analytics.svg "Go to DockerHub Page"
56+
[dhstars]: https://img.shields.io/docker/stars/cloudsuite3/data-analytics.svg "Go to DockerHub Page"

docs/benchmarks/data-caching.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -31,32 +31,32 @@ We will attach the launched containers to this newly created docker network.
3131
### Starting the Server ####
3232
To start the server you have to first `pull` the server image and then run it. To `pull` the server image use the following command:
3333

34-
$ docker pull cloudsuite/data-caching:server
34+
$ docker pull cloudsuite3/data-caching:server
3535

3636
It takes some time to download the image, but this is only required the first time.
3737
The following command will start the server with four threads and 4096MB of dedicated memory, with a minimal object size of 550 bytes listening on port 11211 as default:
3838

39-
$ docker run --name dc-server --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550
39+
$ docker run --name dc-server --net caching_network -d cloudsuite3/data-caching:server -t 4 -m 4096 -n 550
4040

4141
We assigned a name to this server to facilitate linking it with the client. We also used `--net` option to attach the container to our prepared network.
4242
As mentioned before, you can have multiple instances of the Memcached server, just remember to give each of them a unique name. For example, the following commands create four Memcached server instances:
4343

44-
$ docker run --name dc-server1 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550
45-
$ docker run --name dc-server2 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550
46-
$ docker run --name dc-server3 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550
47-
$ docker run --name dc-server4 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550
44+
$ docker run --name dc-server1 --net caching_network -d cloudsuite3/data-caching:server -t 4 -m 4096 -n 550
45+
$ docker run --name dc-server2 --net caching_network -d cloudsuite3/data-caching:server -t 4 -m 4096 -n 550
46+
$ docker run --name dc-server3 --net caching_network -d cloudsuite3/data-caching:server -t 4 -m 4096 -n 550
47+
$ docker run --name dc-server4 --net caching_network -d cloudsuite3/data-caching:server -t 4 -m 4096 -n 550
4848

4949
### Starting the Client ####
5050

5151
To start the client you have to first `pull` the client image and then run it. To `pull` the server image use the following command:
5252

53-
$ docker pull cloudsuite/data-caching:client
53+
$ docker pull cloudsuite3/data-caching:client
5454

5555
It takes some time to download the image, but this is only required the first time.
5656

5757
To start the client container use the following command:
5858

59-
$ docker run -it --name dc-client --net caching_network cloudsuite/data-caching:client bash
59+
$ docker run -it --name dc-client --net caching_network cloudsuite3/data-caching:client bash
6060

6161
This boots up the client container and you'll be logged in as the `memcache` user. Note that by using the `--net` option, you can easily make these containers visible to each other.
6262

@@ -133,11 +133,11 @@ and the client on different sockets of the same machine
133133

134134
[memcachedWeb]: http://memcached.org/ "Memcached Website"
135135

136-
[serverdocker]: https://github.com/parsa-epfl/cloudsuite/blob/master/benchmarks/data-caching/server/Dockerfile "Server Dockerfile"
136+
[serverdocker]: https://github.com/parsa-epfl/cloudsuite/blob/CSv3/benchmarks/data-caching/server/Dockerfile "Server Dockerfile"
137137

138-
[clientdocker]: https://github.com/parsa-epfl/cloudsuite/blob/master/benchmarks/data-caching/client/Dockerfile "Client Dockerfile"
138+
[clientdocker]: https://github.com/parsa-epfl/cloudsuite/blob/CSv3/benchmarks/data-caching/client/Dockerfile "Client Dockerfile"
139139

140140
[repo]: https://github.com/parsa-epfl/cloudsuite "GitHub Repo"
141-
[dhrepo]: https://hub.docker.com/r/cloudsuite/data-caching/ "DockerHub Page"
142-
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/data-caching.svg "Go to DockerHub Page"
143-
[dhstars]: https://img.shields.io/docker/stars/cloudsuite/data-caching.svg "Go to DockerHub Page"
141+
[dhrepo]: https://hub.docker.com/r/cloudsuite3/data-caching/ "DockerHub Page"
142+
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite3/data-caching.svg "Go to DockerHub Page"
143+
[dhstars]: https://img.shields.io/docker/stars/cloudsuite3/data-caching.svg "Go to DockerHub Page"

docs/benchmarks/data-serving.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,22 +21,22 @@ We will attach the launched containers to this newly created docker network.
2121
Start the server container that will run cassandra server and installs a default keyspace usertable:
2222

2323
```bash
24-
$ docker run --name cassandra-server --net serving_network cloudsuite/data-serving:server cassandra
24+
$ docker run --name cassandra-server --net serving_network cloudsuite3/data-serving:server cassandra
2525
```
2626
### Multiple Server Containers
2727

2828
For a cluster setup with multiple servers, we need to instantiate a seed server:
2929

3030
```bash
31-
$ docker run --name cassandra-server-seed --net serving_network cloudsuite/data-serving:server
31+
$ docker run --name cassandra-server-seed --net serving_network cloudsuite3/data-serving:server
3232
```
3333

3434
Then we prepare the server as previously.
3535

3636
The other server containers are instantiated as follows:
3737

3838
```bash
39-
$ docker run --name cassandra-server(id) --net serving_network -e CASSANDRA_SEEDS=cassandra-server-seed cloudsuite/data-serving:server
39+
$ docker run --name cassandra-server(id) --net serving_network -e CASSANDRA_SEEDS=cassandra-server-seed cloudsuite3/data-serving:server
4040
```
4141

4242
You can find more details at the websites: http://wiki.apache.org/cassandra/GettingStarted and https://hub.docker.com/_/cassandra/.
@@ -46,7 +46,7 @@ After successfully creating the aforementioned schema, you are ready to benchmar
4646
Start the client container specifying server name(s), or IP address(es), separated with commas, as the last command argument:
4747

4848
```bash
49-
$ docker run --name cassandra-client --net serving_network cloudsuite/data-serving:client "cassandra-server-seed,cassandra-server1"
49+
$ docker run --name cassandra-client --net serving_network cloudsuite3/data-serving:client "cassandra-server-seed,cassandra-server1"
5050
```
5151

5252
More detailed instructions on generating the dataset can be found in Step 5 at [this](http://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload) link. Although Step 5 in the link describes the data loading procedure, other steps (e.g., 1, 2, 3, 4) are very useful to understand the YCSB settings.
@@ -71,9 +71,9 @@ Running the benchmark
7171
---------------------
7272
The benchmark is run automatically with the client container. One can modify the record count in the database and/or the number of operations performed by the benchmark specifying the corresponding variables when running the client container:
7373
```bash
74-
$ docker run -e RECORDCOUNT=<#> -e OPERATIONCOUNT=<#> --name cassandra-client --net serving_network cloudsuite/data-serving:client "cassandra-server-seed,cassandra-server1"
74+
$ docker run -e RECORDCOUNT=<#> -e OPERATIONCOUNT=<#> --name cassandra-client --net serving_network cloudsuite3/data-serving:client "cassandra-server-seed,cassandra-server1"
7575
```
7676

77-
[dhrepo]: https://hub.docker.com/r/cloudsuite/data-serving/ "DockerHub Page"
78-
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/data-serving.svg "Go to DockerHub Page"
79-
[dhstars]: https://img.shields.io/docker/stars/cloudsuite/data-serving.svg "Go to DockerHub Page"
77+
[dhrepo]: https://hub.docker.com/r/cloudsuite3/data-serving/ "DockerHub Page"
78+
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite3/data-serving.svg "Go to DockerHub Page"
79+
[dhstars]: https://img.shields.io/docker/stars/cloudsuite3/data-serving.svg "Go to DockerHub Page"

docs/benchmarks/graph-analytics.md

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,13 @@ The Graph Analytics benchmark relies the Spark framework to perform graph analyt
1111

1212
Current version of the benchmark is 3.0. To obtain the image:
1313

14-
$ docker pull cloudsuite/graph-analytics
14+
$ docker pull cloudsuite3/graph-analytics
1515

1616
### Datasets
1717

1818
The benchmark uses a graph dataset generated from Twitter. To get the dataset image:
1919

20-
$ docker pull cloudsuite/twitter-dataset-graph
20+
$ docker pull cloudsuite3/twitter-dataset-graph
2121

2222
More information about the dataset is available at
2323
[cloudsuite/twitter-dataset-graph][ml-dhrepo].
@@ -30,8 +30,8 @@ spark-submit.
3030

3131
To run a benchmark with the Twitter dataset:
3232

33-
$ docker create --name data cloudsuite/twitter-dataset-graph
34-
$ docker run --rm --volumes-from data cloudsuite/graph-analytics
33+
$ docker create --name data cloudsuite3/twitter-dataset-graph
34+
$ docker run --rm --volumes-from data cloudsuite3/graph-analytics
3535

3636
### Tweaking the Benchmark
3737

@@ -41,7 +41,7 @@ has enough memory allocated to be able to execute the benchmark
4141
in-memory, supply it with --driver-memory and --executor-memory
4242
arguments:
4343

44-
$ docker run --rm --volumes-from data cloudsuite/graph-analytics \
44+
$ docker run --rm --volumes-from data cloudsuite3/graph-analytics \
4545
--driver-memory 1g --executor-memory 4g
4646

4747
### Multi-node deployment
@@ -54,30 +54,30 @@ with Docker look at [cloudsuite/spark][spark-dhrepo].
5454
First, create a dataset image on every physical node where Spark
5555
workers will be running.
5656

57-
$ docker create --name data cloudsuite/twitter-dataset-graph
57+
$ docker create --name data cloudsuite3/twitter-dataset-graph
5858

5959
Start Spark master and Spark workers. They should all run within the
6060
same Docker network, which we call spark-net here. The workers get
6161
access to the datasets with --volumes-from data.
6262

6363
$ docker run -dP --net spark-net --hostname spark-master --name spark-master \
64-
cloudsuite/spark master
64+
cloudsuite3/spark master
6565
$ docker run -dP --net spark-net --volumes-from data --name spark-worker-01 \
66-
cloudsuite/spark worker spark://spark-master:7077
66+
cloudsuite3/spark worker spark://spark-master:7077
6767
$ docker run -dP --net spark-net --volumes-from data --name spark-worker-02 \
68-
cloudsuite/spark worker spark://spark-master:7077
68+
cloudsuite3/spark worker spark://spark-master:7077
6969
$ ...
7070

7171
Finally, run the benchmark as the client to the Spark master:
7272

7373
$ docker run --rm --net spark-net --volumes-from data \
74-
cloudsuite/graph-analytics \
74+
cloudsuite3/graph-analytics \
7575
--driver-memory 1g --executor-memory 4g \
7676
--master spark://spark-master:7077
7777

78-
[dhrepo]: https://hub.docker.com/r/cloudsuite/graph-analytics/ "DockerHub Page"
79-
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/graph-analytics.svg "Go to DockerHub Page"
80-
[dhstars]: https://img.shields.io/docker/stars/cloudsuite/graph-analytics.svg "Go to DockerHub Page"
81-
[ml-dhrepo]: https://hub.docker.com/r/cloudsuite/twitter-dataset-graph/
82-
[spark-dhrepo]: https://hub.docker.com/r/cloudsuite/spark/
78+
[dhrepo]: https://hub.docker.com/r/cloudsuite3/graph-analytics/ "DockerHub Page"
79+
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite3/graph-analytics.svg "Go to DockerHub Page"
80+
[dhstars]: https://img.shields.io/docker/stars/cloudsuite3/graph-analytics.svg "Go to DockerHub Page"
81+
[ml-dhrepo]: https://hub.docker.com/r/cloudsuite3/twitter-dataset-graph/
82+
[spark-dhrepo]: https://hub.docker.com/r/cloudsuite3/spark/
8383

docs/benchmarks/in-memory-analytics.md

Lines changed: 23 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -22,17 +22,17 @@ squares (ALS) algorithm which is provided by Spark MLlib.
2222

2323
Current version of the benchmark is 3.0. To obtain the image:
2424

25-
$ docker pull cloudsuite/in-memory-analytics
25+
$ docker pull cloudsuite3/in-memory-analytics
2626

2727
### Datasets
2828

2929
The benchmark uses user-movie ratings datasets provided by Movielens. To get
3030
the dataset image:
3131

32-
$ docker pull cloudsuite/movielens-dataset
32+
$ docker pull cloudsuite3/movielens-dataset
3333

3434
More information about the dataset is available at
35-
[cloudsuite/movielens-dataset][ml-dhrepo].
35+
[cloudsuite3/movielens-dataset][ml-dhrepo].
3636

3737
### Running the Benchmark
3838

@@ -41,14 +41,14 @@ distributed with Spark. It takes two arguments: the dataset to use for
4141
training, and the personal ratings file to give recommendations for. Any
4242
remaining arguments are passed to spark-submit.
4343

44-
The cloudsuite/movielens-dataset image has two datasets (one small and one
44+
The cloudsuite3/movielens-dataset image has two datasets (one small and one
4545
large), and a sample personal ratings file.
4646

4747
To run a benchmark with the small dataset and the provided personal ratings
4848
file:
4949

50-
$ docker create --name data cloudsuite/movielens-dataset
51-
$ docker run --rm --volumes-from data cloudsuite/in-memory-analytics \
50+
$ docker create --name data cloudsuite3/movielens-dataset
51+
$ docker run --rm --volumes-from data cloudsuite3/in-memory-analytics \
5252
/data/ml-latest-small /data/myratings.csv
5353

5454
### Tweaking the Benchmark
@@ -58,7 +58,7 @@ be used to tweak execution. For example, to ensure that Spark has enough memory
5858
allocated to be able to execute the benchmark in-memory, supply it with
5959
--driver-memory and --executor-memory arguments:
6060

61-
$ docker run --rm --volumes-from data cloudsuite/in-memory-analytics \
61+
$ docker run --rm --volumes-from data cloudsuite3/in-memory-analytics \
6262
/data/ml-latest /data/myratings.csv \
6363
--driver-memory 2g --executor-memory 2g
6464

@@ -67,32 +67,35 @@ allocated to be able to execute the benchmark in-memory, supply it with
6767
This section explains how to run the benchmark using multiple Spark workers
6868
(each running in a Docker container) that can be spread across multiple nodes
6969
in a cluster. For more information on running Spark with Docker look at
70-
[cloudsuite/spark][spark-dhrepo].
70+
[cloudsuite3/spark][spark-dhrepo].
7171

7272
First, create a dataset image on every physical node where Spark workers will
7373
be running.
7474

75-
$ docker create --name data cloudsuite/movielens-dataset
75+
$ docker create --name data cloudsuite3/movielens-dataset
76+
77+
Then, create dedicated network for spark workers:
78+
79+
$ docker network create spark-net
7680

7781
Start Spark master and Spark workers. They should all run within the same
78-
Docker network, which we call spark-net here. The workers get access to the
79-
datasets with --volumes-from data.
82+
Docker network, which we call spark-net here. The workers get access to the datasets with --volumes-from data.
8083

81-
$ docker run -dP --net spark-net --hostname spark-master --name spark-master cloudsuite/spark master
82-
$ docker run -dP --net spark-net --volumes-from data --name spark-worker-01 cloudsuite/spark worker \
84+
$ docker run -dP --net spark-net --hostname spark-master --name spark-master cloudsuite3/spark master
85+
$ docker run -dP --net spark-net --volumes-from data --name spark-worker-01 cloudsuite3/spark worker \
8386
spark://spark-master:7077
84-
$ docker run -dP --net spark-net --volumes-from data --name spark-worker-02 cloudsuite/spark worker \
87+
$ docker run -dP --net spark-net --volumes-from data --name spark-worker-02 cloudsuite3/spark worker \
8588
spark://spark-master:7077
8689
$ ...
8790

8891
Finally, run the benchmark as the client to the Spark master:
8992

90-
$ docker run --rm --net spark-net --volumes-from data cloudsuite/in-memory-analytics \
93+
$ docker run --rm --net spark-net --volumes-from data cloudsuite3/in-memory-analytics \
9194
/data/ml-latest-small /data/myratings.csv --master spark://spark-master:7077
9295

93-
[dhrepo]: https://hub.docker.com/r/cloudsuite/in-memory-analytics/ "DockerHub Page"
94-
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/in-memory-analytics.svg "Go to DockerHub Page"
95-
[dhstars]: https://img.shields.io/docker/stars/cloudsuite/in-memory-analytics.svg "Go to DockerHub Page"
96-
[ml-dhrepo]: https://hub.docker.com/r/cloudsuite/movielens-dataset/
97-
[spark-dhrepo]: https://hub.docker.com/r/cloudsuite/spark/
96+
[dhrepo]: https://hub.docker.com/r/cloudsuite3/in-memory-analytics/ "DockerHub Page"
97+
[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite3/in-memory-analytics.svg "Go to DockerHub Page"
98+
[dhstars]: https://img.shields.io/docker/stars/cloudsuite3/in-memory-analytics.svg "Go to DockerHub Page"
99+
[ml-dhrepo]: https://hub.docker.com/r/cloudsuite3/movielens-dataset/
100+
[spark-dhrepo]: https://hub.docker.com/r/cloudsuite3/spark/
98101

0 commit comments

Comments
 (0)