Skip to content

Commit 2b7accb

Browse files
cicirellocicirello
authored
Merge pull request #5 from cicirello/readme
Added info on building and running experiments to README
2 parents 1fdb49f + 9f6335c commit 2b7accb

File tree

2 files changed

+173
-1
lines changed

2 files changed

+173
-1
lines changed

‎README.md‎

Lines changed: 148 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,149 @@
11
# MONE2022-experiments
2-
Experiments for paper submitted to the journal MONE
2+
3+
Copyright © 2018-2019, 2022 Vincent A. Cicirello
4+
5+
This repository contains code to reproduce the experiments, and analysis of
6+
experimental data, from the following paper:
7+
8+
> Vincent A. Cicirello. 2022. On Fitness Landscape Analysis of Permutation Problems: From Distance Metrics to Mutation Operator Selection. *CURRENTLY UNDER REVIEW*.
9+
10+
11+
| | |
12+
| :--- | :--- |
13+
| __License__ | [![GitHub](https://img.shields.io/github/license/cicirello/MONE2022-experiments)](LICENSE) |
14+
15+
## Requirements to Build and Run the Experiments
16+
17+
To build and run the experiments on your own machine, you will need the following:
18+
* __JDK 11__: I used OpenJDK 11, but you should be fine with Oracle's
19+
JDK as well.
20+
* __Apache Maven__: In the root of the repository, there is a `pom.xml`
21+
for building the Java programs for the experiments. Using this `pom.xml`,
22+
Maven will take care of downloading the exact versions of
23+
[Chips-n-Salsa](https://chips-n-salsa.cicirello.org/) (release 4.2.1),
24+
[JavaPermutationTools](https://jpt.cicirello.org) (release 3.0.0), and
25+
[ρμ](https://rho-mu.cicirello.org) (release 1.1.0) that were
26+
used in the experiments.
27+
* __Python 3__: The repository contains Python programs that were used to
28+
compute summary statistics, and to generate
29+
graphs for the figures of the paper. If you want to run the Python programs,
30+
you will need Python 3. I specifically used Python 3.9.6. You also need
31+
matplotlib installed.
32+
* __Make__: The repository contains a Makefile to simplify running the build,
33+
running the experiment's Java programs, and running the Python program to
34+
analyze the data. If you are familiar with using the Maven build tool,
35+
and running Python programs, then you can just run these directly, although
36+
the Makefile may be useful to see the specific commands needed.
37+
38+
## Building the Java Programs (Option 1)
39+
40+
The source code of the Java programs, implementing the experiments
41+
is in the [src/main](src/main) directory. You can build the experiment
42+
programs in one of the following ways.
43+
44+
__Using Maven__: Execute the following from the root of the
45+
repository.
46+
47+
```shell
48+
mvn clean package
49+
```
50+
51+
__Using Make__: Or, you can execute the following from the root
52+
of the repository.
53+
54+
```shell
55+
make build
56+
```
57+
58+
This produces a jar file containing 6 Java programs for running
59+
different parts of the experiments and analysis. The jar also contains all
60+
dependencies, including [Chips-n-Salsa](https://chips-n-salsa.cicirello.org/),
61+
[JavaPermutationTools](https://jpt.cicirello.org), and
62+
[ρμ](https://rho-mu.cicirello.org).
63+
If you are unfamiliar with the usual structure of the directories of
64+
a Java project built with Maven, the `.class` files, the `.jar` file,
65+
etc will be found in a `target` directory that is created by the
66+
build process.
67+
68+
## Running the Experiments
69+
70+
If you just want to inspect the data from my runs, then you can find that output
71+
in the [/data](data) directory. If you instead want to run the experiments yourself,
72+
you must first either follow the build instructions or download a prebuilt jar (see above
73+
sections). Once the jar of the experiments is either built or downloaded, you can then run
74+
the experiments with the following executed at the root of the repository:
75+
76+
```shell
77+
make experiments
78+
```
79+
80+
If you don't want to overwrite my original data files, then first change the variable
81+
`pathToDataFiles` in the `Makefile` before running the above command.
82+
83+
This will run each of the experiment programs in sequence,
84+
with the results piped to text files in the [/data](data) directory. Note that
85+
this directory already contains the output from my runs, so if you execute this command,
86+
you will overwrite the data that was used in the paper. Some parts of this will not
87+
change, but certain parts, due to randomization may not be exactly the same, although should
88+
be statistically consistent.
89+
90+
There are also several other targets in the Makefile if you wish to
91+
run only some of the experiments from the paper. See the Makefile for
92+
details.
93+
94+
## Analyzing the Experimental Data
95+
96+
To run the Python program that I used to generate summary statistics,
97+
and generate the graphs for the figures from the paper,
98+
you need Python 3 installed. The source code of the Python programs is
99+
found in the [src/analysis](src/analysis) directory. To run the analysis
100+
execute the following at the root of the repository:
101+
102+
```shell
103+
make analysis
104+
```
105+
106+
If you don't want to overwrite my original data files, and figures, then change the variable
107+
`pathToDataFiles` in the `Makefile` before running the above command.
108+
109+
This will analyze the data from the [/data](data) directory. It will also
110+
generate the figures in that directory, as well as output a few txt files with
111+
summary statistics into that directory. This make command will also take
112+
care of installing any required Python packages if you don't already have them
113+
installed, such as matplotlib.
114+
115+
To convert the `eps` versions of the figures to `pdf`, then after running the above
116+
analysis, run the following (this assumes that you have epstopdf installed):
117+
118+
```shell
119+
make epstopdf
120+
```
121+
122+
## Running the Principle Component Analysis and Fitness Distance Correlation Examples
123+
124+
The above Makefile targets only runs and analyzes the experiments that originated with the
125+
paper for MONE. That paper is an extended version of a conference paper from the
126+
BICT 2019 conference. A couple sections of the MONE paper originated in that BICT 2019
127+
paper, including sections that classified the various distance metrics for permutations
128+
using principal component analysis, along with a few examples of fitness distance
129+
correlation.
130+
131+
To run the code to recreate that, use the following Makefile target:
132+
133+
```shell
134+
make bict2019
135+
```
136+
137+
## Other Files in the Repository
138+
139+
There are a few other files, potentially of interest, in the repository,
140+
which include:
141+
* `system-stats.txt`: This file contains details of the system I
142+
used to run the experiments, such as operating system, processor
143+
specs, Java JDK and VM. It is in the [/data](data) directory.
144+
145+
## License
146+
147+
The code to replicate the experiments from the paper, as well as the
148+
Chips-n-Salsa, JavaPermutationTools, and ρμ libraries are licensed
149+
under the [GNU General Public License 3.0](https://www.gnu.org/licenses/gpl-3.0.en.html).

‎data/system-stats.txt‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
OS: Windows 10
2+
CPU: AMD A10-5700, 3.4 GHz
3+
Memory: 8 GB DDR3
4+
5+
Experiment programs
6+
Compiler: OpenJDK 11.0.8
7+
Target: Java 11
8+
VM: OpenJDK 64-bit Server VM (build 11.0.8+10)
9+
10+
Chips-n-Salsa library
11+
Release: 4.2.1 (from Maven Central)
12+
Compiler: OpenJDK 11 64-bit (compiled via Github Actions just prior to release)
13+
Target: Java 11
14+
15+
JavaPermutationTools library
16+
Release: 3.0.0 (from Maven Central)
17+
Compiler: OpenJDK 11 64-bit (compiled via Github Actions just prior to release)
18+
Target: Java 11
19+
20+
rho-mu library
21+
Release: 1.1.0 (from Maven Central)
22+
Compiler: OpenJDK 11 64-bit (compiled via Github Actions just prior to release)
23+
Target: Java 11
24+
25+

0 commit comments

Comments
 (0)