Skip to content

Commit 104e89f

Browse files
mtesauromtesauro
committed
Added new setup.bash which works for Python 2.x and 3.x
1 parent 3af45cf commit 104e89f

26 files changed

+2825
-0
lines changed

‎setup/README.MD‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
## Documentation on setup.bash
2+
3+
The bash script setup.bash was created to automate installation of Defect Dojo and allow for the following install situations to be automated:
4+
5+
### Supported Installs
6+
7+
* Single Server - simplest DefectDojo install where DefectDojo, Dojo dependencies and 3rd party services are installed on a single server. [default install]
8+
* Dev Install - install for local development where a Single Server install is run with credentials and other passwords set to known values.
9+
* Stand-alone Server - install DefectDojo & Dojo dependencies only where 3rd party services (database) is running on other infrastructure.
10+
* ? Docker Single Server - a Single Server install where DefectDojo, Dojo dependencies and 3rd party services are installed in a single container
11+
* ? Docker Stand-alone - a Stand-alone Server install DefectDojo & Dojo dependencies only are installed in a single container.
12+
13+
Note: Cloning the DefectDojo repo and running ./setup.bash does a single server interactive install. Doing other install methods requires setting configuration values and/or using command-line options.
14+
15+
### TDB install situations
16+
17+
* Docker Dev Install - a dev install that uses docker + a mounted local directory structure to isolate dojo code from the rest of the run-time.
18+
* Fronted Dojo Installs - a install of DefectDojo where a separate HTTP server answers the initial requests for DefectDojo such as using Nginx upstream of DefectDojo
19+
20+
### Assumptions
21+
22+
All installs make these assumption:
23+
24+
* DefectDojo will be run in a virtualenv
25+
* All installs support an interactive and non-interactive install methods
26+
* All installation configuration lives in ./dojo/settings/template-env
27+
* * Running setup.bash without editing template-env assumes a single-server install.
28+
* * Running setup.bash without editing template-env non-interactively assumes a single-server install with MySQL
29+
* Any install configuration variable can be overridden by setting an environmental variable
30+
* One of the following OSes is used as the base for the install
31+
* * Ubuntu Linux - officially supported versions: 16.04 LTS, 18.04 LTS
32+
* * CentOS - officially supported versions: ?
33+
* * Mac OS X - officially supported versions: ?
34+
35+
### Definitions
36+
37+
* DefectDojo - the source code and supporting files for DefectDojo contained in the Github repo at https://github.com/DefectDojo/django-DefectDojo
38+
* Dojo dependencies - any additional software, libraries or services needed to install and run the software in the DefectDojo repo. This includes Django and other pip packages, celery workers, and any binaries required to run DefectDojo such as wkhtmltopdf
39+
* 3rd party services - additional services not maintained by DefectDojo but needed to run DefectDojo - currently a database
40+
41+
### Command-line options
42+
43+
```
44+
./setup.bash --help
45+
Usage: ./setup.bash [OPTION]...
46+
47+
Install DefectDojo in an interactive (default) or non-interactive method
48+
49+
Options:
50+
-h or --help Display this help message and exit with a status code of 0
51+
-n or --non-interactive Run install non-interactivity e.g. for Dockerfiles or automation
52+
53+
Note: No options are required, all are optional
54+
```
55+
56+
### Installer details
57+
58+
setup.bash relies on the following files and directory structure:
59+
60+
```
61+
setup.bash => the main install program
62+
├── scripts
63+
├── common
64+
├── config-vars.sh
65+
├── cmd-args.sh
66+
├── prompt.sh
67+
```
68+
69+
Install configuration is in config-vars.sh contains the following install options and default values:
70+
71+
**Format for this list:** *install option* [default value] - *definition*
72+
73+
* PROMPT [true] - Run the install in interactive mode aka prompt the user for config values
74+
* DB_TYPE [MySQL] - The database type to be used by DefectDojo
75+
* DB_LOCAL [true] - Boolean for if the database is installed locally aka on the same OS as DefectDojo
76+
* DB_EXISTS [false] - Boolean for if the database already exists for DefectDojo to use aka doesn't need to be installed
77+
* DB_NAME [dojodb] - Name of the database created to store DefectDojo data
78+
* DB_USER [dojodbusr] - Database username used to access the DefectDojo database
79+
* DB_PASS [vee0Thoanae1daePooz0ieka] - Default password used only for Dev installs, otherwise a random 24 character password is created at install time
80+
* DB_HOST [localhost] - Database hostname where the DefectDojo database is located
81+
* DB_PORT [3306] - Port database is listening on, default port is for the default database MySQL
82+
* DB_DROP_EXISTING [true] - If the database name already exists in database server for DefectDojo, drop that database if this is true. If false and a database name match occurs, throw an error and exit the installer.
83+
* OS_USER=${OS_USER:-"dojo-srv"}
84+
* OS_PASS=${OS_PASS:-"wahlieboojoKa8aitheibai3"}
85+
* OS_GROUP=${OS_GROUP:-"dojo-srv"}
86+
* INSTALL_ROOT=${INSTALL_ROOT:-"/opt/dojo"}
87+
* DOJO_SOURCE=${DOJO_SOURCE:-"$INSTALL_ROOT/django-DefectDojo"}
88+
* DOJO_FILES=${DOJO_FILES:-"$INSTALL_ROOT/local"}
89+
* MEDIA_ROOT=${MEDIA_ROOT:-"$DOJO_FILES/media"}
90+
* STATIC_ROOT=${STATIC_ROOT:-"$DOJO_FILES/static"}
91+
* ADMIN_USER=${ADMIN_USER:-"admin"}
92+
* ADMIN_PASS=${ADMIN_PASS:-"admin"}
93+
* ADMIN_EMAIL=${ADMIN_EMAIL:-"[email protected]"}
94+
95+
Configuration items for setup.py are in template-env in ./dojo/settings/ and contain
96+
97+
*
98+
99+
### Installers workflow
100+
101+
1. Check for command-line arguments, if none, do an interactive single server install
102+
2. Check for install OS
103+
3. Bootstrap any software needed by the install process
104+
4. Install Dojo dependencies
105+
5. Install 3rd party services
106+
107+
108+
### Installer Bash variables
109+
110+
* SETUP_BASE : The full path to where the setup.bash file is located e.g ./setup if starting from the Dojo repository root
111+
* REPO_BASE : The full path to where the DefectDojo source was cloned usually /opt/dojo/django-DefectDojo
112+
* LIB_PATH : The full path to where the configuration values and libraries are for the DefectDojo installer which is SETUP_BASE + /scripts/common/
113+
* DB_TYPPE : The database type DefectDojo will use - currently either SQLite, MySQL or PostgreSQL
114+
*
115+
116+

‎setup/mysql.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Pip installs needed for MySQL DB support
2+
-r requirements.txt
3+
mysqlclient==1.3.12
4+
#mysqlclient==1.4.1
5+
mysql-connector-python==8.0.14

‎setup/postgresql.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Pip installs needed for PostgreSQL DB support
2+
-r requirements.txt
3+
psycopg2-binary==2.7.5

‎setup/requirements-2.txt‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# requirements.txt for DefectDojo using Python 2.x
2+
asteval==0.9.12
3+
bleach>=3.1.0
4+
bleach-whitelist>=0.0.10
5+
celery==4.2.1
6+
coverage==4.5.1
7+
defusedxml==0.5.0
8+
django_celery_results==1.0.4
9+
django-auditlog==0.4.5
10+
django-custom-field==2.9
11+
django-dbbackup>=3.2.0
12+
django-environ==0.4.5
13+
django-filter==1.0.4
14+
django-imagekit==4.0.2
15+
django-multiselectfield==0.1.8
16+
django-overextends==0.4.3
17+
django-polymorphic==1.2
18+
django-rest-swagger==2.1.2
19+
django-slack==5.11.1
20+
django-tagging==0.4.6
21+
django-taggit-serializer==0.1.7
22+
django-tastypie-swagger==0.1.4
23+
django-tastypie==0.14.0
24+
django-watson==1.5.2
25+
Django==1.11.20
26+
djangorestframework==3.7.7
27+
gunicorn==19.7.1
28+
html2text==2018.1.9
29+
humanize==0.5.1
30+
jira==1.0.13
31+
lxml==4.2.5
32+
Markdown==3.0.1
33+
mysqlclient==1.3.12
34+
pandas>=0.22.0
35+
pdfkit==0.6.1
36+
Pillow==5.0.0 # required by django-imagekit
37+
psycopg2-binary==2.7.5
38+
pycrypto==2.6.1
39+
pygments==2.2.0
40+
python-dateutil==2.6.0
41+
python-nmap==0.6.1
42+
pytz==2017.3
43+
redis==3.2.0
44+
requests>=2.2.1
45+
sqlalchemy # Required by Celery broker transport
46+
supervisor==3.3.3
47+
urllib3==1.24
48+
uWSGI==2.0.18
49+
vobject==0.9.5
50+
whitenoise>=4.0

‎setup/requirements-3.txt‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# requirements.txt for DefectDojo using Python 3.x
2+
celery==4.3.0rc2 # updated from 4.1.1 for django 2.1.5, rc because of a bug in 4.2.1: https://github.com/celery/celery/issues/4849
3+
coverage==4.5.1
4+
defusedxml==0.5.0
5+
Django==2.0.10 # updated from 1.11.16
6+
django-auditlog==0.4.5
7+
git+https://[email protected]/Maffooch/django-custom-field.git
8+
django-filter==1.0.4
9+
django-imagekit==4.0.2
10+
django-multiselectfield==0.1.8
11+
# May not be needed - though ~may~ require some code changes
12+
# See: https://github.com/stephenmcd/django-overextends/issues/36
13+
# and https://code.djangoproject.com/ticket/15053
14+
django-overextends==0.4.3 # Required for extensions # No Django 2.0 Support
15+
django-polymorphic==2.0 # updated from 1.2 for django 2.1.5
16+
django-slack==5.11.1
17+
django-rest-swagger==2.1.2
18+
django-tagging==0.4.6
19+
django-tastypie==0.14.2 # updated from 0.14.0
20+
git+https://[email protected]/Maffooch/django-tastypie-swagger.git
21+
django-watson==1.5.2
22+
django-rest-swagger==2.1.2
23+
djangorestframework==3.9.0
24+
gunicorn==19.7.1
25+
html2text==2018.1.9
26+
humanize==0.5.1
27+
jira==1.0.13
28+
lxml==4.1.1
29+
# TODO mysql-connector-python==8.0.14 # Needs to have the following somewhere : DD_DATABASE_URL=mysql.connector.django://user:[email protected]:3306/dojodb3
30+
pdfkit==0.6.1
31+
Pillow==5.0.0 # required by django-imagekit
32+
# TODO psycopg2-binary==2.7.5
33+
pycrypto==2.6.1
34+
python-dateutil==2.6.0 # downgraded from 2.6.1 to work with django-auditlog 0.4.5
35+
python-nmap==0.6.1
36+
pytz==2018.9
37+
requests>=2.2.1
38+
sqlalchemy==1.2.17 # Required by Celery broker transport
39+
urllib3==1.22
40+
vobject==0.9.5
41+
asteval==0.9.12
42+
Markdown==3.0.1
43+
pandas>=0.22.0
44+
django-dbbackup>=3.2.0
45+
django-markdownx>=2.0.23
46+
django-environ==0.4.5
47+
django-taggit-serializer==0.1.7
48+
whitenoise>=4.0
49+
django-environ==0.4.5
50+
pygments==2.2.0

‎setup/scripts/common/cmd-args.sh‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# DefectDojo install 'library' to handle command-line arguments
2+
#
3+
4+
function help() {
5+
echo "Usage: $0 [OPTION]..."
6+
echo ""
7+
echo "Install DefectDojo in an interactive (default) or non-interactive method"
8+
echo ""
9+
echo "Options:"
10+
echo " -h or --help Display this help message and exit with a status code of 0"
11+
echo " -n or --non-interactive Run install non-interactivity e.g. for Dockerfiles or automation"
12+
echo ""
13+
echo "Note: No options are required, all are optional"
14+
}
15+
16+
function welcome_msg() {
17+
echo ""
18+
echo " ____ ____ __ ____ _ "
19+
echo " / __ \___ / __/__ _____/ /_ / __ \____ (_)___ "
20+
echo " / / / / _ \/ /_/ _ \/ ___/ __/ / / / / __ \ / / __ \ "
21+
echo " / /_/ / __/ __/ __/ /__/ /_ / /_/ / /_/ / / / /_/ / "
22+
echo " /_____/\___/_/ \___/\___/\__/ /_____/\____/_/ /\____/ "
23+
echo " /___/ "
24+
echo ""
25+
echo " Welcome to DefectDojo! This is a quick script to get you up and running."
26+
echo " For more info on how ${0##*/} does an install, see:"
27+
echo " https://github.com/DefectDojo/django-DefectDojo/tree/master/setup"
28+
echo ""
29+
}
30+
31+
function read_cmd_args() {
32+
# Check the arguments sent to setup.bash
33+
# from: https://stackoverflow.com/questions/192249/how-do-i-parse-command-line-arguments-in-bash Method #1
34+
for i in ${BASH_ARGV[*]}
35+
do
36+
case $i in
37+
-h|--help)
38+
help
39+
exit 0
40+
;;
41+
-n|--non-interactive)
42+
PROMPT=false
43+
;;
44+
esac
45+
done
46+
47+
welcome_msg
48+
}

0 commit comments

Comments
 (0)