Skip to content

Commit c5f1fd0

Browse files
Dermot SmythDermot Smyth
Dermot Smyth
authored and
Dermot Smyth
committed
added docker examples
1 parent fe35cc9 commit c5f1fd0

39 files changed

+2154
-0
lines changed

‎examples/docker-projects/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Overview
2+
---------
3+
4+
Sample projects using docker-compose with a custom Dockerfile for MarkLogic.
5+
6+
Easiest way to get started using these examples is to clone the ml-gradle repository and copy your desired project to your workspace directory
7+
8+
Unless otherwise specified, the target platform for the docker images is linux. Note that on mac's you can use linux images on the docker native macOS virtualization. However with windows, you need to run a linux vm.

‎examples/docker-projects/single-node-with-nifi/.gitignore‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
2+
# Created by https://www.gitignore.io/api/gradle
3+
4+
### Gradle ###
5+
.gradle
6+
/build/
7+
8+
# Ignore Gradle GUI config
9+
gradle-app.setting
10+
11+
# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored)
12+
!gradle-wrapper.jar
13+
14+
# Cache of project
15+
.gradletasknamecache
16+
17+
# # Work around https://youtrack.jetbrains.com/issue/IDEA-116898
18+
# gradle/wrapper/gradle-wrapper.properties
19+
20+
21+
# End of https://www.gitignore.io/api/gradle

‎examples/docker-projects/single-node-with-nifi/README.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
Overview
2+
---------
3+
4+
NOTE: WIP
5+
6+
Project will create (in docker)
7+
8+
* A single MarkLogic node
9+
* A nifi instance with the MarkLogic nar files
10+
* A local directory (./logs) that will contain the logs of the MarkLogic container and nifi image
11+
* A data mount (/data) in the nifi container that corresponds to the local project directory (./data)
12+
* This is useful when you want to read files from a processor in nifi
13+
14+
Admin account is
15+
* Username: admin
16+
* Password: admin
17+
18+
Prerequisites
19+
-------------
20+
21+
1. docker toolkit installed
22+
2. Java 8 installed
23+
3. Following ports available -
24+
* 8000-8050
25+
* 8080
26+
* 8443
27+
* 10000
28+
29+
Installation steps (once off)
30+
-----------------------------
31+
32+
1. Download MarkLogic-9.0-5.x86_64.rpm (or any other MarkLogic v9 rpm) and copy it to src/main/docker/marklogic
33+
2. Execute (this will download all required docker dependencies to build marklogic image)
34+
```
35+
docker-compose build
36+
```
37+
3. Execute (this will download all required gradle dependencies)
38+
```
39+
./gradlew build
40+
```
41+
___
42+
43+
Docker Setup (including mlDeploy)
44+
-------------
45+
1. ./gradlew mlDockerDeploy
46+
47+
Docker Start
48+
-------------
49+
1. ./gradlew mlDockerStart
50+
51+
Docker Stop
52+
-------------
53+
1. ./gradlew mlDockerStop
54+
55+
Docker TearDown
56+
-------------
57+
1. ./gradlew mlDockerTeardown
58+
59+
60+
___
61+
62+
63+
Customisations
64+
-------------
65+
If you want to include the MarkLogic Converters rpm in the image, you can. You just need to modify the MarkLogic Dockerfile as specified in the comments of the Dockerfile itself

‎examples/docker-projects/single-node-with-nifi/build.gradle‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
plugins {
2+
// gradle -PenvironmentName=x ...
3+
id 'net.saliman.properties' version '1.4.6'
4+
id "com.marklogic.ml-gradle" version "3.8.1"
5+
id 'com.avast.gradle.docker-compose' version "0.6.11"
6+
}
7+
dockerCompose {
8+
waitForTcpPorts = false
9+
}
10+
11+
task mlDockerSetupNode(dependsOn:composeUp) {
12+
doLast {
13+
sleep(5 * 1000) //ensure node is up
14+
exec {
15+
commandLine 'docker', 'exec', 'node1.single', 'init-marklogic'
16+
}
17+
}
18+
}
19+
20+
mlDeploy.shouldRunAfter mlDockerSetupNode
21+
22+
task mlDockerDeploy(dependsOn:[mlDockerSetupNode,mlDeploy])
23+
24+
task mlDockerStart(dependsOn:composeUp)
25+
26+
task mlDockerStop(type:Exec) {
27+
commandLine "docker-compose", "stop"
28+
}
29+
30+
task mlDockerTeardown(dependsOn:composeDown) {
31+
//remove logs directory
32+
delete 'logs'
33+
}

‎examples/docker-projects/single-node-with-nifi/docker-compose.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
version: '3'
2+
services:
3+
node1:
4+
build: "./src/main/docker/marklogic"
5+
image: "marklogic:latest"
6+
ports:
7+
# choose what ports you want to expose (at least 8000 8001 8002)
8+
- "8000-8050:8000-8050"
9+
hostname: "node1.single"
10+
container_name: "node1.single"
11+
volumes:
12+
- ./logs/node1:/var/opt/MarkLogic/Logs
13+
14+
nifi:
15+
build: "./src/main/docker/nifi"
16+
image: "marklogic-nifi:latest"
17+
ports:
18+
- "8080:8080"
19+
- "8443:8443"
20+
- "10000:10000"
21+
hostname: "nifi.single"
22+
container_name: "nifi.single"
23+
volumes:
24+
- ./data:/data
25+
- ./logs/nifi:/opt/nifi/nifi-1.7.0/logs
26+
27+
# optionally specify the subnet
28+
networks:
29+
default:
30+
driver: bridge
31+
ipam:
32+
config:
33+
- subnet: 172.66.0.0/16

‎examples/docker-projects/single-node-with-nifi/gradle.properties‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
mlAppName=sample-single-node-app
2+
3+
mlRestPort=8003

‎examples/docker-projects/single-node-with-nifi/gradle/wrapper/gradle-wrapper.jar‎

53.1 KB
Binary file not shown.

‎examples/docker-projects/single-node-with-nifi/gradle/wrapper/gradle-wrapper.properties‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
distributionBase=GRADLE_USER_HOME
2+
distributionPath=wrapper/dists
3+
distributionUrl=https\://services.gradle.org/distributions/gradle-4.8.1-bin.zip
4+
zipStoreBase=GRADLE_USER_HOME
5+
zipStorePath=wrapper/dists

‎examples/docker-projects/single-node-with-nifi/gradlew‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
#!/usr/bin/env sh
2+
3+
##############################################################################
4+
##
5+
## Gradle start up script for UN*X
6+
##
7+
##############################################################################
8+
9+
# Attempt to set APP_HOME
10+
# Resolve links: $0 may be a link
11+
PRG="$0"
12+
# Need this for relative symlinks.
13+
while [ -h "$PRG" ] ; do
14+
ls=`ls -ld "$PRG"`
15+
link=`expr "$ls" : '.*-> \(.*\)$'`
16+
if expr "$link" : '/.*' > /dev/null; then
17+
PRG="$link"
18+
else
19+
PRG=`dirname "$PRG"`"/$link"
20+
fi
21+
done
22+
SAVED="`pwd`"
23+
cd "`dirname \"$PRG\"`/" >/dev/null
24+
APP_HOME="`pwd -P`"
25+
cd "$SAVED" >/dev/null
26+
27+
APP_NAME="Gradle"
28+
APP_BASE_NAME=`basename "$0"`
29+
30+
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
31+
DEFAULT_JVM_OPTS=""
32+
33+
# Use the maximum available, or set MAX_FD != -1 to use that value.
34+
MAX_FD="maximum"
35+
36+
warn () {
37+
echo "$*"
38+
}
39+
40+
die () {
41+
echo
42+
echo "$*"
43+
echo
44+
exit 1
45+
}
46+
47+
# OS specific support (must be 'true' or 'false').
48+
cygwin=false
49+
msys=false
50+
darwin=false
51+
nonstop=false
52+
case "`uname`" in
53+
CYGWIN* )
54+
cygwin=true
55+
;;
56+
Darwin* )
57+
darwin=true
58+
;;
59+
MINGW* )
60+
msys=true
61+
;;
62+
NONSTOP* )
63+
nonstop=true
64+
;;
65+
esac
66+
67+
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
68+
69+
# Determine the Java command to use to start the JVM.
70+
if [ -n "$JAVA_HOME" ] ; then
71+
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
72+
# IBM's JDK on AIX uses strange locations for the executables
73+
JAVACMD="$JAVA_HOME/jre/sh/java"
74+
else
75+
JAVACMD="$JAVA_HOME/bin/java"
76+
fi
77+
if [ ! -x "$JAVACMD" ] ; then
78+
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
79+
80+
Please set the JAVA_HOME variable in your environment to match the
81+
location of your Java installation."
82+
fi
83+
else
84+
JAVACMD="java"
85+
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
86+
87+
Please set the JAVA_HOME variable in your environment to match the
88+
location of your Java installation."
89+
fi
90+
91+
# Increase the maximum file descriptors if we can.
92+
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
93+
MAX_FD_LIMIT=`ulimit -H -n`
94+
if [ $? -eq 0 ] ; then
95+
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
96+
MAX_FD="$MAX_FD_LIMIT"
97+
fi
98+
ulimit -n $MAX_FD
99+
if [ $? -ne 0 ] ; then
100+
warn "Could not set maximum file descriptor limit: $MAX_FD"
101+
fi
102+
else
103+
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
104+
fi
105+
fi
106+
107+
# For Darwin, add options to specify how the application appears in the dock
108+
if $darwin; then
109+
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
110+
fi
111+
112+
# For Cygwin, switch paths to Windows format before running java
113+
if $cygwin ; then
114+
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
115+
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
116+
JAVACMD=`cygpath --unix "$JAVACMD"`
117+
118+
# We build the pattern for arguments to be converted via cygpath
119+
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
120+
SEP=""
121+
for dir in $ROOTDIRSRAW ; do
122+
ROOTDIRS="$ROOTDIRS$SEP$dir"
123+
SEP="|"
124+
done
125+
OURCYGPATTERN="(^($ROOTDIRS))"
126+
# Add a user-defined pattern to the cygpath arguments
127+
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
128+
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
129+
fi
130+
# Now convert the arguments - kludge to limit ourselves to /bin/sh
131+
i=0
132+
for arg in "$@" ; do
133+
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
134+
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
135+
136+
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
137+
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
138+
else
139+
eval `echo args$i`="\"$arg\""
140+
fi
141+
i=$((i+1))
142+
done
143+
case $i in
144+
(0) set -- ;;
145+
(1) set -- "$args0" ;;
146+
(2) set -- "$args0" "$args1" ;;
147+
(3) set -- "$args0" "$args1" "$args2" ;;
148+
(4) set -- "$args0" "$args1" "$args2" "$args3" ;;
149+
(5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
150+
(6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
151+
(7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
152+
(8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
153+
(9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
154+
esac
155+
fi
156+
157+
# Escape application args
158+
save () {
159+
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
160+
echo " "
161+
}
162+
APP_ARGS=$(save "$@")
163+
164+
# Collect all arguments for the java command, following the shell quoting and substitution rules
165+
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
166+
167+
# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong
168+
if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then
169+
cd "$(dirname "$0")"
170+
fi
171+
172+
exec "$JAVACMD" "$@"

0 commit comments

Comments
 (0)