Skip to content

Commit 6eacdd8

Browse files
rjrudinrjrudin
committed
#446 Now loading from data paths, including via dependencies
1 parent 1138909 commit 6eacdd8

File tree

15 files changed

+299
-30
lines changed

15 files changed

+299
-30
lines changed

‎.gitignore‎

Lines changed: 4 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
/bin
2-
/.classpath
3-
/build
4-
/.gradle
2+
.classpath
3+
build
4+
.gradle
5+
gradle-local.properties
56
/.nb-gradle
67
/.settings
78
/.project
@@ -10,27 +11,10 @@ ml-gradle-all/.gradle/
1011
ml-gradle-all/.classpath
1112
ml-gradle-all/.project
1213
ml-gradle-all/.settings/
13-
sample-project/rob/
1414
sample-project/data/export/
15-
disconnected-project/build/
16-
disconnected-project/.gradle/
1715
sample-project/local.gradle
18-
examples/disconnected-project/build/
19-
examples/disconnected-project/.gradle/
20-
examples/flexrep-project/.gradle/
2116
examples/sample-project/local.gradle
22-
examples/failover-project/.gradle/
23-
examples/mlcp-project/.gradle/
24-
examples/mlcp-project/build/
2517
examples/mlcp-project/data/export/
26-
examples/shell-project/.gradle/
27-
examples/shell-project/build/
28-
examples/shell-project/.gradle
29-
examples/shell-project/.settings
30-
examples/shell-project/.project
31-
examples/shell-project/.classpath
32-
examples/minimal-project/.gradle
33-
examples/ignore-resources-project/.gradle/
3418
.idea
3519
*.iml
3620
out

‎examples/dependency-project/README.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
This set of projects demonstrates:
2+
3+
- How a "provider" project can publish a zip of MarkLogic modules and data
4+
- How an ml-gradle or DHF project can depend on this zip so that the modules and data are automatically included
5+
in an application
6+
7+
## Publishing the provider
8+
9+
To try this out, you'll first need Gradle installed locally.
10+
11+
Then, begin by publishing the zip from the provider project:
12+
13+
cd provider-project
14+
gradle publishToMavenLocal
15+
16+
If you'd like to inspect the zip, you'll find it at ~/.m2/repository/com/marklogic/example-dependency.
17+
18+
## Deploying and verifying the ml-gradle project
19+
20+
Next, deploy the app in the ml-gradle-client-project:
21+
22+
cd ../ml-gradle-client-project
23+
gradle -i mlDeploy
24+
25+
You'll see logging like this that lets you know that the modules and data from the example-dependency zip
26+
will be included when the application is deployed:
27+
28+
```
29+
Found mlRestApi configuration, will extract all of its dependencies to build/mlRestApi
30+
Extracting file: /Users/rrudin/.m2/repository/com/marklogic/example-dependency/1.0.0/example-dependency-1.0.0.jar
31+
[unzip] Expanding: /Users/rrudin/.m2/repository/com/marklogic/example-dependency/1.0.0/example-dependency-1.0.0.jar into /Users/rrudin/dev/workspace/ml-gradle/examples/dependency-project/ml-gradle-client-project/build/mlRestApi
32+
Finished extracting mlRestApi dependencies
33+
Module paths: [/Users/rrudin/dev/workspace/ml-gradle/examples/dependency-project/ml-gradle-client-project/build/mlRestApi/example-dependency/ml-modules, /Users/rrudin/dev/workspace/ml-gradle/examples/dependency-project/ml-gradle-client-project/src/main/ml-modules]
34+
Data paths: [/Users/rrudin/dev/workspace/ml-gradle/examples/dependency-project/ml-gradle-client-project/build/mlRestApi/example-dependency/ml-data, /Users/rrudin/dev/workspace/ml-gradle/examples/dependency-project/ml-gradle-client-project/src/main/ml-data]
35+
:mlPrepareRestApiDependencies (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 0.165 secs.
36+
```
37+
38+
And you'll see logging like this that indicates that the example-dependency modules were loaded:
39+
40+
```
41+
Executing command [com.marklogic.appdeployer.command.modules.LoadModulesCommand] with sort order [400]
42+
Initializing new instance of ModulesLoader
43+
Loading asset modules from dir: /Users/rrudin/dev/workspace/ml-gradle/examples/dependency-project/ml-gradle-client-project/build/mlRestApi/example-dependency/ml-modules
44+
Writing 1 files
45+
Writing: /example.sjs
46+
Writing 1 documents to MarkLogic; port: 8000; database: ml-gradle-client-modules
47+
```
48+
49+
And you'll see logging like this that indicates that the example-dependency data was loaded:
50+
51+
```
52+
Executing command [com.marklogic.appdeployer.command.data.LoadDataCommand] with sort order [1300]
53+
Initializing ExecutorService
54+
Writing 2 files
55+
Writing: /example/data1.json
56+
Writing: /example/data2.json
57+
Shutting down ExecutorService
58+
Writing 2 documents to MarkLogic; port: 8030
59+
```
60+
61+
You can then use qconsole to verify that the following documents were inserted:
62+
63+
- In ml-gradle-client-modules: /example.sjs
64+
- In ml-gradle-client-content: /example/data1.json and /example/data2.json
65+
66+
See [Loading data](https://github.com/marklogic-community/ml-app-deployer/wiki/Loading-data) for more
67+
information on configuring how data is loaded during a deployment.
68+
69+
70+
## Deploying and verifying the DHF project
71+
72+
The DHF project is deployed the same way as the ml-gradle project, though you'll first need to run hubInit to
73+
initialize the project (this is to avoid adding a bunch of DHF files to version control that aren't needed for
74+
the purposes of this example):
75+
76+
cd ../dhf-client-project
77+
gradle -i mlDeploy
78+
79+
You'll see the same logging as shown above for the ml-gradle project. And likewise, you can use qconsole
80+
to verify that the following documents were inserted:
81+
82+
- In dhf-client-MODULES: /example.sjs
83+
- In dhf-client-FINAL: /example/data1.json, /example/data2.json
84+
85+
Note that in a DHF project, the final database is equivalent to the default content database in an ml-gradle
86+
project. Thus, data is loaded by default to the final database. This can be overridden via the
87+
mlDataDatabaseName property.
88+

‎examples/dependency-project/dhf-client-project/.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
src
2+
gradle
3+
.gradle
4+
build
5+
plugins
6+
gradlew
7+
gradlew.bat

‎examples/dependency-project/dhf-client-project/build.gradle‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
buildscript {
2+
repositories {
3+
mavenLocal()
4+
maven {
5+
url "https://plugins.gradle.org/m2/"
6+
}
7+
}
8+
dependencies {
9+
classpath "com.marklogic:ml-gradle:3.13.dev"
10+
classpath "gradle.plugin.com.marklogic:ml-data-hub:4.2.2"
11+
}
12+
}
13+
14+
plugins {
15+
id 'net.saliman.properties' version '1.4.6'
16+
}
17+
18+
apply plugin: "com.marklogic.ml-data-hub"
19+
20+
repositories {
21+
mavenLocal()
22+
}
23+
24+
dependencies {
25+
mlRestApi "com.marklogic:example-dependency:1.0.0"
26+
}

‎examples/dependency-project/dhf-client-project/gradle.properties‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
mlDHFVersion=4.2.2
2+
mlHost=localhost
3+
4+
# Define these in gradle-local.properties
5+
mlUsername=
6+
mlPassword=
7+
8+
mlStagingAppserverName=dhf-client-STAGING
9+
mlStagingPort=8035
10+
mlStagingDbName=dhf-client-STAGING
11+
mlStagingForestsPerHost=1
12+
mlStagingAuth=digest
13+
14+
mlFinalAppserverName=dhf-client-FINAL
15+
mlFinalPort=8036
16+
mlFinalDbName=dhf-client-FINAL
17+
mlFinalForestsPerHost=1
18+
mlFinalAuth=digest
19+
20+
mlJobAppserverName=dhf-client-JOBS
21+
mlJobPort=8037
22+
mlJobDbName=dhf-client-JOBS
23+
mlJobForestsPerHost=4
24+
mlJobAuth=digest
25+
26+
mlModulesDbName=dhf-client-MODULES
27+
mlModulesForestsPerHost=1
28+
29+
mlStagingTriggersDbName=dhf-client-staging-TRIGGERS
30+
mlStagingTriggersForestsPerHost=1
31+
32+
mlStagingSchemasDbName=dhf-client-staging-SCHEMAS
33+
mlStagingSchemasForestsPerHost=1
34+
35+
mlFinalTriggersDbName=dhf-client-final-TRIGGERS
36+
mlFinalTriggersForestsPerHost=1
37+
38+
mlFinalSchemasDbName=dhf-client-final-SCHEMAS
39+
mlFinalSchemasForestsPerHost=1
40+
41+
# The name of the Role to create for Hub Access
42+
mlHubUserRole=dhf-client-role
43+
mlHubUserName=dhf-client-user
44+
# this password is autogenerated for you via the 'gradle hubInit' task
45+
mlHubUserPassword=RiWi{UE-yXLfK'TIo4Z(
46+
47+
# The name of the role to create for hub deployment/development
48+
mlHubAdminRole=hub-admin-role
49+
mlHubAdminUserName=hub-admin-user
50+
mlHubAdminUserPassword=koq$Qd$5Ex$:3grhHX#6
51+
52+
# Default module permissions which allow dhf-client-role to execute flows
53+
mlModulePermissions=rest-reader,read,rest-writer,insert,rest-writer,update,rest-extension-user,execute,dhf-client-role,read,dhf-client-role,execute

‎examples/dependency-project/ml-gradle-client-project/build.gradle‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
buildscript {
2+
repositories {
3+
mavenLocal()
4+
jcenter()
5+
}
6+
dependencies {
7+
classpath "com.marklogic:ml-gradle:3.13.dev"
8+
}
9+
}
10+
11+
plugins {
12+
id "net.saliman.properties" version "1.4.6"
13+
}
14+
15+
apply plugin: "com.marklogic.ml-gradle"
16+
17+
repositories {
18+
mavenLocal()
19+
}
20+
21+
dependencies {
22+
mlRestApi "com.marklogic:example-dependency:1.0.0"
23+
}

‎examples/dependency-project/ml-gradle-client-project/gradle.properties‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
mlAppName=ml-gradle-client
2+
mlRestPort=8030
3+
mlContentForestsPerHost=1
4+
5+
# Define these in gradle-local.properties
6+
mlUsername=
7+
mlPassword=

‎examples/dependency-project/provider-project/build.gradle‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
plugins {
2+
id "maven-publish"
3+
}
4+
5+
group = "com.marklogic"
6+
version = "1.0.0"
7+
8+
configurations {
9+
bundle
10+
}
11+
12+
task bundleJar(type: Jar) {
13+
from("src/main/ml-modules") {
14+
into("example-dependency/ml-modules")
15+
}
16+
from("src/main/ml-data") {
17+
into("example-dependency/ml-data")
18+
}
19+
destinationDir file("build/libs")
20+
baseName "example-dependency"
21+
}
22+
23+
artifacts {
24+
bundle bundleJar
25+
}
26+
27+
publishing {
28+
publications {
29+
mainBundle(MavenPublication) {
30+
artifactId "example-dependency"
31+
artifact bundleJar
32+
}
33+
}
34+
}

‎examples/dependency-project/provider-project/src/main/ml-data/example/data1.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"hello": "world1"
3+
}

‎examples/dependency-project/provider-project/src/main/ml-data/example/data2.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"hello": "world2"
3+
}

0 commit comments

Comments
 (0)