diff --git a/README.mdown b/README.mdown index 5b2e43d3..579bde60 100644 --- a/README.mdown +++ b/README.mdown @@ -23,8 +23,6 @@ To run this generator, you need: - [Bower](http://bower.io/): A package manager for front-end libraries (`npm install -g bower`) - [Git](https://git-scm.com/) - Roxy depends on this version control system -- [Ruby v1.9.3+](https://www.ruby-lang.org/en/documentation/installation/) - Roxy - depends on Ruby in order to run server configuration scripts Note: the `node` command has been renamed to `nodejs`. Some dependencies still point to `node`, which is out of our control. You may need to manually alias those commands, or install the `node-legacy` package. @@ -62,8 +60,8 @@ On Mac or Linux: npm install bower install gulp init-local - ./ml local install - ./ml local mlcp -options_file import-sample-data.options + ./gradlew mlDeploy + ./gradlew importSampleData On Windows: @@ -71,8 +69,8 @@ On Windows: npm install bower install gulp init-local - ml.bat local install - ml.bat local mlcp -options_file import-sample-data.options + gradlew.bat mlDeploy + gradlew.bat importSampleData ## Prepare your Application @@ -97,9 +95,9 @@ those files don't exist, then use the following command to create them: Note: you can use `gulp init-dev` and `gulp init-prod` to setup properties for the dev and prod environments. -Note: the deploy properties allow tweaking the MarkLogic side of the application -in great detail. Look inside `deploy/build.properties` or the documentation of -the [Roxy deployer](https://github.com/marklogic/roxy) to learn more about this. +Note: ml-gradle supports tweaking the MarkLogic side of the application +in great detail. Look at the documentation of ml-gradle +(https://github.com/marklogic-community/ml-gradle) to learn more about this. Note: consider editing the `_loginMode` variable in `ui/app/login/login.service.js` to change the login mode of the application. The default is set to `full`. The @@ -107,19 +105,15 @@ other options are `top-right` or `modal`. ## Deploy your Application -Run the following Roxy commands to deploy the application to the chosen MarkLogic +Run the following ml-gradle commands to deploy the application to the chosen MarkLogic installation. It will create and configure databases, REST servers, users, and roles for you, and deploy the back-end application code. - ./ml local bootstrap - ./ml local deploy modules - ./ml local deploy content + ./gradlew mlDeploy -PenvironmentName=[local|dev|prod] Or on Windows: - ml.bat local bootstrap - ml.bat local deploy modules - ml.bat local deploy content + gradlew.bat mlDeploy -PenvironmentName=[local|dev|prod] ## Launch your Application @@ -148,29 +142,13 @@ and no browser tab will be opened automatically. In addition you have `--verbose The application comes with 3000 JSON documents generated by json-generator.com. They will allow you to explore all the features you get out of the box in a better way. You can load them with (MLCP)[https://docs.marklogic.com/guide/ingestion/content-pump] using -Roxy. +ml-gradle. -Before you hit off, check if Roxy can find MLCP: - - ./ml local mlcp - -Roxy will print the following message if it cannot find MLCP: - - ERROR: MLCP not found or mis-configured, please check the mlcp-home setting. - -Roxy looks for `/usr/local/mlcp/` (or `\usr\local\mlcp\`) by default. It can be -convenient to just install MLCP there, or create a symlink from `/usr/local/mlcp` -to where MLCP is installed. Alternatively, edit your `deploy/local.properties`, and -append a `mlcp-home=/path/to/your/mlcp/` to the end of it. Repeating above command -should show MLCP usage, not an ERROR. - -Once MLCP works correctly, you can run: - - ./ml local mlcp -options_file import-sample-data.options + ./gradlew importSampleData Or on Windows: - ml.bat local mlcp -options_file import-sample-data.options + gradlew.bat importSampleData Note: the detail controller, the part that handles showing your data, can not only handle JSON, but also XML, Binary, and Text data out of the box. diff --git a/WINDOWS.mdown b/WINDOWS.mdown index e0fd38b0..70b702b1 100644 --- a/WINDOWS.mdown +++ b/WINDOWS.mdown @@ -13,10 +13,3 @@ Some required modules do not have native Windows modules available and require a If your installation of node and npm required elevated priviledges, you will need to use an elevated command prompt to upgrade npm and perform all global `npm install -g` commands. When scaffolding your project using slush, a normal command prompt is sufficient. -## Configuring MLCP - -The default location for MLCP is `/usr/local/mlcp`. This path is unlikely the appropriate location of your MLCP install. To specify the correct location of MLCP in Windows, edit/create `deploy\local.properties` and add -``` -mlcp-home=C:\\path\\to\\mlcp -``` -Please note that the backslash is being escaped by a second backslash. This inserts a literal backslash instead of inserting an escape character in Java. diff --git a/app/templates/INSTALL.mdown b/app/templates/INSTALL.mdown index e2661725..5eb82488 100644 --- a/app/templates/INSTALL.mdown +++ b/app/templates/INSTALL.mdown @@ -81,23 +81,14 @@ If dev or prod configuration does not exist yet, run: Provide settings as needed. After that you follow the same steps as for deploying to your local MarkLogic, but with a different environment parameter: - ./ml dev bootstrap - ./ml dev clean content - ./ml dev deploy modules - ./ml dev deploy content - ./ml dev mlcp -options_file import-sample-data.options - -(Or ./ml prod .. depending on need. Check README.mdown to see if the deployment steps have been altered for this project.) + ./gradlew mlDeploy -PenvironmentName=[environment] + ./gradlew importSampleData -PenvironmentName=[environment] ## Deploying updates Deploying updates is a matter of repeating part of the initial deployment. For fully updating the back-end you repeat: - ./ml dev bootstrap - ./ml dev clean content (Be careful with cleaning content on production!) - ./ml dev deploy modules - ./ml dev deploy content - ./ml dev mlcp -options_file import-sample-data.options + ./gradlew mlDeploy -PenvironmentName=[environment] The middle-tier service is started on the server, and will not have to be stopped or restarted when deploying updates. pm2 will automatically restart the app if any of it's files are changed so all you need to do from your local machine is: diff --git a/app/templates/README.mdown b/app/templates/README.mdown index 240af5cc..b49845f7 100644 --- a/app/templates/README.mdown +++ b/app/templates/README.mdown @@ -6,7 +6,7 @@ generator, and uses the following components: - [AngularJS](https://angularjs.org/) - [Gulp](http://gulpjs.com/) - [node.js](http://nodejs.org/): very thin layer, hosting the Angular code and proxying MarkLogic REST API requests -- [Roxy Deployer](https://github.com/marklogic/roxy): bootstrap MarkLogic databases, application servers, etc; scaffolding for MarkLogic REST API service extensions +- [ml-gradle](https://github.com/marklogic-community/ml-gradle): configure MarkLogic databases, app-servers, users & roles. Deploy rest extensions and modules. ## Install Required Dependencies @@ -17,8 +17,6 @@ To deploy and run the application you need: - [gulp](http://gulpjs.com/): Javascript task automation (`npm install -g gulp`) - [Bower](http://bower.io/): A package manager for front-end libraries (`npm install -g bower`) - [Git](https://git-scm.com/) - Roxy depends on this version control system -- [Ruby v1.9.3+](https://www.ruby-lang.org/en/documentation/installation/) - Roxy - depends on Ruby in order to run server configuration scripts Note: the `node` command has been renamed to `nodejs`. Some dependencies still point to `node`, which is out of our control. You may need to manually alias those commands, or install the `node-legacy` package. @@ -32,16 +30,16 @@ On Mac or Linux: npm install bower install gulp init-local - ./ml local install - ./ml local mlcp -options_file import-sample-data.options + ./gradlew mlDeploy + ./gradlew importSampleData On Windows: npm install bower install gulp init-local - ml.bat local install - ml.bat local mlcp -options_file import-sample-data.options + gradlew.bat mlDeploy + gradlew.bat importSampleData ## Prepare your Application @@ -76,19 +74,15 @@ other options are `top-right` or `modal`. ## Deploy your Application -Run the following Roxy commands to deploy the application to the chosen MarkLogic +Run the following ml-gradle commands to deploy the application to the chosen MarkLogic installation. It will create and configure databases, REST servers, users, and roles for you, and deploy the back-end application code. - ./ml local bootstrap - ./ml local deploy modules - ./ml local deploy content + ./gradlew mlDeploy -PenvironmentName=[local|dev|prod] Or on Windows: - ml.bat local bootstrap - ml.bat local deploy modules - ml.bat local deploy content + gradlew.bat mlDeploy -PenvironmentName=[local|dev|prod] ## Launch your Application @@ -117,29 +111,13 @@ and no browser tab will be opened automatically. In addition you have `--verbose The application comes with 3000 JSON documents generated by json-generator.com. They will allow you to explore all the features you get out of the box in a better way. You can load them with (MLCP)[https://docs.marklogic.com/guide/ingestion/content-pump] using -Roxy. +ml-gradle. -Before you hit off, check if Roxy can find MLCP: - - ./ml local mlcp - -Roxy will print the following message if it cannot find MLCP: - - ERROR: MLCP not found or mis-configured, please check the mlcp-home setting. - -Roxy looks for `/usr/local/mlcp/` (or `\usr\local\mlcp\`) by default. It can be -convenient to just install MLCP there, or create a symlink from `/usr/local/mlcp` -to where MLCP is installed. Alternatively, edit your `deploy/local.properties`, and -append a `mlcp-home=/path/to/your/mlcp/` to the end of it. Repeating above command -should show MLCP usage, not an ERROR. - -Once MLCP works correctly, you can run: - - ./ml local mlcp -options_file import-sample-data.options + ./gradlew importSampleData Or on Windows: - ml.bat local mlcp -options_file import-sample-data.options + gradlew.bat importSampleData Note: the detail controller, the part that handles showing your data, can not only handle JSON, but also XML, Binary, and Text data out of the box. diff --git a/app/templates/WINDOWS.mdown b/app/templates/WINDOWS.mdown index e0fd38b0..70b702b1 100644 --- a/app/templates/WINDOWS.mdown +++ b/app/templates/WINDOWS.mdown @@ -13,10 +13,3 @@ Some required modules do not have native Windows modules available and require a If your installation of node and npm required elevated priviledges, you will need to use an elevated command prompt to upgrade npm and perform all global `npm install -g` commands. When scaffolding your project using slush, a normal command prompt is sufficient. -## Configuring MLCP - -The default location for MLCP is `/usr/local/mlcp`. This path is unlikely the appropriate location of your MLCP install. To specify the correct location of MLCP in Windows, edit/create `deploy\local.properties` and add -``` -mlcp-home=C:\\path\\to\\mlcp -``` -Please note that the backslash is being escaped by a second backslash. This inserts a literal backslash instead of inserting an escape character in Java. diff --git a/app/templates/build.gradle b/app/templates/build.gradle new file mode 100644 index 00000000..847fd3fd --- /dev/null +++ b/app/templates/build.gradle @@ -0,0 +1,38 @@ +/* + * The most minimal ml-gradle file just has one thing - the ml-gradle plugin declaration. + * + * Without a gradle.properties file with properties such as mlAppName and mlRestPort, ml-gradle will use some sensible + * defaults to generate a new application - an app name of my-app, port 8003, and a username/password combo of + * admin/admin. + */ + +plugins { + id "net.saliman.properties" version "1.4.6" + id "com.marklogic.ml-gradle" version "3.6.0" +} + +repositories { + jcenter() + maven { url "https://developer.marklogic.com/maven2/" } +} + +configurations { + mlcp +} + +dependencies { + mlcp "com.marklogic:mlcp:9.0.4" + mlcp files("lib") +} + +task importSampleData(type: com.marklogic.gradle.task.MlcpTask) { + classpath = configurations.mlcp + command = "IMPORT" + database = mlAppConfig.contentDatabaseName + logOutputUri = "/mlcp-log.txt" + input_file_path = "sample-data.zip" + input_compressed = "true" + output_collections = "data,data/people" + output_permissions = "slush-demo-role,read,slush-demo-role,update" + output_uri_replace = ".*.zip,''" +} diff --git a/app/templates/data/api/users/admin.json b/app/templates/data/api/users/admin.json deleted file mode 100644 index 9e26dfee..00000000 --- a/app/templates/data/api/users/admin.json +++ /dev/null @@ -1 +0,0 @@ -{} \ No newline at end of file diff --git a/app/templates/deploy/app_specific.rb b/app/templates/deploy/app_specific.rb deleted file mode 100644 index 977598cc..00000000 --- a/app/templates/deploy/app_specific.rb +++ /dev/null @@ -1,229 +0,0 @@ -# -# Put your custom functions in this class in order to keep the files under lib untainted -# -# This class has access to all of the private variables in deploy/lib/server_config.rb -# -# any public method you create here can be called from the command line. See -# the examples below for more information. -# -class ServerConfig - - # - # You can easily "override" existing methods with your own implementations. - # In ruby this is called monkey patching - # - # first you would rename the original method - # alias_method :original_deploy_modules, :deploy_modules - - # then you would define your new method - # def deploy_modules - # # do your stuff here - # # ... - - # # you can optionally call the original - # original_deploy_modules - # end - - # - # you can define your own methods and call them from the command line - # just like other roxy commands - # ml local my_custom_method - # - # def my_custom_method() - # # since we are monkey patching we have access to the private methods - # # in ServerConfig - # @logger.info(@properties["ml.content-db"]) - # end - - # - # to create a method that doesn't require an environment (local, prod, etc) - # you woudl define a class method - # ml my_static_method - # - # def self.my_static_method() - # # This method is static and thus cannot access private variables - # # but it can be called without an environment - # end - - # Show-casing some useful overrides, as well as adjusting some module doc permissions - alias_method :original_deploy_modules, :deploy_modules - alias_method :original_deploy_rest, :deploy_rest - alias_method :original_deploy, :deploy - alias_method :original_clean, :clean - - # Integrate deploy_packages into the Roxy deploy command - def deploy - what = ARGV.shift - - case what - when 'packages' - deploy_packages - else - ARGV.unshift what - original_deploy - end - end - - def deploy_modules - # Uncomment deploy_packages if you would like to use MLPM to deploy MLPM packages, and - # include MLPM deploy in deploy modules to make sure MLPM depencencies are loaded first. - - # Note: you can also move mlpm.json into src/ext/ and deploy plain modules (not REST extensions) that way. - - #deploy_packages - original_deploy_modules - end - - def deploy_packages - password_prompt - system %Q!mlpm deploy -u #{ @ml_username } \ - -p #{ @ml_password } \ - -H #{ @properties['ml.server'] } \ - -P #{ @properties['ml.app-port'] }! - change_permissions(@properties["ml.modules-db"]) - end - - def deploy_rest - original_deploy_rest - change_permissions(@properties["ml.modules-db"]) - end - - # Permissions need to be changed for executable code that was not deployed via Roxy directly, - # to make sure users with app-role can read and execute it. Typically applies to artifacts - # installed via REST api, which only applies permissions for rest roles. Effectively also includes - # MLPM, which uses REST api for deployment. It often also applies to artifacts installed with - # custom code (via app_specific for instance), like alerts. - def change_permissions(where) - logger.info "Changing permissions in #{where} for:" - r = execute_query( - %Q{ - xquery version "1.0-ml"; - - let $new-permissions := ( - xdmp:permission("#{@properties["ml.app-role"]}", "read"), - xdmp:permission("#{@properties["ml.app-role"]}", "update"), - xdmp:permission("#{@properties["ml.app-role"]}", "execute") - ) - - let $uris := - if (fn:contains(xdmp:database-name(xdmp:database()), "content")) then - - (: This is to make sure all alert files are accessible :) - cts:uri-match("*alert*") - - else - - (: This is to make sure all triggers, schemas, modules and REST extensions are accessible :) - cts:uris() - - let $fixes := - for $uri in $uris - let $existing-permissions := xdmp:document-get-permissions($uri) - - (: Only apply new permissions if really necessary (gives better logging too):) - where not(ends-with($uri, "/")) - and count($existing-permissions[fn:string(.) = $new-permissions/fn:string(.)]) ne 3 - - return ( - " " || $uri, - xdmp:document-set-permissions($uri, $new-permissions) - ) - return - if ($fixes) then - $fixes - else - " no changes needed.." - }, - { :db_name => where } - ) - r.body = parse_body r.body - logger.info r.body - logger.info "" - end - - # Integrate clean_collections into the Roxy clean command - def clean - what = ARGV.shift - - case what - when 'collections' - clean_collections - else - ARGV.unshift what - original_clean - end - end - - def clean_collections() - what = ARGV.shift - r = execute_query( - %Q{ - xquery version "1.0-ml"; - - for $collection in fn:tokenize("#{what}", ",") - where fn:exists(fn:collection($collection)[1]) - return ( - xdmp:collection-delete($collection), - "Cleaned collection " || $collection - ) - }, - { :db_name => @properties["ml.content-db"]} - ) - r.body = parse_body r.body - logger.info r.body - end - -end - -# -# Uncomment, and adjust below code to get help about your app_specific -# commands included into Roxy help. (ml -h) -# - -class Help -# def self.app_specific -# <<-DOC.strip_heredoc -# -# App-specific commands: -# example Installs app-specific alerting -# DOC -# end -# -# def self.example -# <<-DOC.strip_heredoc -# Usage: ml {env} example [args] [options] -# -# Runs a special example task against given environment. -# -# Arguments: -# this Do this -# that Do that -# -# Options: -# --whatever=value -# DOC -# end - class < \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi +done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null + +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS="" + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD="maximum" + +warn ( ) { + echo "$*" +} + +die ( ) { + echo + echo "$*" + echo + exit 1 +} + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; + NONSTOP* ) + nonstop=true + ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD="java" + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi +fi + +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi + +# For Cygwin, switch paths to Windows format before running java +if $cygwin ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi + # Now convert the arguments - kludge to limit ourselves to /bin/sh + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" + fi + i=$((i+1)) + done + case $i in + (0) set -- ;; + (1) set -- "$args0" ;; + (2) set -- "$args0" "$args1" ;; + (3) set -- "$args0" "$args1" "$args2" ;; + (4) set -- "$args0" "$args1" "$args2" "$args3" ;; + (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac +fi + +# Escape application args +save ( ) { + for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done + echo " " +} +APP_ARGS=$(save "$@") + +# Collect all arguments for the java command, following the shell quoting and substitution rules +eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" + +# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong +if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then + cd "$(dirname "$0")" +fi + +exec "$JAVACMD" "$@" diff --git a/app/templates/gradlew.bat b/app/templates/gradlew.bat new file mode 100644 index 00000000..f9553162 --- /dev/null +++ b/app/templates/gradlew.bat @@ -0,0 +1,84 @@ +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS= + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto init + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto init + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:init +@rem Get command-line arguments, handling Windows variants + +if not "%OS%" == "Windows_NT" goto win9xME_args + +:win9xME_args +@rem Slurp the command line arguments. +set CMD_LINE_ARGS= +set _SKIP=2 + +:win9xME_args_slurp +if "x%~1" == "x" goto execute + +set CMD_LINE_ARGS=%* + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/app/templates/gulpfile.js b/app/templates/gulpfile.js index 9478b5fd..1ef59d62 100644 --- a/app/templates/gulpfile.js +++ b/app/templates/gulpfile.js @@ -64,8 +64,8 @@ gulp.task('check-config', function(done) { log($.util.colors.red('WARN: ' + env + '.json is missing!')); missingJsonEnv = true; } - if (env && ! fs.existsSync('deploy/' + env + '.properties')) { - log($.util.colors.red('WARN: deploy/' + env + '.properties is missing!')); + if (env && ! fs.existsSync('gradle-' + env + '.properties')) { + log($.util.colors.red('WARN: gradle-' + env + '.properties is missing!')); missingPropsEnv = true; } if (! fs.existsSync('bower_components')) { @@ -309,9 +309,9 @@ gulp.task('init-prod', function(done) { gulp.task('add-deploy-target', function(done) { log('Update ecosystem.json targets or create new ones!'); - var properties = fs.readFileSync('deploy/build.properties', encoding); + var properties = fs.readFileSync('gradle.properties', encoding); - var name = properties.match(/app-name=(.*)/)[1]; + var name = properties.match(/mlAppName=(.*)/)[1]; var gitUrl = 'https://github.com/'; var folderPath = '/space/projects/' + name; var osHomedir = require('os-homedir'); @@ -770,10 +770,10 @@ function clean(files) { function init(env, done) { if (fs.existsSync(env + '.json')) { log('NOTE: ' + env + '.json already exists, change manually if needed.'); - if (fs.existsSync('deploy/' + env + '.properties')) { - log('NOTE: deploy/' + env + '.properties already exists, change manually too.'); + if (fs.existsSync('gradle-' + env + '.properties')) { + log('NOTE: gradle-' + env + '.properties already exists, change manually too.'); } else { - log('WARN: deploy/' + env + '.properties is missing!'); + log('WARN: gradle-' + env + '.properties is missing!'); } done(); } else { @@ -781,45 +781,35 @@ function init(env, done) { var inquirer = require('inquirer'); var ignoreProps = false; - if (!fs.existsSync('deploy/' + env + '.properties')) { - fs.writeFileSync('deploy/' + env + '.properties', env + '-server=localhost', 'utf8'); + if (!fs.existsSync('gradle-' + env + '.properties')) { + fs.writeFileSync('gradle-' + env + '.properties', env + '-server=localhost', 'utf8'); ignoreProps = true; } - run('./ml', [env, 'info', '--format=json']).then(function(output) { + // run('./ml', [env, 'info', '--format=json']).then(function(output) { var localJson = fs.existsSync('local.json') ? JSON.parse(fs.readFileSync('local.json', 'utf8')) : {}; var localAppName = localJson['app-name']; - var localMlVersion = localJson['ml-version']; var localMlHost = localJson['ml-host']; var localMlAdminUser = localJson['ml-admin-user']; var localMlAppUser = localJson['ml-app-user']; var localMlAppPass = localJson['ml-app-pass']; var localMlHttpPort = localJson['ml-http-port']; - var localMlXccPort = localJson['ml-xcc-port']; var localNodePort = localJson['node-port'] || 9070; var localGuestAccess = ['false', 'true'].indexOf(localJson['guest-access']); var localDisallowUpdates = ['false', 'true'].indexOf(localJson['disallow-updates']); var localAppUsersOnly = ['false', 'true'].indexOf(localJson['appusers-only']); - var properties = JSON.parse(output).properties || {}; - - var mlVersion = ['9', '8', '7'].indexOf(localMlVersion || properties['ml.server-version'] || '9'); + //var properties = JSON.parse(output).properties || {}; + var properties = {}; var marklogicHost = properties['ml.' + env + '-server'] || localMlHost || 'localhost'; var marklogicAdminUser = properties['ml.user'] || localMlAdminUser || 'admin'; var appName = properties['ml.app-name'] || localAppName; var appUserName = properties['ml.default-user'] || localMlAppUser; - var appUserPass = unescape(properties['ml.appuser-password']) || localMlAppPass; + var appUserPass = properties['ml.appuser-password'] || localMlAppPass; var appPort = localMlHttpPort || properties['ml.app-port'] || 8040; - var xccPort = localMlXccPort || properties['ml.xcc-port'] || 8041; var prompts = [{ - type: 'list', - name: 'mlVersion', - message: 'MarkLogic version?', - choices: ['9', '8', '7'], - default: mlVersion > 0 ? mlVersion : 0 - }, { type: 'input', name: 'marklogicHost', message: 'MarkLogic Host?', @@ -840,14 +830,6 @@ function init(env, done) { name: 'appPort', message: 'MarkLogic App/Rest port?', default: appPort - }, { - type: 'input', - name: 'xccPort', - message: 'XCC port?', - default: xccPort, - when: function(answers) { - return answers.mlVersion < 8; - } }, { type: 'input', name: 'nodePort', @@ -901,10 +883,6 @@ function init(env, done) { configJSON['ml-app-pass'] = appUserPass || ''; configJSON['ml-http-port'] = settings.appPort; - if (settings.mlVersion < 8) { - configJSON['ml-xcc-port'] = settings.xccPort; - } - configJSON['node-port'] = settings.nodePort; configJSON['guest-access'] = settings.guestAccess; configJSON['disallow-updates'] = settings.disallowUpdates; @@ -914,27 +892,20 @@ function init(env, done) { fs.writeFileSync(env + '.json', configString, encoding); log('Created ' + env + '.json.'); - if (!ignoreProps && fs.existsSync('deploy/' + env + '.properties')) { - log('NOTE: deploy/' + env + '.properties already exists, change manually please!'); + if (!ignoreProps && fs.existsSync('gradle-' + env + '.properties')) { + log('NOTE: gradle-' + env + '.properties already exists, change manually please!'); } else { var envProperties = '#################################################################\n' + - '# This file contains overrides to values in build.properties\n' + + '# This file contains overrides to values in gradle.properties\n' + '# These only affect your local environment and should not be checked in\n' + '#################################################################\n' + '\n' + - 'server-version=' + settings.mlVersion + '\n' + + 'mlAppName=' + settings.nameDashed + '\n' + '\n' + '#\n' + '# The ports used by your application\n' + '#\n' + - 'app-port=' + settings.appPort + '\n'; - if (settings.mlVersion < 8) { - envProperties += 'xcc-port=' + settings.xccPort + '\n'; - } else { - envProperties += '# Taking advantage of not needing a XCC Port for ML8\n' + - 'xcc-port=${app-port}\n' + - 'install-xcc=false\n'; - } + 'mlRestPort=' + settings.appPort + '\n'; envProperties += '\n' + '#\n' + @@ -942,17 +913,17 @@ function init(env, done) { '# WARNING: if you are running these scripts on WINDOWS you may need to change localhost to 127.0.0.1\n' + '# There have been reported issues with dns resolution when localhost wasn\'t in the hosts file.\n' + '#\n' + - env + '-server=' + settings.marklogicHost + '\n' + - 'content-forests-per-host=3\n' + + 'mlHost=' + settings.marklogicHost + '\n' + + 'mlContentForestsPerHost=3\n' + '\n' + '#\n' + '# Admin username/password that will exist on the local/dev/prod servers\n' + '#\n' + - 'user=' + settings.marklogicAdminUser + '\n' + - 'password=' + settings.marklogicAdminPass + '\n'; + 'mlUsername=' + settings.marklogicAdminUser + '\n' + + 'mlPassword=' + settings.marklogicAdminPass + '\n'; - fs.writeFileSync('deploy/' + env + '.properties', envProperties, encoding); - log('Created deploy/' + env + '.properties.'); + fs.writeFileSync('gradle-' + env + '.properties', envProperties, encoding); + log('Created gradle-' + env + '.properties.'); } done(); } catch (e) { @@ -960,13 +931,9 @@ function init(env, done) { done(); } }); - }); + } } -} -// bypass Roxy bug that causes special XML chars to get escaped as entities -function unescape(s) { - return s.replace(''', '\'').replace('"', '"').replace('<', '<').replace('>', '>').replace('&', '&').replace('{{', '{').replace('}}', '}'); -} +//} /** * Inject files in a sorted sequence at a specified inject label diff --git a/app/templates/src/main/ml-config/README.md b/app/templates/src/main/ml-config/README.md new file mode 100644 index 00000000..2599d625 --- /dev/null +++ b/app/templates/src/main/ml-config/README.md @@ -0,0 +1,11 @@ +The rest-api.json file in the root of the ml-config directory is used for the initial call to +[the Client REST API](http://docs.marklogic.com/REST/POST/v1/rest-apis) to create a REST API server. This API is +separate from the Management REST API, and it only allows for a handful of configuration properties. But it's a very +handy API to use, as it automatically creates content and modules databases to with the REST API server. + +You don't need to include a rest-api.json file in this directory; if you don't, some sensible defaults will be used +by ml-gradle. + +Note that in order to "fully" configure the REST API server - e.g. modifying its authentication strategy or its +rewriter - you will instead use ml-config/servers/rest-api-server.json, which is instead processed via the +[endpoint for creating and updating servers](http://docs.marklogic.com/REST/PUT/manage/v2/servers/[id-or-name]/properties). diff --git a/app/templates/src/main/ml-config/databases/README.md b/app/templates/src/main/ml-config/databases/README.md new file mode 100644 index 00000000..1d3f3db3 --- /dev/null +++ b/app/templates/src/main/ml-config/databases/README.md @@ -0,0 +1,5 @@ +See [the MarkLogic docs](http://docs.marklogic.com/REST/POST/manage/v2/databases) for what a database JSON/XML file can +contain. + +As shown in this directory, you can have multiple files for configuring your content database. See the build.gradle +file to see how more-content-database-config.json is added to the list of content database config files. diff --git a/app/templates/src/main/ml-config/databases/content-database.json b/app/templates/src/main/ml-config/databases/content-database.json new file mode 100644 index 00000000..16e59c3d --- /dev/null +++ b/app/templates/src/main/ml-config/databases/content-database.json @@ -0,0 +1,67 @@ +{ + "database-name": "%%DATABASE%%", + "maintain-last-modified": true, + "range-element-index": [ + { + "invalid-values": "reject", + "collation": "", + "localname": "testDateTime", + "namespace-uri": "http://marklogic.com/test", + "range-value-positions": false, + "scalar-type": "dateTime" + }, + { + "invalid-values": "reject", + "collation": "", + "localname": "secondTestDateTime", + "namespace-uri": "http://marklogic.com/test", + "range-value-positions": false, + "scalar-type": "dateTime" + } + ], + "range-path-index": [ + { + "invalid-values": "reject", + "collation": "http://marklogic.com/collation/codepoint", + "path-expression": "docFormat", + "range-value-positions": false, + "scalar-type": "string" + }, + { + "invalid-values": "reject", + "collation": "http://marklogic.com/collation/codepoint", + "path-expression": "eyeColor", + "range-value-positions": false, + "scalar-type": "string" + }, + { + "invalid-values": "reject", + "collation": "http://marklogic.com/collation/codepoint", + "path-expression": "gender", + "range-value-positions": false, + "scalar-type": "string" + }, + { + "invalid-values": "reject", + "collation": "", + "path-expression": "age", + "range-value-positions": false, + "scalar-type": "unsignedInt" + } + ], + "geospatial-element-pair-index": [ + { + "parent-namespace-uri": "", + "parent-localname": "location", + "latitude-namespace-uri": "", + "latitude-localname": "latitude", + "longitude-namespace-uri": "", + "longitude-localname": "longitude", + "coordinate-system": "wgs84", + "range-value-positions": false, + "invalid-values": "reject" + } + ], + "schema-database": "%%SCHEMAS_DATABASE%%", + "triggers-database": "%%TRIGGERS_DATABASE%%" +} diff --git a/app/templates/src/main/ml-config/databases/schemas-database.json b/app/templates/src/main/ml-config/databases/schemas-database.json new file mode 100644 index 00000000..f0f14c74 --- /dev/null +++ b/app/templates/src/main/ml-config/databases/schemas-database.json @@ -0,0 +1,3 @@ +{ + "database-name": "%%SCHEMAS_DATABASE%%" +} \ No newline at end of file diff --git a/app/templates/src/main/ml-config/databases/test-database.json b/app/templates/src/main/ml-config/databases/test-database.json new file mode 100644 index 00000000..215412d9 --- /dev/null +++ b/app/templates/src/main/ml-config/databases/test-database.json @@ -0,0 +1,67 @@ +{ + "database-name": "test-database", + "maintain-last-modified": true, + "range-element-index": [ + { + "invalid-values": "reject", + "collation": "", + "localname": "testDateTime", + "namespace-uri": "http://marklogic.com/test", + "range-value-positions": false, + "scalar-type": "dateTime" + }, + { + "invalid-values": "reject", + "collation": "", + "localname": "secondTestDateTime", + "namespace-uri": "http://marklogic.com/test", + "range-value-positions": false, + "scalar-type": "dateTime" + } + ], + "range-path-index": [ + { + "invalid-values": "reject", + "collation": "http://marklogic.com/collation/codepoint", + "path-expression": "docFormat", + "range-value-positions": false, + "scalar-type": "string" + }, + { + "invalid-values": "reject", + "collation": "http://marklogic.com/collation/codepoint", + "path-expression": "eyeColor", + "range-value-positions": false, + "scalar-type": "string" + }, + { + "invalid-values": "reject", + "collation": "http://marklogic.com/collation/codepoint", + "path-expression": "gender", + "range-value-positions": false, + "scalar-type": "string" + }, + { + "invalid-values": "reject", + "collation": "", + "path-expression": "age", + "range-value-positions": false, + "scalar-type": "unsignedInt" + } + ], + "geospatial-element-pair-index": [ + { + "parent-namespace-uri": "", + "parent-localname": "location", + "latitude-namespace-uri": "", + "latitude-localname": "latitude", + "longitude-namespace-uri": "", + "longitude-localname": "longitude", + "coordinate-system": "wgs84", + "range-value-positions": false, + "invalid-values": "reject" + } + ], + "schema-database": "%%SCHEMAS_DATABASE%%", + "triggers-database": "%%TRIGGERS_DATABASE%%" +} diff --git a/app/templates/src/main/ml-config/databases/triggers-database.json b/app/templates/src/main/ml-config/databases/triggers-database.json new file mode 100644 index 00000000..ae16da7f --- /dev/null +++ b/app/templates/src/main/ml-config/databases/triggers-database.json @@ -0,0 +1,3 @@ +{ + "database-name": "%%TRIGGERS_DATABASE%%" +} \ No newline at end of file diff --git a/app/templates/src/main/ml-config/groups/default-group.xml b/app/templates/src/main/ml-config/groups/default-group.xml new file mode 100644 index 00000000..2098aa8f --- /dev/null +++ b/app/templates/src/main/ml-config/groups/default-group.xml @@ -0,0 +1,6 @@ + + Default + + diff --git a/app/templates/src/main/ml-config/rest-api.json b/app/templates/src/main/ml-config/rest-api.json new file mode 100644 index 00000000..838d3289 --- /dev/null +++ b/app/templates/src/main/ml-config/rest-api.json @@ -0,0 +1,12 @@ +{ + "rest-api": { + "name": "%%NAME%%", + "group": "%%GROUP%%", + "database": "%%DATABASE%%", + "modules-database": "%%MODULES_DATABASE%%", + "port": "%%PORT%%", + "xdbc-enabled": true, + "forests-per-host": 3, + "error-format": "json" + } +} diff --git a/app/templates/src/main/ml-config/security/roles/1-sample-project-unit-test-role.json b/app/templates/src/main/ml-config/security/roles/1-sample-project-unit-test-role.json new file mode 100644 index 00000000..3d7ab485 --- /dev/null +++ b/app/templates/src/main/ml-config/security/roles/1-sample-project-unit-test-role.json @@ -0,0 +1,4 @@ +{ + "role-name": "%%mlAppName%%-unit-test", + "description": "a role for unit testing the %%mlAppName%% application" +} diff --git a/app/templates/src/main/ml-config/security/roles/1-sample-project-user-role.json b/app/templates/src/main/ml-config/security/roles/1-sample-project-user-role.json new file mode 100644 index 00000000..9060e96c --- /dev/null +++ b/app/templates/src/main/ml-config/security/roles/1-sample-project-user-role.json @@ -0,0 +1,4 @@ +{ + "role-name": "%%mlAppName%%-role", + "description": "a role for users of the %%mlAppName%% application" +} diff --git a/app/templates/src/main/ml-config/security/roles/2-sample-project-unit-test-roles.json b/app/templates/src/main/ml-config/security/roles/2-sample-project-unit-test-roles.json new file mode 100644 index 00000000..08658902 --- /dev/null +++ b/app/templates/src/main/ml-config/security/roles/2-sample-project-unit-test-roles.json @@ -0,0 +1,15 @@ +{ + "role-name": "%%mlAppName%%-unit-test", + "description": "a role for unit testing the %%mlAppName%% application", + "role": [ "%%mlAppName%%-role" ], + "privilege": [ + { "privilege-name": "any-uri", "action": "http://marklogic.com/xdmp/privileges/any-uri", "kind": "execute" }, + { "privilege-name": "xdmp:eval", "action": "http://marklogic.com/xdmp/privileges/xdmp-eval", "kind": "execute" }, + { "privilege-name": "xdmp:eval-in", "action": "http://marklogic.com/xdmp/privileges/xdmp-eval-in", "kind": "execute" }, + { "privilege-name": "xdmp:http-delete", "action": "http://marklogic.com/xdmp/privileges/xdmp-http-delete", "kind": "execute" }, + { "privilege-name": "xdmp:http-get", "action": "http://marklogic.com/xdmp/privileges/xdmp-http-get", "kind": "execute" }, + { "privilege-name": "xdmp:http-head", "action": "http://marklogic.com/xdmp/privileges/xdmp-http-head", "kind": "execute" }, + { "privilege-name": "xdmp:http-post", "action": "http://marklogic.com/xdmp/privileges/xdmp-http-post", "kind": "execute" }, + { "privilege-name": "xdmp:http-put", "action": "http://marklogic.com/xdmp/privileges/xdmp-http-put", "kind": "execute" } + ] +} diff --git a/app/templates/src/main/ml-config/security/roles/2-sample-project-user-roles.json b/app/templates/src/main/ml-config/security/roles/2-sample-project-user-roles.json new file mode 100644 index 00000000..bbaeb9e5 --- /dev/null +++ b/app/templates/src/main/ml-config/security/roles/2-sample-project-user-roles.json @@ -0,0 +1,14 @@ +{ + "role-name": "%%mlAppName%%-role", + "description": "a role for users of the %%mlAppName%% application", + "permission": [ + { "capability": "execute", "role-name": "%%mlAppName%%-role" }, + { "capability": "update", "role-name": "%%mlAppName%%-role" }, + { "capability": "insert", "role-name": "%%mlAppName%%-role" }, + { "capability": "read", "role-name": "%%mlAppName%%-role" } + ], + "privilege": [ + { "privilege-name": "rest-reader", "action": "http://marklogic.com/xdmp/privileges/rest-reader", "kind": "execute" }, + { "privilege-name": "rest-writer", "action": "http://marklogic.com/xdmp/privileges/rest-writer", "kind": "execute" } + ] +} diff --git a/app/templates/src/main/ml-config/security/roles/README.md b/app/templates/src/main/ml-config/security/roles/README.md new file mode 100644 index 00000000..84fabb38 --- /dev/null +++ b/app/templates/src/main/ml-config/security/roles/README.md @@ -0,0 +1 @@ +See [the MarkLogic docs](http://docs.marklogic.com/REST/POST/manage/v2/roles) for what a role JSON/XML file can contain. \ No newline at end of file diff --git a/app/templates/src/main/ml-config/security/users/README.md b/app/templates/src/main/ml-config/security/users/README.md new file mode 100644 index 00000000..d37e3840 --- /dev/null +++ b/app/templates/src/main/ml-config/security/users/README.md @@ -0,0 +1 @@ +See [the MarkLogic docs](http://docs.marklogic.com/REST/POST/manage/v2/users) for what a user JSON/XML file can contain. \ No newline at end of file diff --git a/app/templates/src/main/ml-config/security/users/sample-project-user.json b/app/templates/src/main/ml-config/security/users/sample-project-user.json new file mode 100644 index 00000000..81e73a1b --- /dev/null +++ b/app/templates/src/main/ml-config/security/users/sample-project-user.json @@ -0,0 +1,6 @@ +{ + "user-name": "%%mlAppName%%-user", + "description": "sample-project user with minimal roles", + "password": "%%appUserPassword%%", + "role": ["%%mlAppName%%-role"] +} diff --git a/app/templates/src/main/ml-config/servers/README.md b/app/templates/src/main/ml-config/servers/README.md new file mode 100644 index 00000000..7795c02a --- /dev/null +++ b/app/templates/src/main/ml-config/servers/README.md @@ -0,0 +1,15 @@ +See [the MarkLogic docs](http://docs.marklogic.com/REST/POST/manage/v2/servers) for what a server JSON/XML file can contain. + +ml-gradle will always look for a file named "rest-api-server.json" in this directory. If found, this file is used to +update the REST API application whose name equals that of the "mlAppName" property. ml-gradle is really designed for +applications based on the REST API, so it's likely that you'll want to use this file to configure the REST API server. +Note that the endpoint for creating a REST API application - http://docs.marklogic.com/REST/POST/v1/rest-apis - is +part of the Client REST API and only provides a few options for configuring the REST API server. You can override +those options via the ml-config/rest-properties.json file. + +ml-gradle will process any other .json/.xml file found in this directory, creating/updating a server as necessary. + +Note that while this example shows an XDBC server being created, you won't often need one with MarkLogic, as you can +either use port 8000 for XCC calls, or you can use the new [/v1/eval](http://docs.marklogic.com/REST/POST/v1/eval) +endpoint on a REST API server. + \ No newline at end of file diff --git a/app/templates/src/main/ml-config/servers/rest-api-server.json b/app/templates/src/main/ml-config/servers/rest-api-server.json new file mode 100644 index 00000000..5656da83 --- /dev/null +++ b/app/templates/src/main/ml-config/servers/rest-api-server.json @@ -0,0 +1,4 @@ +{ + "server-name": "%%NAME%%", + "authentication": "digestbasic" +} diff --git a/app/templates/src/main/ml-modules/README.md b/app/templates/src/main/ml-modules/README.md new file mode 100644 index 00000000..36d2afe9 --- /dev/null +++ b/app/templates/src/main/ml-modules/README.md @@ -0,0 +1,18 @@ +The ml-modules directory contains all of the modules for an application. Its structure is based on that of the +[MarkLogic samplestack reference application](https://github.com/marklogic/marklogic-samplestack/tree/master/database): + +- /ext contains libraries and other application assets; these map to the /v1/ext endpoint in the Client API, though +they are still loaded via XCC, which is much more efficient than using that endpoint. It is considered good practice +to store all application libraries and assets in this directory to avoid any chance of naming collisions with REST API-managed +modules. +- /options contains search options, to be used with /v1/search in the Client API +- /root contains libraries that for some reason cannot be stored under /ext; the best example of this is when you want +to override modules in the Marklogic/Modules directory, such as overriding a REST API module. In that case, you cannot +put them under /ext because the path won't match that of the MarkLogic module. +- /services contains [custom services](http://docs.marklogic.com/REST/GET/v1/config/resources), exposed via /v1/resources/(name of service) +- /transforms contains [custom transforms](http://docs.marklogic.com/REST/GET/v1/config/transforms), to be used with /v1/documents and /v1/search in the Client API + +In addition, the rest-properties.json file in the root of this directory is used to configure the +[properties of the REST API server](http://docs.marklogic.com/REST/GET/v1/config/properties). Note that these properties +are different from the configuration of the server itself - the properties are specific to the REST API out-of-the-box +endpoints. \ No newline at end of file diff --git a/app/templates/data/dictionary-large.xml b/app/templates/src/main/ml-modules/dictionary-large.xml similarity index 100% rename from app/templates/data/dictionary-large.xml rename to app/templates/src/main/ml-modules/dictionary-large.xml diff --git a/app/templates/src/lib/utilities.xqy b/app/templates/src/main/ml-modules/lib/utilities.xqy similarity index 100% rename from app/templates/src/lib/utilities.xqy rename to app/templates/src/main/ml-modules/lib/utilities.xqy diff --git a/app/templates/rest-api/config/options/all.xml b/app/templates/src/main/ml-modules/options/all.xml similarity index 100% rename from app/templates/rest-api/config/options/all.xml rename to app/templates/src/main/ml-modules/options/all.xml diff --git a/app/templates/src/main/ml-modules/rest-properties.json b/app/templates/src/main/ml-modules/rest-properties.json new file mode 100644 index 00000000..970fb27f --- /dev/null +++ b/app/templates/src/main/ml-modules/rest-properties.json @@ -0,0 +1,8 @@ +{ + "content-versions": "none", + "debug": false, + "validate-queries": false, + "document-transform-all": false, + "update-policy": "MERGE_METADATA", + "validate-options": true +} diff --git a/app/templates/src/robots.txt b/app/templates/src/main/ml-modules/robots.txt similarity index 100% rename from app/templates/src/robots.txt rename to app/templates/src/main/ml-modules/robots.txt diff --git a/app/templates/rest-api/ext/analyze-data.xqy b/app/templates/src/main/ml-modules/services/analyze-data.xqy similarity index 99% rename from app/templates/rest-api/ext/analyze-data.xqy rename to app/templates/src/main/ml-modules/services/analyze-data.xqy index 2c91d5a4..0e744d8d 100644 --- a/app/templates/rest-api/ext/analyze-data.xqy +++ b/app/templates/src/main/ml-modules/services/analyze-data.xqy @@ -6,7 +6,6 @@ import module namespace admin = "http://marklogic.com/xdmp/admin" at "/MarkLogic declare default function namespace "http://www.w3.org/2005/xpath-functions"; -declare namespace roxy = "http://marklogic.com/roxy"; declare namespace xs = "http://www.w3.org/2001/XMLSchema"; declare namespace db = "http://marklogic.com/xdmp/database"; @@ -19,9 +18,10 @@ declare variable $analyze-sample-only := false(); declare variable $use-path-indexes := true(); (: + : Params: + : sample-size (integer), action (string), index (string), collection (string) :) declare -%roxy:params("sample-size=xs:integer?", "action=xs:string?", "index=xs:string*", "collection=xs:string*") function ext:get( $context as map:map, $params as map:map @@ -31,9 +31,10 @@ function ext:get( }; (: + : Params: + : sample-size (integer), action (string), index (string), collection (string) :) declare -%roxy:params("sample-size=xs:integer?", "action=xs:string?", "index=xs:string*", "collection=xs:string*") function ext:post( $context as map:map, $params as map:map, diff --git a/app/templates/rest-api/ext/extsimilar.xqy b/app/templates/src/main/ml-modules/services/extsimilar.xqy similarity index 76% rename from app/templates/rest-api/ext/extsimilar.xqy rename to app/templates/src/main/ml-modules/services/extsimilar.xqy index df31ea79..88cfbe09 100644 --- a/app/templates/rest-api/ext/extsimilar.xqy +++ b/app/templates/src/main/ml-modules/services/extsimilar.xqy @@ -4,19 +4,11 @@ module namespace ext = "http://marklogic.com/rest-api/resource/extsimilar"; declare default function namespace "http://www.w3.org/2005/xpath-functions"; -declare namespace roxy = "http://marklogic.com/roxy"; - -(: - : To add parameters to the functions, specify them in the params annotations. - : Example - : declare %roxy:params("uri=xs:string", "priority=xs:int") ext:get(...) - : This means that the get function will take two parameters, a string and an int. - :) - (: + : Params + : uri (string), limit (int) :) declare -%roxy:params("uri=xs:string", "limit=xs:int?") function ext:get( $context as map:map, $params as map:map @@ -26,9 +18,10 @@ function ext:get( }; (: + : Params + : uri (string), limit (int) :) declare -%roxy:params("uri=xs:string", "limit=xs:int?") function ext:post( $context as map:map, $params as map:map, diff --git a/app/templates/rest-api/ext/extspell.xqy b/app/templates/src/main/ml-modules/services/extspell.xqy similarity index 75% rename from app/templates/rest-api/ext/extspell.xqy rename to app/templates/src/main/ml-modules/services/extspell.xqy index 13b38c6a..7c2e006e 100644 --- a/app/templates/rest-api/ext/extspell.xqy +++ b/app/templates/src/main/ml-modules/services/extspell.xqy @@ -6,21 +6,13 @@ import module namespace spell = "http://marklogic.com/xdmp/spell" at "/MarkLogic declare default function namespace "http://www.w3.org/2005/xpath-functions"; -declare namespace roxy = "http://marklogic.com/roxy"; - declare variable $dictionary := "/dictionary-large.xml"; (: - : To add parameters to the functions, specify them in the params annotations. - : Example - : declare %roxy:params("uri=xs:string", "priority=xs:int") ext:get(...) - : This means that the get function will take two parameters, a string and an int. - :) - -(: + : Params + : pqtxt (string), limit (int) :) declare -%roxy:params("pqtxt=xs:string", "limit=xs:int?") function ext:get( $context as map:map, $params as map:map @@ -30,9 +22,10 @@ function ext:get( }; (: + : Params + : pqtxt (string), limit (int) :) declare -%roxy:params("pqtxt=xs:string", "limit=xs:int?") function ext:post( $context as map:map, $params as map:map, diff --git a/app/templates/rest-api/transforms/download.xsl b/app/templates/src/main/ml-modules/transforms/download.xsl similarity index 100% rename from app/templates/rest-api/transforms/download.xsl rename to app/templates/src/main/ml-modules/transforms/download.xsl diff --git a/app/templates/rest-api/transforms/extractedToJson.sjs b/app/templates/src/main/ml-modules/transforms/extractedToJson.sjs similarity index 100% rename from app/templates/rest-api/transforms/extractedToJson.sjs rename to app/templates/src/main/ml-modules/transforms/extractedToJson.sjs diff --git a/app/templates/rest-api/transforms/filter-docs.xqy b/app/templates/src/main/ml-modules/transforms/filter-docs.xqy similarity index 100% rename from app/templates/rest-api/transforms/filter-docs.xqy rename to app/templates/src/main/ml-modules/transforms/filter-docs.xqy diff --git a/app/templates/rest-api/transforms/from-json.xsl b/app/templates/src/main/ml-modules/transforms/from-json.xsl similarity index 100% rename from app/templates/rest-api/transforms/from-json.xsl rename to app/templates/src/main/ml-modules/transforms/from-json.xsl diff --git a/app/templates/rest-api/transforms/get-property.xsl b/app/templates/src/main/ml-modules/transforms/get-property.xsl similarity index 100% rename from app/templates/rest-api/transforms/get-property.xsl rename to app/templates/src/main/ml-modules/transforms/get-property.xsl diff --git a/app/templates/rest-api/transforms/indent.xsl b/app/templates/src/main/ml-modules/transforms/indent.xsl similarity index 100% rename from app/templates/rest-api/transforms/indent.xsl rename to app/templates/src/main/ml-modules/transforms/indent.xsl diff --git a/app/templates/rest-api/transforms/sanitize.xsl b/app/templates/src/main/ml-modules/transforms/sanitize.xsl similarity index 100% rename from app/templates/rest-api/transforms/sanitize.xsl rename to app/templates/src/main/ml-modules/transforms/sanitize.xsl diff --git a/app/templates/rest-api/transforms/to-json.xsl b/app/templates/src/main/ml-modules/transforms/to-json.xsl similarity index 100% rename from app/templates/rest-api/transforms/to-json.xsl rename to app/templates/src/main/ml-modules/transforms/to-json.xsl diff --git a/slushfile.js b/slushfile.js index 40d7324e..890de669 100644 --- a/slushfile.js +++ b/slushfile.js @@ -39,17 +39,13 @@ function printUsage() { process.stdout.write(' slush marklogic-node [args]\n'); process.stdout.write('\n'); process.stdout.write('Arguments:\n'); - process.stdout.write(' fork=<..> Github fork to use for Roxy. Defaults to: marklogic\n'); - process.stdout.write(' branch=<..> Github branch to use for Roxy. Defaults to: master\n'); process.stdout.write(' theme=<..> Slush theme to use. Defaults to: default\n'); - process.stdout.write(' ml-version=<..> MarkLogic version. Defaults to: 9\n'); process.stdout.write(' ml-host=<..> Host on which MarkLogic runs. Defaults to: localhost\n'); process.stdout.write(' ml-admin-user=<..> User for MarkLogic deployments. Defaults to: admin\n'); process.stdout.write(' ml-admin-pass=<..> Pass for MarkLogic deployments. Defaults to: \n'); process.stdout.write(' ml-app-user=<..> MarkLogic user for guest access. Defaults to: -user\n'); process.stdout.write(' ml-app-pass=<..> MarkLogic pass for guest access. Defaults to: \n'); process.stdout.write(' ml-http-port=<..> Port at which MarkLogic app-server runs. Defaults to: 8070\n'); - process.stdout.write(' ml-xcc-port=<..> Port at which MarkLogic xcc-server runs. Defaults to: \n'); process.stdout.write(' node-port=<..> Port at which Node.js middle-tier runs. Defaults to: 9070\n'); process.stdout.write(' guest-access=<..> Whether guests are automatically logged in. Defaults to: false\n'); process.stdout.write(' disallow-updates=<..> Whether updates to MarkLogic should be disallowed. Defaults to: false\n'); @@ -105,26 +101,19 @@ function isFlag(arg) { function processInput() { var allowedFlags = [ 'app-name', - 'fork', - 'branch', 'theme', - 'ml-version', 'ml-host', 'ml-admin-user', 'ml-admin-pass', 'ml-app-user', 'ml-app-pass', 'ml-http-port', - 'ml-xcc-port', 'node-port', 'guest-access', 'disallow-updates', 'appusers-only' ]; - var inputs = { - fork: 'marklogic', - branch: 'master' - }; + var inputs = {}; (gulp.args || process.argv).forEach(function(arg) { if (arg === 'help') { printUsage(); @@ -158,135 +147,43 @@ function getNameProposal() { } } -// Download the Roxy ml script from GitHub -function getRoxyScript(appName, mlVersion, fork, branch) { - fork = fork || 'marklogic'; - branch = branch || 'master'; - - var d = q.defer(), - out; - - var scriptName = (win32 ? '' : './') + 'ml' + (win32 ? '.bat' : ''); - - console.log('Retrieving Roxy script'); - out = fs.createWriteStream(scriptName); - var stream = new FetchStream('https://github.com/' + fork + '/roxy/raw/' + branch + '/' + scriptName); - stream.pipe(out); - stream.on('end', function() { - console.log('Got Roxy script'); - out.end(); - - fs.chmod(scriptName, '755', function(err) { - if (err) { - console.log(err); - d.reject(err); - } else { - console.log('chmod done; appName=' + appName + '; mlVersion=' + mlVersion + '; fork=' + fork + '; branch=' + branch); - d.resolve({ - 'script': scriptName, - 'app': appName, - 'mlVersion': mlVersion, - 'fork': fork, - 'branch': branch - }); - } - }); - }); - - return d.promise; -} - -// Run the Roxy 'ml new' command for the new project -function runRoxy(config) { - var scriptName = config.script, - appName = config.app, - mlVersion = config.mlVersion, - fork = config.fork, - branch = config.branch; - - var d = q.defer(); - - var args = [ - 'new', - appName, - '--server-version=' + mlVersion, - '--app-type=rest', - '--fork=' + fork, - '--branch=' + branch - ]; - - console.log('Spawning Roxy new command: ' + scriptName + ' ' + args.join(' ')); - var child = spawn(scriptName, args, { - stdio: [ - 0, // Use parents stdin for child - 'pipe', // Pipe child's stdout to parent (default) - 'pipe' // Pipe child's stderr to parent (default) - ] - }); - - child.on('close', function() { - console.log('done running ml new'); - d.resolve('done'); - }); - - child.stdout.on('data', function(data) { - console.log('' + data); - }); - - child.stderr.on('data', function(data) { - console.log('' + data); - }); - - return d.promise; -} - -// Make some changes to Roxy's deploy/build.properties file for the out-of-the-box application -function configRoxy() { - console.log('Configuring Roxy'); +// Make some changes to Gradle's gradle.properties file for the out-of-the-box application +function configGradle() { + console.log('Generating local overrides for gradle.properties'); try { - var properties = fs.readFileSync('deploy/build.properties', encoding); - - // Ensure the authentication-method property is set to digest - properties = properties.replace(/^authentication\-method=basic/m, 'authentication-method=digest'); - properties = properties.replace(/^authentication\-method=digestbasic/m, 'authentication-method=digest'); + var properties = fs.readFileSync(__dirname + '/app/templates/gradle.properties', encoding); - // Capture appuser-password from Roxy properties for later use - var passwordMatch = /^appuser\-password=(.*)$/m; + // Capture appuser-password from gradle properties for later use + var passwordMatch = /^appUserPassword=(.*)$/m; var matches = passwordMatch.exec(properties); settings.appuserPassword = matches[1]; // Bypass bug in Roxy causing {, } and \ in pwd to get mangled + // Is this needed for gradle? if (settings.appuserPassword.match(/[\{\}\\]/g)) { settings.appuserPassword = settings.appuserPassword.replace(/[\{\}\\]/g, ''); - properties = properties.replace(passwordMatch, 'appuser-password=' + settings.appuserPassword); + properties = properties.replace(passwordMatch, 'appUserPassword=' + settings.appuserPassword); } - fs.writeFileSync('deploy/build.properties', properties); + fs.writeFileSync('gradle.properties', properties); } catch (e) { - console.log('failed to update properties: ' + e.message); + console.log('failed to update gradle.properties: ' + e.message); process.exit(1); } try { var localProperties = '#################################################################\n' + - '# This file contains overrides to values in build.properties\n' + + '# This file contains overrides to values in gradle.properties\n' + '# These only affect your local environment and should not be checked in\n' + '#################################################################\n' + '\n' + - 'server-version=' + settings.mlVersion + '\n' + + 'mlAppName=' + settings.appName + '\n' + '\n' + '#\n' + '# The ports used by your application\n' + '#\n' + - 'app-port=' + settings.appPort + '\n'; - if (settings.xccPort || (settings.mlVersion < 8)) { - localProperties += 'xcc-port=' + settings.xccPort + '\n'; - } else { - localProperties += '# Taking advantage of not needing a XCC Port for ML8\n' + - 'xcc-port=${app-port}\n' + - 'install-xcc=false\n'; - } + 'mlRestPort=' + settings.appPort + '\n'; localProperties += '\n' + '#\n' + @@ -294,108 +191,20 @@ function configRoxy() { '# WARNING: if you are running these scripts on WINDOWS you may need to change localhost to 127.0.0.1\n' + '# There have been reported issues with dns resolution when localhost wasn\'t in the hosts file.\n' + '#\n' + - 'local-server=' + settings.marklogicHost + '\n' + + 'mlHost=' + settings.marklogicHost + '\n' + 'content-forests-per-host=3\n' + '\n' + '#\n' + '# Admin username/password that will exist on the local/dev/prod servers\n' + '#\n' + - 'user=' + settings.marklogicAdminUser + '\n' + - 'password=' + settings.marklogicAdminPass + '\n'; + 'mlUsername=' + settings.marklogicAdminUser + '\n' + + 'mlPassword=' + settings.marklogicAdminPass + '\n'; - fs.writeFileSync('deploy/local.properties', localProperties, encoding); + fs.writeFileSync('gradle-local.properties', localProperties, encoding); } catch (e) { - console.log('failed to write roxy local.properties'); + console.log('failed to write gradle-local.properties'); process.exit(1); } - - try { - var foo = fs.readFileSync('deploy/ml-config.xml', encoding); - - // add a geospatial index for the default content - foo = foo.replace(/^\s*/m, - ' \n' + - ' \n' + - ' \n' + - ' location\n' + - ' \n' + - ' latitude\n' + - ' \n' + - ' longitude\n' + - ' wgs84\n' + - ' false\n' + - ' \n'); - - // add range path index for the default content - foo = foo.replace(/^\s*/m, - ' \n' + - ' \n' + - ' string\n' + - ' http://marklogic.com/collation/codepoint\n' + - ' docFormat\n' + - ' false\n' + - ' reject\n' + - ' \n' + - ' \n' + - ' string\n' + - ' http://marklogic.com/collation/codepoint\n' + - ' eyeColor\n' + - ' false\n' + - ' reject\n' + - ' \n' + - ' \n' + - ' string\n' + - ' http://marklogic.com/collation/codepoint\n' + - ' gender\n' + - ' false\n' + - ' reject\n' + - ' \n' + - ' \n' + - ' unsignedInt\n' + - ' age\n' + - ' false\n' + - ' reject\n' + - ' \n'); - - // fix default app-role privileges to match rest-style applications - foo = foo.replace(/[^]*?<\/privileges>/, - '\n' + - ' \n' + - ' rest-reader\n' + - ' \n' + - ' \n' + - ' \n' + - ' rest-writer\n' + - ' \n' + - ' \n'); - - fs.writeFileSync('deploy/ml-config.xml', foo); - - console.log('Applying replacements..'); - gulp.src(['deploy/ml-config.xml']) - .pipe(xmlpoke({ - replacements: [{ - namespaces: { - db: 'http://marklogic.com/xdmp/database' - }, - xpath: ['//db:database[1]/db:stemmed-searches','//db:database[1]/db:word-searches','//db:database[1]/db:trailing-wildcard-searches'], - valueType: 'remove' - },{ - namespaces: { - db: 'http://marklogic.com/xdmp/database' - }, - xpath: '//db:database[1]', - valueType: 'append', - value: ' basic\n true\n true\n ' - }] - })) - .pipe(gulp.dest('deploy')); - - } catch (e) { - console.log('failed to update configuration: ' + e.message); - process.exit(1); - } - } gulp.task('npmInstall', ['init', 'generateSecret', 'configGulp'], function(done) { @@ -440,11 +249,6 @@ gulp.task('configGulp', ['init'], function(done) { configJSON['ml-app-user'] = settings.appName + '-user'; configJSON['ml-app-pass'] = settings.appuserPassword; configJSON['ml-http-port'] = settings.appPort; - - if (settings.xccPort || (settings.mlVersion < 8)) { - configJSON['ml-xcc-port'] = settings.xccPort; - } - configJSON['node-port'] = settings.nodePort; configJSON['guest-access'] = settings.guestAccess; configJSON['disallow-updates'] = settings.disallowUpdates; @@ -472,15 +276,7 @@ gulp.task('init', ['checkForUpdates'], function(done) { var appName = clArgs['app-name']; var prompts = []; - if (!clArgs['ml-version']) { - prompts.push({ - type: 'list', - name: 'mlVersion', - message: 'MarkLogic version?', - choices: ['9', '8', '7'], - default: 0 - }); - } + if (!clArgs['ml-host']) { prompts.push({ type: 'input', @@ -514,17 +310,6 @@ gulp.task('init', ['checkForUpdates'], function(done) { default: 8070 }); } - if (!clArgs['ml-xcc-port']) { - prompts.push({ - type: 'input', - name: 'xccPort', - message: 'XCC port?', - default: 8071, - when: function(answers) { - return (answers.mlVersion || clArgs['ml-version']) < 8; - } - }); - } if (!clArgs['node-port']) { prompts.push({ type: 'input', @@ -630,54 +415,46 @@ gulp.task('init', ['checkForUpdates'], function(done) { } else { settings.appName = _.slugify(appName); } - settings.mlVersion = answers.mlVersion || clArgs['ml-version']; settings.marklogicHost = answers.marklogicHost || clArgs['ml-host']; settings.marklogicAdminUser = answers.marklogicAdminUser || clArgs['ml-admin-user']; settings.marklogicAdminPass = answers.marklogicAdminPass || clArgs['ml-admin-pass'] || ''; settings.nodePort = answers.nodePort || clArgs['node-port']; settings.appPort = answers.appPort || clArgs['ml-http-port']; - settings.xccPort = answers.xccPort || clArgs['ml-xcc-port'] || null; settings.guestAccess = answers.guestAccess || clArgs['guest-access']; settings.disallowUpdates = answers.disallowUpdates || clArgs['disallowed-updates']; settings.appUsersOnly = answers.appUsersOnly || clArgs['appusers-only']; settings.theme = answers.theme || answers.template || clArgs.theme || 'default'; - getRoxyScript(settings.appName, settings.mlVersion, clArgs.fork, clArgs.branch) - .then(runRoxy) - .then(function() { - // Copy over the Angular files - var files = [__dirname + '/app/templates/**']; - if (settings.theme !== 'default') { // overlay the theme if not the default theme chosen - files.push(__dirname + '/app/themes/' + settings.theme + '/**'); - } - - process.chdir('./' + settings.appName); - - configRoxy(); - - gulp.src(files) - .pipe(rename(function(file) { - // change _foo to .foo - if (file.basename[0] === '_') { - file.basename = '.' + file.basename.slice(1); - } - - })) - .pipe(replace('@slush-version', pkgSettings.version.trim(), skipBinary)) - .pipe(replace('@sample-app-name', settings.appName, skipBinary)) - .pipe(replace('@node-port', settings.nodePort, skipBinary)) - .pipe(replace('@ml-http-port', settings.appPort, skipBinary)) - .pipe(replace('@ml-xcc-port', settings.xccPort || settings.appPort, skipBinary)) - .pipe(replace('@ml-host', settings.marklogicHost, skipBinary)) - .pipe(gulp.dest('./')) // Relative to cwd - .on('end', function() { - done(); // Finished! - }); - }, - function(reason) { - console.log('Caught an error: ' + reason); - }); + var files = [__dirname + '/app/templates/**']; + if (settings.theme !== 'default') { + // overlay the theme if not the default theme chosen + files.push(__dirname + '/app/themes/' + settings.theme + '/**'); + } + + if(!fs.existsSync('./' + settings.appName)) { + fs.mkdirSync('./' + settings.appName); + } + process.chdir('./' + settings.appName); + configGradle(); + + gulp.src(files) + .pipe(rename(function(file) { + // change _foo to .foo + if (file.basename[0] === '_') { + file.basename = '.' + file.basename.slice(1); + } + + })) + .pipe(replace('@slush-version', pkgSettings.version.trim(), skipBinary)) + .pipe(replace('@sample-app-name', settings.appName, skipBinary)) + .pipe(replace('@node-port', settings.nodePort, skipBinary)) + .pipe(replace('@ml-http-port', settings.appPort, skipBinary)) + .pipe(replace('@ml-host', settings.marklogicHost, skipBinary)) + .pipe(gulp.dest('./')) // Relative to cwd + .on('end', function() { + done(); // Finished! + }); });