diff --git a/.gitignore b/.gitignore index 4a19e2ac..0ab3d12c 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,8 @@ .sass-cache node_modules npm-debug.log +.gradle/ +.project +.classpath +build/ +.settings/ diff --git a/README-JavaScript.md b/README-JavaScript.md index b1bf01d8..89832451 100644 --- a/README-JavaScript.md +++ b/README-JavaScript.md @@ -23,8 +23,8 @@ This README focuses on the developer automation for JavaScript development that To configure and develop the browser app for Samplestack, you need the following software: -* Node.js, version 0.10.x (it is mot yet compatible with Node.js 0.11 or 0.12). See [nodejs.org](http://nodejs.org). -* npm, version 2.1.1 or higher. See [npmjs.com](https://www.npmjs.com/) +* Node.js, version 0.10.33 or higher. See [nodejs.org](http://nodejs.org). +* npm, version 2.1.12 or higher. See [npmjs.com](https://www.npmjs.com/) * git. See [git-scm.com](http://git-scm.com/) This is "tech-stack" or minor variants thereof are almost ubiquitous in JavaScript dwvelopment shops in 2015. diff --git a/README.md b/README.md index 33bd2a4b..412dfabd 100644 --- a/README.md +++ b/README.md @@ -20,18 +20,19 @@ REMEMBER to change the branch name in this code when preparing releases > Samplestack is a comprehensive sample application that demonstrates how to build an effective MarkLogic application. Based on the idea of a "Question and Answer" website, Samplestack shows you how to integrate MarkLogic into a three-tier application architecture (browser, application server, and database). -## README for Version 1.1.0 +## README for Version 1.2.0 This release features two middle tiers - one for the Java enterprise developer, implemented using Java, Spring and Gradle -- one for the JavaScript developer, imple,emted using JavaScript, Node.js and Gulp. +- one for the JavaScript developer, implemented using JavaScript, Node.js and Gulp. The project includes the following major components: * [MarkLogic](http://www.marklogic.com/) for the database tier * MVC browser application implemented in [Angular.js](https://angularjs.org) * Middle-tier REST server implemented in Java/[Spring](http://projects.spring.io/spring-framework/) * [Gradle](http://www.gradle.org/) framework to drive build and configuration of the appserver and database tiers in Java/Groovy +* [ml-grade](https://github.com/rjrudin/ml-gradle) Gradle plugin that supports a number of tasks pertaining to deploying an application to MarkLogic and interacting with other features of MarkLogic via a Gradle build file. * Middle-tier REST server implemented in Node.js/[Express](expressjs.com) * [Gulp](http://www.gradle.org/)-based automation to drive build and configuration of Javascript and database tiers (**note: database-tier Gulp-based setup coming soon**) * Unit and end-to-end tests diff --git a/appserver/.gitignore b/appserver/.gitignore new file mode 100644 index 00000000..ae3c1726 --- /dev/null +++ b/appserver/.gitignore @@ -0,0 +1 @@ +/bin/ diff --git a/appserver/java-spring/README.md b/appserver/java-spring/README.md index f9d28933..1c98facf 100644 --- a/appserver/java-spring/README.md +++ b/appserver/java-spring/README.md @@ -7,7 +7,7 @@ The implementation of samplestack that runs using * gradle as build tool * Java as middle-tier development language * Spring Boot as application framework -* MarkLogic as database tier +* MarkLogic as database tier (and [ml-gradle](https://github.com/rjrudin/ml-gradle) as the MarkLogic-specific bulid tool) This README covers running samplestack quickly, then documents each of the commands available to the Java developer as she iterates through code exploration. @@ -16,7 +16,7 @@ available to the Java developer as she iterates through code exploration. *To build and run:* -Before running anything here, you need MarkLogic 8.0-1.1, installed and +Before running anything here, you need MarkLogic 8.0-1.1 or later, installed and running. Start this quickstart with it installed and running. By default this process will will secure MarkLogic with username admin, password admin. If you have already secured MarkLogic, you need to update gradle.properties with the @@ -57,21 +57,17 @@ Its possible that you will need to clean your environment, especially if you're `./gradlew --stop` (if you've been using the gradle daemon.) `./gradlew clean` -`./gradlew dbteardown` +`./gradlew undeploy` (an ml-gradle task) -*gradle tasks used in Samplestack development* +*other gradle tasks used in Samplestack development* -* `./gradlew dbInit` This is the task that initializes a MarkLogic server and preps it for runninng Samplestack. It is a one-time task; after running, the users, roles, database, and REST server will be available for configuration. -* `./gradlew dbTeardown` This command removes Samplestack's entire REST server, database, and security objects from the MarkLogic server. +* `./gradlew mlInit` (an ml-gradle task) This is the task that initializes a MarkLogic server and preps it for runninng Samplestack. It is a one-time task; after running, the users, roles, database, and REST server will be available for configuration. +* `./gradlew mlUndeploy` (an ml-gradle task) This command removes Samplestack's entire REST server, database, and security objects from the MarkLogic server. * `./gradlew clean` This built-in gradle task cleans the build directory. When in doubt, try it. * `./gradlew assemble` This command bootstraps the middle tier, runs tests, and builds the Java project. When it is done you have verified the unit tests and built samplestack. * `./gradlew tasks` Lists tasks available to the samplestack project. -* `./gradlew dbConfigure` Incrementally applies configuration changes in ../../database/* to the MarkLogic instance. -* `./gradlew dbConfigureClean` Removes cache info for configuration from build direcotiry (so next dbConfigure will process all files). -* `./gradlew dbConfigureAll` dbConfigureClean then runs dbConfigure. Uploads all config files to MarkLogic. - * `./gradlew bootRun` This command runs the middle tier and MarkLogic services. This project also contains a built version of the front-end angular application. If you want to use and exercise the front-end independently, see the sibling project in /browser for instructions on running the browser application * `./gradlew seedDataFetch` Fetches seed data from a remote location to the build directory. * `./gradlew seedDataExtract` Extracts the fetched seed data tgz to within the directory. @@ -86,12 +82,14 @@ Its possible that you will need to clean your environment, especially if you're * `./gradlew dbTest` This command runs the unit tests for server extensions. * `./gradlew integrationTest` This command runs the tests that exercise the whole middle and database tier apparatus. +In previous releases of Samplestack, the tasks `dbConfigure`, `dbConfigure` and `dbConfigureAll` were present, allowing for incrementally +applying configuration and re-applying configuration. Those tasks are no longer included -- however, similar functionality is exposed by ml-gradle +and may be used in Samplestack, as well (e.g. `mlClearModules`, `mlLoadModules`. **Documentation of each task that ml-gradle supports is beyond the scope of this document. Please see the [ml-gradle documentation](https://github.com/rjrudin/ml-gradle) for more information.** + *To use with Eclipse* See project wiki https://github.com/marklogic/marklogic-samplestack/wiki/Getting-Started-in-Eclipse - - ### Endpoints Here are the endpoints supported by the middle tier appserver: diff --git a/appserver/java-spring/build.gradle b/appserver/java-spring/build.gradle index de637064..a53ac329 100644 --- a/appserver/java-spring/build.gradle +++ b/appserver/java-spring/build.gradle @@ -16,6 +16,9 @@ buildscript { maven { url "http://repo.spring.io/snapshot" } maven { url "http://repo.spring.io/milestone" } jcenter() + + maven {url "http://developer.marklogic.com/maven2/"} + maven {url "http://rjrudin.github.io/marklogic-java/releases"} } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}") @@ -24,100 +27,17 @@ buildscript { /* plugins */ apply plugin: 'groovy' -apply plugin: 'java' -apply plugin: 'eclipse' -apply plugin: 'idea' apply plugin: 'spring-boot' + group = samplestackGroup version = samplestackVersion sourceCompatibility = samplestackSourceCompatibility -/* - * INIT - * Run this task to initialize a fresh MarkLogic server - * and install security objects. - */ -task dbInit(type: MarkLogicInitTask) -dbInit.roles = file("../../database/security/roles") -dbInit.users = file("../../database/security/users") - -/* - * TEARDOWN - * This task removes all trace of samplestack - */ -task dbTeardown(type: MarkLogicTeardownTask) - -/* - * CONFIGURE - * - * This task configures the database and REST API instance - * based on files in the database directory - */ -task dbConfigure(type: MarkLogicConfigureTask) - -task dbConfigureClean << { - delete("${buildDir}/database") -} - - -dbConfigure.inputDir = file('../../database') -dbConfigure.outputDir = file("${buildDir}/database") -dbConfigure.inputProperty = project.properties['taskInputProperty'] ?: "original" -dbConfigure.shouldRunAfter dbInit -dbConfigure.shouldRunAfter dbConfigureClean -dbInit.dependsOn(dbConfigureClean) - -task dbConfigureAll -dbConfigureAll.dependsOn dbConfigureClean -dbConfigureAll.dependsOn dbConfigure - /** * ASSEMBLE */ -assemble.dependsOn(dbInit, dbConfigure, test) - -/** - * FETCH SEED DATA - */ -task seedDataFetch(type: MarkLogicFetchTask) { - url = project.hasProperty("seedDataUrl") ? project.seedDataUrl : "http://developer.marklogic.com/media/gh/seed-data1.8.2.tgz" - destFile = file("${buildDir}/seed.tgz") -} -seedDataFetch.dependsOn(processResources) - -task seedDataExtract { - ext.destDir = file("${buildDir}/seed-data") - ext.srcFile = file("${buildDir}/seed.tgz") - inputs.file srcFile - outputs.dir destDir - doLast { - destDir.mkdirs() - copy { - from tarTree(resources.gzip(srcFile)) - into destDir - } - } -} -seedDataExtract.dependsOn(seedDataFetch) - -/* - * LOAD - */ -task dbLoad(type: MarkLogicSlurpTask) { - seedDirectory = file("${buildDir}/seed-data") - inputs.dir seedDirectory -} -dbLoad.mustRunAfter(assemble) -dbLoad.dependsOn(seedDataExtract) -/* never skip dbload if requested */ -dbLoad.outputs.upToDateWhen { false } - -/* - * CLEAR - */ -task dbClear(type: MarkLogicClearTask) - +assemble.dependsOn(":database:mlDeploy") /* use same properties for gradle and Java runtime */ task props(type: Copy) { @@ -135,8 +55,9 @@ task appserver << { println "Bootstrapping, seeding and starting Samplestack appserver" } -appserver.dependsOn(clean, assemble, dbLoad, bootRun) -bootRun.shouldRunAfter(dbLoad) +appserver.dependsOn(clean, ":appserver:java-spring:assemble", ":database:dbLoad", bootRun) +bootRun.shouldRunAfter(":database:dbLoad") + /* The code repositories to consult for dependencies */ repositories { @@ -173,7 +94,7 @@ task unitTest(type: Test) { } } -test.dependsOn(props, dbConfigure) +test.dependsOn(props, ":database:mlLoadModules") task dbTest(type: Test) { useJUnit { @@ -187,7 +108,3 @@ task integrationTest(type: Test) { } } -/* task to generate the gradle wrapper script */ -task wrapper(type: Wrapper) { - gradleVersion = '2.3' -} diff --git a/appserver/java-spring/buildSrc/build.gradle b/appserver/java-spring/buildSrc/build.gradle deleted file mode 100644 index 8d2def27..00000000 --- a/appserver/java-spring/buildSrc/build.gradle +++ /dev/null @@ -1,18 +0,0 @@ -apply plugin: 'groovy' - -repositories { - mavenCentral() -} - -dependencies { - compile gradleApi() - compile("org.codehaus.groovy:groovy-all:2.3.7") - compile("org.codehaus.groovy.modules.http-builder:http-builder:0.7") - compile('com.marklogic:java-client-api:3.0.1') { - exclude(group: 'org.slf4j') - exclude(group: 'ch.qos.logback') - } - testCompile("junit:junit:4.10") -} - - diff --git a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicClearTask.groovy b/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicClearTask.groovy deleted file mode 100644 index 088d9b5c..00000000 --- a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicClearTask.groovy +++ /dev/null @@ -1,18 +0,0 @@ -import groovy.json.* -import groovyx.net.http.RESTClient -import org.gradle.api.DefaultTask -import org.gradle.api.tasks.TaskAction - -public class MarkLogicClearTask extends MarkLogicTask { - - @TaskAction - void updateDatabase() { - logger.error("Saving Database Configuration") - RESTClient client = manageClient("databases/" + config.marklogic.rest.name) - def params = [:] - params.contentType = "application/json" - params.body = '{"operation":"clear-database"}' - post(client,params) - } - -} diff --git a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicConfigureTask.groovy b/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicConfigureTask.groovy deleted file mode 100644 index 07a394d1..00000000 --- a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicConfigureTask.groovy +++ /dev/null @@ -1,186 +0,0 @@ -import groovy.json.* -import groovyx.net.http.RESTClient -import org.gradle.api.DefaultTask -import org.gradle.api.tasks.TaskAction -import org.gradle.api.tasks.InputDirectory -import org.gradle.api.tasks.OutputDirectory -import org.gradle.api.tasks.Input -import org.gradle.api.file.FileCollection -import org.gradle.api.file.FileTree -import org.gradle.api.tasks.incremental.IncrementalTaskInputs - -public class MarkLogicConfigureTask extends MarkLogicTask { - - @InputDirectory - def File inputDir - - @OutputDirectory - def File outputDir - - @Input - def inputProperty - - def sep = java.io.File.separator - def database = "database" - def dbProperties = database + sep + "database-properties.json" - def transforms = database + sep + "transforms" - def restExtensions = database + sep + "ext" - def restServices = database + sep +"services" - def options = database + sep + "options" - def restProperties = database + sep + "rest-properties.json" - def file = null - - - @TaskAction - void configureREST(IncrementalTaskInputs inputs) { - logger.info(inputs.incremental ? "CHANGED inputs considered out of date" : "ALL inputs considered out of date") - logger.debug("File Separator: " + sep) - inputs.outOfDate { change -> - logger.debug("out of date: ${change.file.name}") - def targetFile = new File(outputDir, change.file.name) - targetFile.text = "done" - logger.debug("Processing file " + change.file.path) - def changeFile = change.file - if (changeFile.path =~ /\/\./) { - logger.info("Skipping hidden file " + change.file.name) - } - else if (changeFile.path =~ /seed-data/) { - logger.info("Skipping seed data") - } - else if (changeFile.path.contains(transforms)) { - logger.warn("Putting transform " + change.file.name) - putTransform(changeFile) - } - else if (changeFile.path.contains(restExtensions)) { - logger.warn("Putting library extension " + change.file.name) - putExtension(changeFile) - } - else if (changeFile.path.contains(restServices)) { - logger.warn("Putting service extension " + change.file.name) - putServiceExtension(changeFile) - } - else if (changeFile.path.contains(options)) { - logger.warn("Putting search options " + change.file.name) - putOptions(changeFile) - } - else if (changeFile.path.contains(dbProperties)) { - logger.warn("Putting database configuration " + change.file.name) - putDatabaseConfig(changeFile) - } - else if (changeFile.path.contains(restProperties)) { - putRESTProperties(changeFile) - } - else if (changeFile.path.contains("security")) { - logger.info("Skipping security configuration" + change.file.name) - } else { - logger.warn("Looks like " + change.file.path + " is not a MarkLogic configuration file... skipping") - } - } - - inputs.removed { change -> - logger.debug( "removed: ${change.file.name}") - def targetFile = new File(outputDir, change.file.path) - targetFile.delete() - } - - } - - void putDatabaseConfig(c) { - logger.info( "Saving Database Configuration") - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8002/manage/v2/databases/" + config.marklogic.rest.name + "/properties") - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - def params = [:] - params.contentType = "application/json" - params.body = c.text - put(client,params) - } - - void putTransform(transform) { - def transformFileName = transform.getPath().replaceAll(~"\\\\","/") - def transformName = transformFileName.split("/")[-1].replaceAll(~"\\.[^\\.]+", "") - if (transformName) { - logger.info( "Saving transform " + transformName) - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":" + config.marklogic.rest.port + "/v1/config/transforms/" + transformName) - client.getEncoder().putAt("application/xquery", client.getEncoder().getAt("text/plain")) - client.getEncoder().putAt("application/javascript", client.getEncoder().getAt("text/plain")) - client.getEncoder().putAt("application/vnd.marklogic-javascript", client.getEncoder().getAt("text/plain")) - client.getParser().putAt("application/vnd.marklogic-javascript", client.getParser().getAt("text/plain")) - client.getParser().putAt("*/*", client.getParser().getAt("application/json")) - client.auth.basic config.marklogic.rest.admin.user, config.marklogic.rest.admin.password - def params = [:] - if (transformFileName.endsWith("sjs")) { - params.contentType = "application/javascript"; - } else { - params.contentType = "application/xquery" - } - params.body = transform.text - put(client, params) - } - else { - logger.debug( "Skipping filename " + transformName) - } - } - - void putExtension(extension) { - def extensionFileName = extension.getPath().replaceAll(~"\\\\","/") - def extensionName = - extensionFileName.replaceAll(~".*\\/","") - logger.info( "Saving library extension " + extensionFileName) - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":" + config.marklogic.rest.port + "/v1/ext/" + extensionName) - client.getEncoder().putAt("application/xquery", client.getEncoder().getAt("text/plain")) - client.getEncoder().putAt("application/javascript", client.getEncoder().getAt("text/plain")) - client.auth.basic config.marklogic.rest.admin.user, config.marklogic.rest.admin.password - def params = [:] - if (extensionFileName.endsWith("sjs")) { - params.contentType = "application/javascript" - } else { - params.contentType = "application/xquery" - } - params.body = extension.text - put(client,params) - } - - void putServiceExtension(extension) { - def extensionFileName = extension.getPath().replaceAll(~"\\\\","/") - def extensionName = - extensionFileName.replaceAll(~".*\\/","").replaceAll(~"\\.(sjs|xqy)","") - logger.info( "Saving service extension " + extensionFileName) - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":" + config.marklogic.rest.port + "/v1/config/resources/" + extensionName) - client.getEncoder().putAt("application/xquery", client.getEncoder().getAt("text/plain")) - client.getEncoder().putAt("application/javascript", client.getEncoder().getAt("text/plain")) - client.auth.basic config.marklogic.rest.admin.user, config.marklogic.rest.admin.password - def params = [:] - if (extensionFileName.endsWith("sjs")) { - params.contentType = "application/javascript" - } else { - params.contentType = "application/xquery" - } - params.body = extension.text - put(client,params) - } - - void putOptions(options) { - def optionsFileName = options.getPath().replaceAll(~"\\\\","/").replaceAll(~"\\.json","") - def optionsName = - optionsFileName.replaceAll(~".*\\/","") - logger.info( "Saving options " + optionsFileName) - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":" + config.marklogic.rest.port + "/v1/config/query/" + optionsName) - client.auth.basic config.marklogic.rest.admin.user, config.marklogic.rest.admin.password - def params = [:] - params.contentType = "application/json" - params.body = options.text - put(client,params) - } - - void putRESTProperties(File f) { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":" + config.marklogic.rest.port + "/v1/config/properties") - client.auth.basic config.marklogic.rest.admin.user, config.marklogic.rest.admin.password - def params = [:] - params.contentType = "application/json" - params.body = f.text - logger.info( "Configuring Properties") - put(client,params) - } - -} - diff --git a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicFetchTask.groovy b/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicFetchTask.groovy deleted file mode 100644 index 4d7795c3..00000000 --- a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicFetchTask.groovy +++ /dev/null @@ -1,43 +0,0 @@ -import groovy.json.* -import groovyx.net.http.RESTClient -import org.gradle.api.DefaultTask -import org.gradle.api.tasks.TaskAction -import org.gradle.api.tasks.OutputFile -import org.gradle.api.tasks.Input - -public class MarkLogicFetchTask extends MarkLogicTask { - - @Input - String url - - @OutputFile - File destFile - - @TaskAction - void fetchSeedData() { - logger.warn("Fetching data from " + url) - def is = new URL(url).openStream() - - try { - OutputStream os = new FileOutputStream(destFile); - boolean finished = false; - try { - byte[] buf = new byte[1024 * 10]; - int read; - while ((read = is.read(buf)) >= 0) { - os.write(buf, 0, read); - } - os.flush(); - finished = true; - } finally { - os.close(); - if (!finished) { - destFile.delete(); - } - } - } finally { - is.close(); - } - } - -} diff --git a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicInitTask.groovy b/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicInitTask.groovy deleted file mode 100644 index b075d385..00000000 --- a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicInitTask.groovy +++ /dev/null @@ -1,124 +0,0 @@ -import groovy.json.* -import groovyx.net.http.RESTClient -import org.gradle.api.DefaultTask -import org.gradle.api.Project -import org.gradle.api.tasks.* -import java.net.URLDecoder - - -public class MarkLogicInitTask extends MarkLogicTask { - - def roles - def users - - @TaskAction - void initMarkLogic() { - adminInit() - adminSetup() - createUsers() - restBoot() - } - - void adminInit() { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8001/admin/v1/init") - def params = [:] - params.contentType = "application/json" - params.body = "{}" - try { - def response = client.post(params) - logger.warn("MarkLogic initialized.") - delay() - } - catch (ex) { - if ( ex.response.status == 403 ) { - logger.warn("Unauthorized. Check your installation, try to start with a fresh one.") - throw new StopActionException(ex) - } - else if ( ex.response.status == 401 ) - logger.warn("Server already secured. Initialization skipped.") - else if ( ex.response.status == 500 ) - logger.warn("Server already initialized. Initialization skipped") - else - throw ex; - } - } - - void adminSetup() { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8001/admin/v1/instance-admin") - def params = [:] - params.contentType = "application/json" - params.body = String.format('{ "admin-username" : "%s", "admin-password" : "%s", "realm" : "public" }', config.marklogic.admin.user, config.marklogic.admin.password) - try { - client.post(params) - logger.warn("MarkLogic admin secured.") - delay() - } - catch (ex) { - if ( ex.response.status == 403 ) { - logger.warn("Unauthorized. Check your installation, try to start with a fresh one.") - throw new StopActionException(ex) - } - else if ( ex.response.status == 401 ) - logger.warn("Server already secured. Initialization skipped") - else - logger.warn("Got " + ex.response.status) - } - } - - private void create(path, jsonObject) { - try { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8002/manage/v2/" + path) - client.headers."accept" = "application/json" - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - - def params = [:] - params.contentType = "application/json" - params.body = jsonObject.text - client.post(params) - } catch (ex) { - if (ex.response) - { - try { - logger.warn("" + ex.response.data) - } catch (ex2) { - logger.warn("Error parsing JSON response: " + ex.response) - } - } - else - { - logger.warn("No response") - } - } - } - - void createUser(jsonUser) { - create("users", jsonUser) - } - - void createRole(jsonRole) { - create("roles", jsonRole) - } - - void createUsers() { - logger.warn("Creating users and roles...") - roles.listFiles().findAll { it.name.contains("guest") }.each { createRole(it) } - roles.listFiles().findAll { it.name.contains("writer") }.each { createRole(it) } - users.listFiles().each { createUser(it) } - } - - void restBoot() { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8002/v1/rest-apis").with { - headers."accept" = "application/json" - auth.basic config.marklogic.admin.user, config.marklogic.admin.password - it - } - def params = [:] - params.contentType = "application/json" - params.body = String.format('{"rest-api" : { "name" : "%s", "port" : %s }}', config.marklogic.rest.name, config.marklogic.rest.port) - try { - post(client, params) - } catch (ex) { - logger.warn("Ignoring server creation error... ") - } - } -} diff --git a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicTask.groovy b/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicTask.groovy deleted file mode 100644 index 527781a1..00000000 --- a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicTask.groovy +++ /dev/null @@ -1,93 +0,0 @@ -import groovy.json.* -import groovyx.net.http.RESTClient -import org.gradle.api.DefaultTask -import org.gradle.api.tasks.TaskAction -import com.marklogic.client.DatabaseClientFactory -import com.marklogic.client.DatabaseClientFactory.Authentication - -public class MarkLogicTask extends DefaultTask { - - protected props = new Properties() - protected client - protected docMgr - - MarkLogicTask() { - super() - project.file("gradle.properties").withInputStream { props.load(it) } - ext.config = new ConfigSlurper().parse(props) - client = DatabaseClientFactory.newClient( - config.marklogic.rest.host, - Integer.parseInt(config.marklogic.rest.port), - config.marklogic.writer.user, - config.marklogic.writer.password, - Authentication.DIGEST) - docMgr = client.newJSONDocumentManager() - } - - /* returns a client that can use port 8002 to nanage MarkLogic */ - protected manageClient(String url) { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8002/manage/v2/" + url) - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - return client; - } - - /* returns a client that can bootstrap and teardown REST APIs */ - protected bootstrapClient() { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8002/v1/rest-apis/" + config.marklogic.rest.name) - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - return client; - } - - - protected storeJson(uri, handle) { - docMgr.write(uri, handle); - } - - - protected put(client, params) { - try { - def response = client.put(params) - logger.warn("Success: " + response.status) - } catch (ex) - { - if (ex.response) - { - logger.warn("" + ex.response.data.text) - } - else - { - logger.warn("No response") - } - } - } - - protected post(client, params) { - try { - def response = client.post(params) - logger.warn("Success: " + response.status) - } catch (ex) - { - if (ex.response) - { - logger.warn("" + ex.response.data) - } - else - { - logger.warn("No response") - } - } - } - - protected void delay() { - RESTClient client = new RESTClient("http://" + config.marklogic.rest.host + ":8001/admin/v1/timestamp") - def params = [:] - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - try { - Thread.sleep(1000) - client.get(params) - } catch (ex) { - logger.warn("Waiting for server restart..."); - delay(); - } - } -} diff --git a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicTeardownTask.groovy b/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicTeardownTask.groovy deleted file mode 100644 index 8495ece4..00000000 --- a/appserver/java-spring/buildSrc/src/main/groovy/MarkLogicTeardownTask.groovy +++ /dev/null @@ -1,54 +0,0 @@ -import groovy.json.* -import groovyx.net.http.RESTClient -import org.gradle.api.DefaultTask -import org.gradle.api.tasks.TaskAction - -public class MarkLogicTeardownTask extends MarkLogicTask { - - @TaskAction - void teardown() { - teardownRest(); - delay(); - removeUsers(); - } - - void teardownRest() { - RESTClient client = bootstrapClient(); - def params = [:] - params.queryString = "include=content&include=modules" - try { - client.delete(params) - } catch (ex) { - if (ex.response) - { - logger.error(": " + ex.response.data.text) - } - else - { - logger.warn("No response from server") - } - throw new RuntimeException("Failed Teardown. Check environment before proceeding.", ex) - } - } - - void removeUsers() { - def url = "http://" + config.marklogic.rest.host + ":8002/manage/v2/" - RESTClient client = new RESTClient(url + "users/samplestack-admin") - def params = [:] - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - client.delete(params) - client = new RESTClient(url + "users/samplestack-guest") - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - client.delete(params) - client = new RESTClient(url + "users/samplestack-contributor") - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - client.delete(params) - client = new RESTClient(url + "roles/samplestack-guest") - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - client.delete(params) - client = new RESTClient(url + "roles/samplestack-writer") - client.auth.basic config.marklogic.admin.user, config.marklogic.admin.password - client.delete(params) - } - -} diff --git a/appserver/java-spring/gradle.properties b/appserver/java-spring/gradle.properties index a8c2dbbe..6213e7b1 100644 --- a/appserver/java-spring/gradle.properties +++ b/appserver/java-spring/gradle.properties @@ -1,31 +1,6 @@ org.gradle.daemon=false samplestackGroup=com.marklogic -samplestackVersion=1.1.0 +samplestackVersion=1.2.0 samplestackSourceCompatibility=1.7 -# dbInit secures the server with these credentials -# dbInit uses these credentials to create users and the REST instance -# dbConfigure uses these credentials to manage indexes and database properties -marklogic.admin.user=admin -marklogic.admin.password=admin - -# dbConfigure uses these credentials to manage the REST API instance -marklogic.rest.admin.user=samplestack-admin -marklogic.rest.admin.password=samplestack-admin-password - -# dbLoad uses these credentials to load the database with seed data -# the Java runtime uses these credentials for "contributor" access -marklogic.writer.user=samplestack-contributor -marklogic.writer.password=sc-pass - -# the Java runtime uses these credentials for "guest" access -marklogic.guest.user=samplestack-guest -marklogic.guest.password=sa-pass - -# the host for dbInit, dbConfigure and the MarkLogic REST API -marklogic.rest.host=localhost -# the port for the MarkLogic REST API -marklogic.rest.port=8006 -# the name of the MarkLogic REST API instance -marklogic.rest.name=samplestack diff --git a/appserver/java-spring/src/main/java/com/marklogic/samplestack/dbclient/Clients.java b/appserver/java-spring/src/main/java/com/marklogic/samplestack/dbclient/Clients.java index 6feae264..b294cd7f 100644 --- a/appserver/java-spring/src/main/java/com/marklogic/samplestack/dbclient/Clients.java +++ b/appserver/java-spring/src/main/java/com/marklogic/samplestack/dbclient/Clients.java @@ -52,8 +52,8 @@ public class Clients extends HashMap { * @return A DatabaseClient for accessing MarkLogic */ private DatabaseClient databaseClient(ClientRole role) { - String host = env.getProperty("marklogic.rest.host"); - Integer port = Integer.parseInt(env.getProperty("marklogic.rest.port")); + String host = env.getProperty("mlHost"); + Integer port = Integer.parseInt(env.getProperty("mlRestPort")); String username = env.getProperty(role.getUserParam()); String password = env.getProperty(role.getPasswordParam()); return DatabaseClientFactory.newClient(host, port, username, password, diff --git a/appserver/java-spring/src/main/resources/gradle.properties b/appserver/java-spring/src/main/resources/gradle.properties index b09b1380..93b9473f 100644 --- a/appserver/java-spring/src/main/resources/gradle.properties +++ b/appserver/java-spring/src/main/resources/gradle.properties @@ -1,8 +1,14 @@ org.gradle.daemon=false samplestackGroup=com.marklogic -samplestackVersion=1.0.1-SNAPSHOT +samplestackVersion=1.1.0 samplestackSourceCompatibility=1.7 +# ml-gradle properties +mlHost=localhost +mlRestPort=8006 +mlUserame=admin +mlPassword=admin +mlAppName=samplestack # dbInit secures the server with these credentials # dbInit uses these credentials to create users and the REST instance @@ -22,11 +28,3 @@ marklogic.writer.password=sc-pass # the Java runtime uses these credentials for "guest" access marklogic.guest.user=samplestack-guest marklogic.guest.password=sa-pass - -# the host for dbInit, dbConfigure and the MarkLogic REST API -marklogic.rest.host=localhost -# the port for the MarkLogic REST API -marklogic.rest.port=8006 -# the name of the MarkLogic REST API instance -marklogic.rest.name=samplestack - diff --git a/appserver/java-spring/static/_marklogic/services/data/_jsonschema.browserify.js b/appserver/java-spring/static/_marklogic/services/data/_jsonschema.browserify.js index 30072865..2f809ffc 100644 --- a/appserver/java-spring/static/_marklogic/services/data/_jsonschema.browserify.js +++ b/appserver/java-spring/static/_marklogic/services/data/_jsonschema.browserify.js @@ -26,15 +26,20 @@ define([], function () { },{"jsonschema":9}],2:[function(require,module,exports){ (function (global){ -/*! http://mths.be/punycode v1.2.4 by @mathias */ +/*! https://mths.be/punycode v1.3.2 by @mathias */ ;(function(root) { /** Detect free variables */ - var freeExports = typeof exports == 'object' && exports; + var freeExports = typeof exports == 'object' && exports && + !exports.nodeType && exports; var freeModule = typeof module == 'object' && module && - module.exports == freeExports && module; + !module.nodeType && module; var freeGlobal = typeof global == 'object' && global; - if (freeGlobal.global === freeGlobal || freeGlobal.window === freeGlobal) { + if ( + freeGlobal.global === freeGlobal || + freeGlobal.window === freeGlobal || + freeGlobal.self === freeGlobal + ) { root = freeGlobal; } @@ -60,8 +65,8 @@ define([], function () { /** Regular expressions */ regexPunycode = /^xn--/, - regexNonASCII = /[^ -~]/, // unprintable ASCII chars + non-ASCII chars - regexSeparators = /\x2E|\u3002|\uFF0E|\uFF61/g, // RFC 3490 separators + regexNonASCII = /[^\x20-\x7E]/, // unprintable ASCII chars + non-ASCII chars + regexSeparators = /[\x2E\u3002\uFF0E\uFF61]/g, // RFC 3490 separators /** Error messages */ errors = { @@ -100,23 +105,37 @@ define([], function () { */ function map(array, fn) { var length = array.length; + var result = []; while (length--) { - array[length] = fn(array[length]); + result[length] = fn(array[length]); } - return array; + return result; } /** - * A simple `Array#map`-like wrapper to work with domain name strings. + * A simple `Array#map`-like wrapper to work with domain name strings or email + * addresses. * @private - * @param {String} domain The domain name. + * @param {String} domain The domain name or email address. * @param {Function} callback The function that gets called for every * character. * @returns {Array} A new string of characters returned by the callback * function. */ function mapDomain(string, fn) { - return map(string.split(regexSeparators), fn).join('.'); + var parts = string.split('@'); + var result = ''; + if (parts.length > 1) { + // In email addresses, only the domain name should be punycoded. Leave + // the local part (i.e. everything up to `@`) intact. + result = parts[0] + '@'; + string = parts[1]; + } + // Avoid `split(regex)` for IE8 compatibility. See #17. + string = string.replace(regexSeparators, '\x2E'); + var labels = string.split('.'); + var encoded = map(labels, fn).join('.'); + return result + encoded; } /** @@ -126,7 +145,7 @@ define([], function () { * UCS-2 exposes as separate characters) into a single code point, * matching UTF-16. * @see `punycode.ucs2.encode` - * @see + * @see * @memberOf punycode.ucs2 * @name decode * @param {String} string The Unicode input string (UCS-2). @@ -335,8 +354,8 @@ define([], function () { } /** - * Converts a string of Unicode symbols to a Punycode string of ASCII-only - * symbols. + * Converts a string of Unicode symbols (e.g. a domain name label) to a + * Punycode string of ASCII-only symbols. * @memberOf punycode * @param {String} input The string of Unicode symbols. * @returns {String} The resulting Punycode string of ASCII-only symbols. @@ -449,17 +468,18 @@ define([], function () { } /** - * Converts a Punycode string representing a domain name to Unicode. Only the - * Punycoded parts of the domain name will be converted, i.e. it doesn't - * matter if you call it on a string that has already been converted to - * Unicode. + * Converts a Punycode string representing a domain name or an email address + * to Unicode. Only the Punycoded parts of the input will be converted, i.e. + * it doesn't matter if you call it on a string that has already been + * converted to Unicode. * @memberOf punycode - * @param {String} domain The Punycode domain name to convert to Unicode. + * @param {String} input The Punycoded domain name or email address to + * convert to Unicode. * @returns {String} The Unicode representation of the given Punycode * string. */ - function toUnicode(domain) { - return mapDomain(domain, function(string) { + function toUnicode(input) { + return mapDomain(input, function(string) { return regexPunycode.test(string) ? decode(string.slice(4).toLowerCase()) : string; @@ -467,15 +487,18 @@ define([], function () { } /** - * Converts a Unicode string representing a domain name to Punycode. Only the - * non-ASCII parts of the domain name will be converted, i.e. it doesn't - * matter if you call it with a domain that's already in ASCII. + * Converts a Unicode string representing a domain name or an email address to + * Punycode. Only the non-ASCII parts of the domain name will be converted, + * i.e. it doesn't matter if you call it with a domain that's already in + * ASCII. * @memberOf punycode - * @param {String} domain The domain name to convert, as a Unicode string. - * @returns {String} The Punycode representation of the given domain name. + * @param {String} input The domain name or email address to convert, as a + * Unicode string. + * @returns {String} The Punycode representation of the given domain name or + * email address. */ - function toASCII(domain) { - return mapDomain(domain, function(string) { + function toASCII(input) { + return mapDomain(input, function(string) { return regexNonASCII.test(string) ? 'xn--' + encode(string) : string; @@ -491,11 +514,11 @@ define([], function () { * @memberOf punycode * @type String */ - 'version': '1.2.4', + 'version': '1.3.2', /** * An object of methods to convert from JavaScript's internal character * representation (UCS-2) to Unicode code points, and back. - * @see + * @see * @memberOf punycode * @type Object */ @@ -520,8 +543,8 @@ define([], function () { define('punycode', function() { return punycode; }); - } else if (freeExports && !freeExports.nodeType) { - if (freeModule) { // in Node.js or RingoJS v0.8.0+ + } else if (freeExports && freeModule) { + if (module.exports == freeExports) { // in Node.js or RingoJS v0.8.0+ freeModule.exports = punycode; } else { // in Narwhal or RingoJS v0.7.0- for (key in punycode) { diff --git a/appserver/java-spring/static/app/dialogs/allTags.html b/appserver/java-spring/static/app/dialogs/allTags.html index 7dfe96f1..46a0f68f 100644 --- a/appserver/java-spring/static/app/dialogs/allTags.html +++ b/appserver/java-spring/static/app/dialogs/allTags.html @@ -90,7 +90,7 @@ ng-checked="selTags.indexOf(tag.name) > -1" ng-click="onTagClick(tag.name)" /> - {{tag.name}} + ({{tag.count}}) diff --git a/appserver/java-spring/static/app/dialogs/allTags.js b/appserver/java-spring/static/app/dialogs/allTags.js index ab03f2be..daccb4c9 100644 --- a/appserver/java-spring/static/app/dialogs/allTags.js +++ b/appserver/java-spring/static/app/dialogs/allTags.js @@ -124,6 +124,30 @@ define(['app/module'], function (module) { ]; $scope.selectedSort = $scope.sorts[1]; // Default sort + + // format a tag by bolding the searchText portion of the text + $scope.highlighted = function (tagText, searchText) { + if (searchText && searchText.length) { + var begin = tagText.indexOf(searchText); + if (begin < 0) { + return tagText; + } + + var formatted = tagText.substr(0, begin) + + '' + + searchText + + '' + + tagText.substr(begin + searchText.length); + console.log(tagText, searchText, formatted); + + return formatted; + } + else { + return tagText; + } + }; + + /** * @ngdoc method * @name allTagsDialogCtlr#$scope.onTagClick diff --git a/appserver/java-spring/static/app/dialogs/login.js b/appserver/java-spring/static/app/dialogs/login.js index be94a14c..cb7deafb 100644 --- a/appserver/java-spring/static/app/dialogs/login.js +++ b/appserver/java-spring/static/app/dialogs/login.js @@ -1,18 +1,18 @@ -/* - * Copyright 2012-2015 MarkLogic Corporation - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ +/* + * Copyright 2012-2015 MarkLogic Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ define(['app/module'], function (module) { @@ -78,6 +78,7 @@ define(['app/module'], function (module) { * property. */ $scope.authenticate = function () { + $scope.error = null; credentialsHack(); mlAuth.authenticate($scope.session).then( onAuthSuccess, diff --git a/appserver/java-spring/static/app/directives/ssFacetTags.html b/appserver/java-spring/static/app/directives/ssFacetTags.html index 260fc6ed..e03ee024 100644 --- a/appserver/java-spring/static/app/directives/ssFacetTags.html +++ b/appserver/java-spring/static/app/directives/ssFacetTags.html @@ -64,6 +64,7 @@ showRelated(tag);$event.stopPropagation(); " ng-class="{'rel-selected': relatedShown === tag}" + title="Show tags related to this tag" > -
+