Merge branch 'master' into dino/197-Refactor-to-autodeploy

* master:
  fixing bug in restoring previous load options
  removing unused files
  - updating readme - adding appveyor yaml
  tweaking the form validation
  Added default input file path in loading data
  Used angular.forEach instead of jQuery's $.each
  - Fixed issue in saving options and loading of previous options - Also added validation for missing input file path
  Changes to fix JS errors in Swagger UI in master
  enhance performance of hub install
  pointing build status to correct link cleaning up logging in tests
  adding appveyor build status for windows
  1.0.0-beta.4 release
  Cleaned up Load UI Fixed unit tests on Windows
  fixed #89  - refactored mlcp to use mlcp-util from @rjrudin  - removed RunInputFlow from the DataHub class and jar. It doesn't make sense to live there  - removed the dependencies on ngMaterial because it was conflicting with bootstrap ui  - removed unused files to reduce the size of the final quick-start jar  - fixed compile errors in Groovy Tasks  - added some mlcp options to the load dialog  - added the ability to copy the mlcp command to the clipboard  - added .. to the directory list to navigate back up  - made the "hover over labels" info less "in your face"  - updated the MarkLogic Data Hub Banner to fit on smaller terminal windows  - broke out mlcp loading stuff in the UI into its own module  - moved the download mlcp options to be client side only  - updated the saving of current mlcp properties to use json (compatible with mlcp-util)
  Removed automatic closing of notification

Conflicts:
	marklogic-data-hub/src/main/java/com/marklogic/hub/DataHub.java
	quick-start/src/main/resources/static/app/services/dataHubService.js

Author: maeisabelle08

CHANGELOG.md

@@ -1,7 +1,31 @@
 # Change Log
 
-## [v1.0.0-beta.3](https://github.com/marklogic/marklogic-data-hub/tree/v1.0.0-beta.3)
+## [v1.0.0-beta.4](https://github.com/marklogic/marklogic-data-hub/tree/HEAD)
 
+[Full Changelog](https://github.com/marklogic/marklogic-data-hub/compare/v1.0.0-beta.3...v1.0.0-beta.4)
+
+**Implemented enhancements:**
+
+- Minor improvements to Load Data Feature [\#205](https://github.com/marklogic/marklogic-data-hub/issues/205)
+
+**Fixed bugs:**
+
+- File permission error running hadoop to do data load [\#89](https://github.com/marklogic/marklogic-data-hub/issues/89)
+
+**Closed issues:**
+
+- Rename conform/conformance to harmonize [\#201](https://github.com/marklogic/marklogic-data-hub/issues/201)
+- Errors only flash on the GUI for a short time [\#192](https://github.com/marklogic/marklogic-data-hub/issues/192)
+- Investigate MLCP UI for creating MLCP cmd line options [\#164](https://github.com/marklogic/marklogic-data-hub/issues/164)
+- Examples need demo scripts [\#99](https://github.com/marklogic/marklogic-data-hub/issues/99)
+- DataHub.installUserModules should be "syncUserModules" [\#45](https://github.com/marklogic/marklogic-data-hub/issues/45)
+
+**Merged pull requests:**
+
+- Fix mlcp windows issue 89 [\#215](https://github.com/marklogic/marklogic-data-hub/pull/215) ([paxtonhare](https://github.com/paxtonhare))
+- 192 - Removed automatic closing of notification [\#208](https://github.com/marklogic/marklogic-data-hub/pull/208) ([maeisabelle](https://github.com/maeisabelle))
+
+## [v1.0.0-beta.3](https://github.com/marklogic/marklogic-data-hub/tree/v1.0.0-beta.3) (2016-04-15)
 [Full Changelog](https://github.com/marklogic/marklogic-data-hub/compare/v1.0.0-beta.2...v1.0.0-beta.3)
 
 **Merged pull requests:**

CONTRIBUTING.md

@@ -46,22 +46,19 @@ $ cd marklogic-data-hub
 $ git remote add upstream git://github.com/marklogic/marklogic-data-hub.git
 ```
 
-All bug fixes and new features go into the dev branch.
-
 We ask that you open an issue in the [issue tracker][] and get agreement from
 at least one of the project maintainers before you start coding.
 
 Nothing is more frustrating than seeing your hard work go to waste because
 your vision does not align with that of a project maintainer.
 
-
 #### Create a branch for your changes
 
 Okay, so you have decided to fix something. Create a feature branch
 and start hacking:
 
 ```sh
-$ git checkout -b my-feature-branch -t origin/dev
+$ git checkout -b my-feature-branch -t origin/master
 ```
 
 #### Formatting code

README.md

@@ -1,88 +1,14 @@
-[![Build Status](https://travis-ci.org/marklogic/marklogic-data-hub.svg?branch=master)](https://travis-ci.org/marklogic/marklogic-data-hub)
+OS | Status
+--- | --- | ---
+Linux/Mac | [![Build Status](https://travis-ci.org/marklogic/marklogic-data-hub.svg?branch=master)](https://travis-ci.org/marklogic/marklogic-data-hub)
+Windows | [![Windows Build status](https://ci.appveyor.com/api/projects/status/kgj0k5na59uhkvbv?svg=true)](https://ci.appveyor.com/project/paxtonhare/marklogic-data-hub)
 
 # MarkLogic Data Hub
 
 Go from nothing to Enterprise Data Hub in a matter of minutes.  
 
-
 This project allows you to deploy a skeleton Data Hub into MarkLogic. With some basic configuration you will be running an Enterprise Data Hub in no time.
 
-# Quick Start
-Want to get up and running quickly? Try the quick-start jar.
-
-- Download the jar from the [releases page](https://github.com/marklogic/marklogic-data-hub/releases/latest).
-- Run the Jar
-  `java -jar quick-start-1.0.0-alpha.1.jar`
-- Open the Quickstart Application in your browser:
-  http://localhost:8080
-
-
-# Hacking on the Hub
-If you want to start hacking on the internals of the Hub then look here.
-
-#### Clone the Repo
-First clone the repo
-
-#### Building the Hub
-Note that the Unit tests take a very long time to run. This command skips them
-run: `./gradlew build -x test`
-
-#### Running the Hub
-run: `./gradlew bootRun`
-
-#### Running Tests
-run: `./gradlew test`
-
-#### Using with an IDE
-##### Eclipse
-To generate eclipse project files run:
-`./gradlew eclipse`
-
-Then import the project into eclipse
-
-####Available Transforms
-Data Hub has provided several transforms that can be used when installed along with the created entities and flows in the MarkLogic server.
-####run-flow
-This transform can be used to run the flow when inserting document. It accepts the following `entity-name` and `flow-name`. 
-
-#####Use Cases:
-
-1. Using [MarkLogic REST API PUT /v1/documents](http://docs.marklogic.com/REST/PUT/v1/documents)
-
-`curl --anyauth --user <user>:<password> -X PUT -T <documentDirectory> -H "Content-type:<contentType>" 'http://<mlHost>:<port>/<version>/documents?uri=<uri>&transform=run-flow&trans:entity-name=<entityName>&trans:flow-name=<flowName>'`
-
-Example:
-
-`cat ./documents/employee1.xml`
-
-`<employee xmlns="http://company.com/ns/employee">`
-`<id>1</id>`
-`</employee>`
-
-To insert/update this document with uri '/employee1.xml' into the database 'data-hub-STAGING' (with host 'localhost' and port '8010'), given a user 'admin' with password 'admin' and rest-writer role AND to be able to run the flow 'IngestFlow' of the 'Customer' entity, run the following:
-
-`curl --anyauth --user admin:admin -X PUT -T ./documents/employee1.xml -H "Content-type:application/xml" 'http://localhost:8010/LATEST/documents?uri=/employee1.xml&transform=run-flow&trans:entity-name=Customer&trans:flow-name=IngestFlow'`
-
-This will create a document with uri '/employee1.xml'. The content will depend on the data format of the entity. 
-
-If it is JSON ('application/json'), the content will be:
-
-`{`
-`"identifier": "/employee1.xml", `
-`"content": "<employee xmlns=\"http://company.com/ns/employee\">\n  <id>1</id>\n</employee>"`
-`}`
-
-Else if it is XML ('application/xml'), it will be:
+# Getting Started
 
-`<?xml  version="1.0" encoding="UTF-8"?>`
-`<envelope xmlns="http://marklogic.com/data-hub/envelope">`
-`<headers>`
-`</headers>`
-`<triples>`
-`</triples>`
-`<content>`
-`<employee xmlns="http://company.com/ns/employee">`
-`<id>1</id>`
-`</employee>`
-`</content>`
-`</envelope>`
+Head over to our [Getting Started Tutorial](http://marklogic.github.io/marklogic-data-hub/) to get up and running with the Data Hub.

appveyor.yml

@@ -0,0 +1,78 @@
+version: 1.0.{build}
+branches:
+  only:
+  - master
+image: WMF 5
+environment:
+  ML_USER:
+    secure: nioOZ/YaPcyFoqN9bVZHMllinShpPT+lG1uyi78H23c=
+  ML_PASSWORD:
+    secure: LaFtKimqM2zAtGUkciP5RA==
+  ML_VERSION: 8.0-5.1
+install:
+- ps: >-
+    function HttpPost($cookieContainer, $url, $postData) {
+        $uri = New-Object "System.Uri" "$url"
+        $webrequest = [System.Net.HTTPWebRequest]::Create($uri)
+        $webrequest.CookieContainer = $cookieContainer
+        $webrequest.ContentType = "application/x-www-form-urlencoded"
+        $webrequest.Method = "POST"
+
+        $encoding = New-Object System.Text.ASCIIEncoding
+        $bytes = $encoding.GetBytes($postData)
+        $webrequest.ContentLength = $bytes.Length
+
+        $newStream = $webrequest.GetRequestStream()
+        $newStream.Write($bytes, 0, $bytes.Length)
+        $response = $webrequest.GetResponse().GetResponseStream()
+        $sr = New-Object System.IO.StreamReader($response)
+        $respTxt = $sr.ReadToEnd()
+        $sr.Close()
+        $response.Dispose()
+        return $respTxt
+    }
+
+    function downloadFile($cookieContainer, $url, $targetFile) {
+        $cookie = $cookieContainer.GetCookies("https://developer.marklogic.com")[0]
+        $cookieTxt = $cookie.NAME + "=" + $cookie.Value
+        $wc = New-Object System.Net.WebClient
+        $wc.Headers.Add([System.Net.HttpRequestHeader]::Cookie, $cookieTxt)
+        $wc.DownloadFile($url, $targetFile)
+    }
+
+    $cookieContainer = New-Object System.Net.CookieContainer
+    $email = $env:ML_USER
+    $password = $env:ML_PASSWORD
+    $loginData = "email=$email&password=$password"
+
+    # Login
+    $url = "https://developer.marklogic.com/login"
+    $response = HttpPost $cookieContainer $url $loginData
+
+    # Get Download url
+    $ver = $env:ML_VERSION
+    $fname = "MarkLogic-$ver-amd64.msi"
+    $url = "https://developer.marklogic.com/get-download-url"
+    $postData = "download=/download/binaries/8.0/$fname"
+    $response = HttpPost $cookieContainer $url $postData
+    $json = $response | ConvertFrom-Json
+    $downloadLink = $json.path
+    $downloadLink = "https://developer.marklogic.com$downloadLink"
+
+    # Download File
+    downloadFile $cookieContainer $downloadLink "$PSScriptRoot\$fname"
+    msiexec /i "$PSScriptRoot\$fname" /quiet /qn /norestart /log install.log
+    Start-Sleep -s 90
+    cat .\install.log
+    Start-Service -displayname "MarkLogic"
+    HttpPost $cookieContainer "http://localhost:8001/admin/v1/init" ""
+    Start-Sleep -s 10
+    $postData = "admin-username=admin&admin-password=admin&realm=public"
+    HttpPost $cookieContainer "http://localhost:8001/admin/v1/instance-admin" $postData
+    Start-Sleep -s 10
+build: off
+test_script:
+- ps: >-
+    c:
+    cd \projects\marklogic-data-hub
+    .\gradlew.bat test

build.gradle

@@ -18,7 +18,7 @@ subprojects {
 
     test{
         testLogging{
-            events 'started','passed', 'skipped'
+            events 'started','passed', 'skipped', 'failed'
         }
     }
 

examples/gradle-advanced/build.gradle

@@ -10,7 +10,7 @@ plugins {
     id 'eclipse'
     id 'idea'
     id 'net.saliman.properties' version '1.4.5'
-    id 'com.marklogic.ml-data-hub-plugin' version '1.0.0-beta.3'
+    id 'com.marklogic.ml-data-hub-plugin' version '1.0.0-beta.4'
     id 'com.marklogic.ml-gradle' version '2.1.0'
 }
 

marklogic-data-hub/build.gradle

@@ -27,11 +27,7 @@ dependencies {
     compile 'com.marklogic:ml-app-deployer:2.0'
     compile 'commons-io:commons-io:2.4'
     compile 'org.apache.commons:commons-csv:1.2'
-    compile('com.marklogic:mlcp:8.0-5') {
-        exclude module : 'servlet-api'
-        exclude group: 'com.google.guava', module: 'guava'
-    }
-    compile 'com.google.guava:guava:19.0'
+    compile 'org.glassfish.jaxb:txw2:2.2.11'
     testCompile 'org.springframework.batch:spring-batch-test:3.0.6.RELEASE'
     testCompile 'junit:junit:4.12'
     testCompile 'xmlunit:xmlunit:1.3'

marklogic-data-hub/gradle.properties

@@ -28,6 +28,6 @@ mlSchemasDbName=data-hub-SCHEMAS
 
 hubModulesPath=exmaples/hr-hub/plugins
 
-version=1.0.0-beta.3
+version=1.0.0-beta.4