Skip to content

Commit a941dfc

Browse files
committed
updated spring batch README
1 parent 1a70a13 commit a941dfc

File tree

1 file changed

+28
-4
lines changed

1 file changed

+28
-4
lines changed

examples/spring-batch/README.md

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,30 @@
22

33
This example demonstrates how to run a custom Spring Batch job against the Data Hub Framework.
44

5-
This is based on the [base job from MarkLogic Spring batch](https://github.com/sastafford/marklogic-spring-batch/tree/master/examples/base). The base job is intended to be a starter template for writing any MarkLogic Spring Batch job. Just copy and paste this directory into your workspace and modify the classes to your specific batch processing use case.
5+
Learning [Spring Batch](http://docs.spring.io/spring-batch/reference/html/spring-batch-intro.html) is beyond the scope of this README. But let's pretend you know enough to be dangerous.
6+
7+
Now you want to use Spring Batch to load a bunch of data into MarkLogic. Maybe that data is coming from a message queue. Maybe it's a bunch of files in a folder. It can be anything really.
8+
9+
## What's the Big Idea?
10+
The idea is pretty simple. You read the data, do a little processing (maybe), and then write it into MarkLogic. But to properly integrate with the Data Hub Framework you will want to run your data through an [input flow](https://github.com/marklogic/marklogic-data-hub/wiki/The-MarkLogic-Data-Hub-Overview#ingest).
11+
12+
## How does it work?
13+
This example includes a sample Spring Boot Configuration [LoadAndRunFlow.java](https://github.com/marklogic/marklogic-data-hub/blob/develop/examples/spring-batch/src/main/java/example/LoadAndRunFlow.java) that configures a job to ingest some xml files and run a flow.
14+
15+
This example depends on a runtime class **com.marklogic.spring.batch.hub.HubJobRunner** that is responsible for reading command line parameters and connects to the Data Hub by reading your gradle project files.
16+
17+
## How do I Run this Example?
18+
19+
First you compile it.
20+
21+
`gradle installDist`
22+
23+
Then you launch it.
24+
25+
`./run.sh`
26+
27+
28+
## How do I add this to my existing Data Hub Project?
629

730
This is an example where the Data Hub Project artifacts have already been initialized. If you are wanting to add this ability to existing Data Hub Projects then you simply need to modify the build.gradle file.
831

@@ -35,9 +58,10 @@ mainClassName = "com.marklogic.spring.batch.hub.HubJobRunner"
3558
3659
```
3760

38-
# How to Run
61+
Then drop in your custom Java Config class in src/main/java/.....
3962

40-
gradle installDist
63+
Next you simply Compile your code with `gradle installDist`.
4164

42-
./run.sh
65+
Then you can run take a look at the [run.sh script](https://github.com/marklogic/marklogic-data-hub/blob/develop/examples/spring-batch/run.sh) to see how to run your custom config.
4366

67+
Note that this is not the only way to run it. It's merely the easiest. Java Ninjas can directly call the main() function of the [HubJobRunner class](https://github.com/marklogic/marklogic-data-hub/blob/develop/marklogic-data-hub/src/main/java/com/marklogic/spring/batch/hub/HubJobRunner.java). Or you can make your own class to start up Spring Batch by reading the HubJobRunner code and doing something similar.

0 commit comments

Comments
 (0)