|
| 1 | +ifdef::env-github[] |
| 2 | +:artifactId: batch-processing |
| 3 | +endif::[] |
| 4 | + |
| 5 | +Unresolved directive in README-source.adoc - include::../shared-doc/attributes.adoc[] |
| 6 | + |
| 7 | += batch-processing: Chunk oriented Batch 1.0 processing |
| 8 | +:author: Rafael Benevides |
| 9 | +:level: Intermediate |
| 10 | +:technologies: CDI, Batch 1.0, JSF |
| 11 | +:openshift: true |
| 12 | + |
| 13 | +[abstract] |
| 14 | +The `batch-processing` quickstart shows how to use chunk oriented batch jobs to import a file to a database. |
| 15 | + |
| 16 | +:standalone-server-type: default |
| 17 | +:archiveType: war |
| 18 | +:uses-h2: |
| 19 | + |
| 20 | +== What is it? |
| 21 | + |
| 22 | +This quickstart simulates a file importation using batch jobs. To make it easy, this quickstart offers the user a way to generate files. The generated file can have its name and the number of records customized. The user can also specify if the file contains an error or not. |
| 23 | + |
| 24 | +The job contains two tasks: |
| 25 | + |
| 26 | +. It imports the file using a chunk oriented approach. |
| 27 | + |
| 28 | +* The chunk size is set to `3`. |
| 29 | +* The `RecordsReader` is responsible for parsing the file and create an instance of `Contact`. |
| 30 | +* The `ContactsFormatter` applies the proper case to the contact name and it also applies a mask to the phone number. |
| 31 | +* Finally, `ContactsPersister` sends the contact instance to the database. |
| 32 | + |
| 33 | +. It logs the number of records imported. |
| 34 | + |
| 35 | +The database schema defines that the column for name is unique. For that reason, any attempt to persist a duplicate value will throw an exception. On the second attempt to run the job, the `ChunkCheckpoint` provides information to skip the contacts that were already persisted. |
| 36 | + |
| 37 | +// Link to the quickstart source |
| 38 | +Unresolved directive in README-source.adoc - include::../shared-doc/view-the-source.adoc[leveloffset=+1] |
| 39 | +// Considerations for Use in a Production Environment |
| 40 | +Unresolved directive in README-source.adoc - include::../shared-doc/development-shortcuts.adoc[leveloffset=+1] |
| 41 | +// System Requirements |
| 42 | +Unresolved directive in README-source.adoc - include::../shared-doc/system-requirements.adoc[leveloffset=+1] |
| 43 | +// Use of {jbossHomeName} |
| 44 | +Unresolved directive in README-source.adoc - include::../shared-doc/use-of-jboss-home-name.adoc[leveloffset=+1] |
| 45 | +// Start the {productName} Standalone Server |
| 46 | +Unresolved directive in README-source.adoc - include::../shared-doc/start-the-standalone-server.adoc[leveloffset=+1] |
| 47 | +// Build and Deploy the Quickstart |
| 48 | +Unresolved directive in README-source.adoc - include::../shared-doc/build-and-deploy-the-quickstart.adoc[leveloffset=+1] |
| 49 | + |
| 50 | +== Access the Application |
| 51 | + |
| 52 | +Access the running application in a browser at the following URL: http://localhost:8080/{artifactId}/ |
| 53 | + |
| 54 | +You are presented with a simple form that allows you to generate sample files to be imported. |
| 55 | + |
| 56 | +=== Usage 1: Import the file without any errors |
| 57 | + |
| 58 | +. Click on *Generate a new file and start import job* button. This generates a new file with 10 unique records to be imported. The import job starts after the file is generated. |
| 59 | + |
| 60 | +. A table is displayed containing information about the task that was started. Click on the *Update jobs list* button and verify that the job was completed. |
| 61 | + |
| 62 | +. Investigate the console output. It shows that files with 10 records were processed, 3 records at a time. |
| 63 | ++ |
| 64 | +[source,options="nowrap"] |
| 65 | +---- |
| 66 | +INFO [org.jboss.as.quickstarts.batch.controller.BatchController] (default task-3) Starting to generate 10 in file /var/folders/j8/63sgdmbn5tqdkyw0tz6df53r0000gn/T/temp-file.txt |
| 67 | +INFO [org.jboss.as.quickstarts.batch.controller.BatchController] (default task-3) File generated at /var/folders/j8/63sgdmbn5tqdkyw0tz6df53r0000gn/T/temp-file.txt |
| 68 | +INFO [org.jboss.as.quickstarts.batch.job.listener.JobListener] (Batch Thread - 1) Job import-file - Execution #1 starting. |
| 69 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsPersister] (Batch Thread - 1) No checkpoint detected. Cleaning the Database |
| 70 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #1 - Changing name ZIqYKITxiM -> Ziqykitxim | phone 978913851 -> (978)-913-851 |
| 71 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #2 - Changing name JbHjnaThps -> Jbhjnathps | phone 095108018 -> (095)-108-018 |
| 72 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #3 - Changing name FJTlXRtCdR -> Fjtlxrtcdr | phone 286847939 -> (286)-847-939 |
| 73 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Preparing to persist 3 contacts |
| 74 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Persisting 3 contacts |
| 75 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #4 - Changing name mlmBABWzfL -> Mlmbabwzfl | phone 744478648 -> (744)-478-648 |
| 76 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #5 - Changing name jVlTYiBRMP -> Jvltyibrmp | phone 135063841 -> (135)-063-841 |
| 77 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #6 - Changing name DwEFbSjfQE -> Dwefbsjfqe | phone 404572175 -> (404)-572-175 |
| 78 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Preparing to persist 3 contacts |
| 79 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Persisting 3 contacts |
| 80 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #7 - Changing name niDXWwGJuQ -> Nidxwwgjuq | phone 949448390 -> (949)-448-390 |
| 81 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #8 - Changing name VZBArfowSe -> Vzbarfowse | phone 902370961 -> (902)-370-961 |
| 82 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #9 - Changing name aSpyWCWwje -> Aspywcwwje | phone 246977695 -> (246)-977-695 |
| 83 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Preparing to persist 3 contacts |
| 84 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Persisting 3 contacts |
| 85 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 1) Register #10 - Changing name TofTfbRBzI -> Toftfbrbzi | phone 868339088 -> (868)-339-088 |
| 86 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Preparing to persist 1 contacts |
| 87 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 1) Persisting 1 contacts |
| 88 | +INFO [org.jboss.as.quickstarts.batch.job.ReportBatchelet] (Batch Thread - 1) Imported 10 to Database |
| 89 | +INFO [org.jboss.as.quickstarts.batch.job.listener.JobListener] (Batch Thread - 1) Job import-file - Execution #1 finished. Status: COMPLETED |
| 90 | +---- |
| 91 | + |
| 92 | +=== Usage 2: Import an error file and fix it |
| 93 | + |
| 94 | +Now you will simulate a file with duplicate records. This will raise an exception and stop the processing. After that, you will fix the file and continue the importing where it stopped. |
| 95 | + |
| 96 | +. Mark the *Generate a duplicate record* checkbox and click on *Generate a new file and start import job* button. If you click on *Update jobs list* button, you will see that the job failed with the following Exit Status: `Error : org.hibernate.exception.ConstraintViolationException: could not execute statement`. This was caused because the job tried to insert a duplicate record at the Database. You will also see `org.h2.jdbc.JdbcSQLException: Unique index or primary key violation` exception stacktraces in the server log. |
| 97 | + |
| 98 | +. Next, you will fix the file and restart that job execution. Uncheck the *Generate a duplicate record* checkbox and click on *Generate a new file* button. This will generate the file without errors. |
| 99 | + |
| 100 | +. Click on *Restart* button in the last column for that job instance in the *List of Jobs* table. If you click on *Update jobs list* button, you will see that the job was completed. |
| 101 | + |
| 102 | +. Review the server logs and verify that the job started from the last checkpoint. |
| 103 | ++ |
| 104 | +[source,options="nowrap"] |
| 105 | +---- |
| 106 | +INFO [org.jboss.as.quickstarts.batch.job.RecordsReader] (Batch Thread - 3) Skipping to line 3 as marked by previous checkpoint |
| 107 | +---- |
| 108 | + |
| 109 | +. Investigate the console output. |
| 110 | ++ |
| 111 | +[source,options="nowrap"] |
| 112 | +---- |
| 113 | +INFO [org.jboss.as.quickstarts.batch.job.listener.JobListener] (Batch Thread - 3) Job import-file - Execution #3 starting. |
| 114 | +INFO [org.jboss.as.quickstarts.batch.job.RecordsReader] (Batch Thread - 3) Skipping to line 3 as marked by previous checkpoint |
| 115 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #4 - Changing name HdeqwzEjbA -> Hdeqwzejba | phone 686417040 -> (686)-417-040 |
| 116 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #5 - Changing name veEEbtpYTJ -> Veeebtpytj | phone 367981821 -> (367)-981-821 |
| 117 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #6 - Changing name bQIKTUyqMW -> Bqiktuyqmw | phone 103363182 -> (103)-363-182 |
| 118 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 3) Preparing to persist 3 contacts |
| 119 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 3) Persisting 3 contacts |
| 120 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #7 - Changing name KVLIGXhCry -> Kvligxhcry | phone 117327691 -> (117)-327-691 |
| 121 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #8 - Changing name PBAZgernHy -> Pbazgernhy | phone 066203468 -> (066)-203-468 |
| 122 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #9 - Changing name DGtNZdteGB -> Dgtnzdtegb | phone 908779587 -> (908)-779-587 |
| 123 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 3) Preparing to persist 3 contacts |
| 124 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 3) Persisting 3 contacts |
| 125 | +INFO [org.jboss.as.quickstarts.batch.job.ContactsFormatter] (Batch Thread - 3) Register #10 - Changing name mhmIHhZMhv -> Mhmihhzmhv | phone 094518410 -> (094)-518-410 |
| 126 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 3) Preparing to persist 1 contacts |
| 127 | +INFO [org.jboss.as.quickstarts.batch.job.listener.PersistListener] (Batch Thread - 3) Persisting 1 contacts |
| 128 | +WARN [org.jberet] (Batch Thread - 3) JBERET000018: Could not find the original step execution to restart. Current step execution id: 0, step name: reportBatchelet |
| 129 | +INFO [org.jboss.as.quickstarts.batch.job.ReportBatchelet] (Batch Thread - 3) Imported 10 to Database |
| 130 | +INFO [org.jboss.as.quickstarts.batch.job.listener.JobListener] (Batch Thread - 3) Job import-file - Execution #3 finished. Status: COMPLETED |
| 131 | +---- |
| 132 | + |
| 133 | +=== Usage 3: Import an error file and do not fix the errors |
| 134 | + |
| 135 | +. Check the *Generate a duplicate record* checkbox and click on *Generate a new file ans start import job* button. If you click on *Update jobs list* button, you will see that the job failed with the following Exit Status: `Error : org.hibernate.exception.ConstraintViolationException: could not execute statement`. This was caused because you tried to insert a duplicate record at the Database. |
| 136 | + |
| 137 | +. This time you will not fix the file. Just click on *Restart* button again. If you click on *Update jobs list* button, you will see that the job was marked as `ABANDONED` this time because it was restarted once. Notice that there is a new parameter: `restartedOnce=true`. This behavior was implemented in `JobListener` for demonstration purposes, to prevent a `FAILED` job from being restarted twice. |
| 138 | + |
| 139 | +// Server Distribution Testing |
| 140 | +Unresolved directive in README-source.adoc - include::../shared-doc/run-integration-tests-with-server-distribution.adoc[leveloffset=+2] |
| 141 | +// Undeploy the Quickstart |
| 142 | +Unresolved directive in README-source.adoc - include::../shared-doc/undeploy-the-quickstart.adoc[leveloffset=+2] |
| 143 | + |
| 144 | +// Build and run sections for other environments/builds |
| 145 | +Unresolved directive in README-source.adoc - include::../shared-doc/build-and-run-the-quickstart-with-provisioned-server.adoc[leveloffset=+1] |
| 146 | +Unresolved directive in README-source.adoc - include::../shared-doc/build-and-run-the-quickstart-with-openshift.adoc[leveloffset=+1] |
| 147 | +ifndef::ProductRelease,EAPXPRelease[] |
| 148 | +// Kubernetes |
| 149 | +Unresolved directive in README-source.adoc - include::../shared-doc/build-and-run-the-quickstart-with-kubernetes.adoc[leveloffset=+1] |
| 150 | +endif::[] |
0 commit comments