Allow MongoDB creation scripts to be idempotent

[snicoll]

Right now, Spring Batch offers two ways to create or drop the necessary infrastructure to use Spring Batch with MongoDB.

Chatting with @mp911de, he's mentioning that some aspect of the initialization can be done on first request, while others can be made conditional. I don't what the best practices would be, but a feature where things are validated/created on startup would be quite useful for Spring Boot.

Right now, we have a way to create the store, or drop it. "create" isn't very useful in production so you'd switch it off. In dev you'd use create-drop, which isn't suitable when restarting the app and hacking on something.

In short, I am wondering if we could provide something in Spring Batch we could leverage. Doing so in Spring Boot is not an option and the lack of support would probably mean users will have to setup the schema manually.

[mp911de]

Indexes can be created directly through the MongoDB API as well as ensuring presence of the counter sequence. To avoid overwriting existing counter states and to make schema creation idempotent, we should use upserts with setOnInsert to only apply the initial value if the document doesn't exist yet, something along the lines of:

MongoCollection<org.bson.Document> collection = db.getCollection("BATCH_SEQUENCES");

collection.updateOne(new org.bson.Document("_id", "BATCH_JOB_INSTANCE_SEQ"),
		new org.bson.Document("$setOnInsert", new org.bson.Document("count", 0L)), 
		new UpdateOptions().upsert(true));

collection.createIndex(new org.bson.Document("jobName", 1), new IndexOptions().name("job_name_idx"));

[fmbenhassine]

We have a similar feature request to make JDBC tables creation scripts idempotent as well: #1855 (IIRC, the only issue was that not all db servers support the "IF NOT EXIST" clause).

But I am open to improve the jdbc/mongodb scripts provided by Spring Batch and make them idempotent.

a feature where things are validated/created on startup would be quite useful for Spring Boot.

I have created a feature request for this here: #4865

[answndud]

I’d like to work on this one. I’ll update the MongoDB creation scripts to be idempotent, using upserts for sequence initialization and safe index creation, and I’ll add regression coverage to verify the scripts can be applied more than once without resetting state.

[answndud]

I opened PR #5339 with a proposed fix and regression test for the MongoDB schema scripts.