How i can split a SQL query in multiple tasks ?!

[ricardojlrufino]

I would like to know if this project can meet the following use case.

Today I have a project that has a scheduler, which queries the database, and for each line it has to make an HTTP call.

I would like to parallelize this across several PODs (in Kubernets), however the first problem would be that @schedule needs to run in only 1 of the PODs to avoid conflicts, then it creates tasks to be executed in the other PODs

To solve the problem of running only 1 schedule at a time, I think about creating a Lock (https://github.com/coditory/sherlock-distributed-lock), and then creating the tasks separating the database query using a Limit.
But there would only be one problem that I would have to release the lock when all X tasks have finished.

How could I do that?

[kagkarlsson]

Yeah db-scheduler can support such a use-case.

• Use a RecurringTask to regularly poll the table (it uses a distributed lock under the hood, so it will not run in multiple pods, but you cannot control which pod it runs in)
• Scheduler a OneTime task for each of the calls that should be parallellized. All Scheduler-instances will compete to execute these ( there is no guarantee that they will be evenly distributed)

Here is an example of such a spawner task
(of course also possible using spring boot starter)