Current implementation make equal chunks and send for scan. But if one chunk takes more time to complete the job all other machine sits idle. There is an option -d to delete upon completion but this is not helpful if some chunk requires more time.
Instead of making static chunk like lines_count(input_file)/len(fleet), dynamically distributing using a loop by sending few lines for fleet be helpful.
Current implementation make equal chunks and send for scan. But if one chunk takes more time to complete the job all other machine sits idle. There is an option -d to delete upon completion but this is not helpful if some chunk requires more time.
Instead of making static chunk like lines_count(input_file)/len(fleet), dynamically distributing using a loop by sending few lines for fleet be helpful.