Add/Remove concurrent workers dynamically

[Kikobeats]

Hello,

I'm trying to add an autobalance feature for add/remove concurrents calls per workers automatically when the CPU is idle or saturate (respectively).

See more information in: Kikobeats/farm-cli#12 😄

I reviewed the code for know the current code approach for that and I saw this:

if (this.children[childId].activeCalls < this.options.maxConcurrentCallsPerWorker
        && this.children[childId].calls.length < this.options.maxCallsPerWorker) {
      this.send(childId, this.callQueue.shift())

so for add an extra worker I can use .send with the child id and generate an extra calllQueue. Fine!

My problematic is how to kill this concurrent workers.

In the code you use .stopChild but this call all the worker; I need kill just one (the last) concurrent call in the worker.

I tried to code a mockup for that, something like:

function killLastConcurrentCall (farm) {
  Object.keys(farm.children).forEach(function (childId) {
    const calls = farm.children[childId].calls
    const call = calls[calls.length -1]
    const idx = calls.length - 1

    farm.receive({
      idx,
      child: childId,
      args: [ new ProcessTerminatedError('cancel by autobalance') ]
    })
  })
}

But doesn't worker. Any idea how I can handle this feature from the code side?

[Kikobeats]

Related:

• How to limit job CPU time #7
• process.send is async and I can't guarantee IPC messages get sent before process.exit nodejs/node#6767
• https://medium.freecodecamp.com/node-js-child-processes-everything-you-need-to-know-e69498fe970a

[rvagg]

so sorry @Kikobeats for the 2 year delay, I hope you managed to find answers for this, feel free to re-open if this is an ongoing concern although I can't promise I can get my head into the right space!

[Kikobeats]

Hey, I still interested that for adding it into cli level, any idea is welcome