Efficient way of waiting until all goroutine workers have completed processing

[sandykellagher]

We are using ants as a convenient Goroutine worker pool, that allows us to make requests in parallel with a limit on the maximum number of parallel Goroutines at any one time.

In some cases, it would be helpful to know when the set of workers that we have fired off via Submit() calls have all completed their processing. By analogy, using a simple sync.WaitGroup wg, we can call wg.Wait().

At the moment, the only answer I can see from the ants API is to poll the Running() method (eg every 10 ms) until it returns zero. But that seems a bit clumsy.

I guess we could still use a sync.WaitGroup in combination with ants, eg

var wg sync.WaitGroup
for <someListOfTasks> {
  wg.Add(1)
  pool.Submit(func(){
     <doSomeProcessing>
     wg.Done()
  })
}

wg.Wait()

But it'd be more elegant if I could just call a poolWait() function

[panjf2000]

You can call Pool.ReleaseTimeout to wait for all running workers to exit and release the pool. Create a new pool or call Pool.Reboot for the next batch.

Alternatively, you can define a wrapper that integrates ants.Pool with sync.WaitGroup, which shouldn't incur too much cost.