Query flow high complexity

[kumavis]

found it a little difficult to follow the flow of an execution of a query
Mostly posting this to share notes and gather discussion

here is a async execution graph i wrote by hand

query.run
  run.execute(peers)
    run.executePaths
      paths.map(path => path.execute())
        path.initialPeers.map(peer => path.addPeerToQuery(peer))
          path.peersToQuery.enqueue
        run.workerQueue
          run.init
          run.startWorker
            worker.execute
              async/queue: worker.fill -> worker.stop
                worker.processNext(peer)
                  run.continueQuerying
                    run.peersQueried.anyCloser
                  worker.execQuery
                    path.queryFuncAsync
                    run.peersQueried.add(peer)
                    path.addPeerToQuery
    run.stop()
    run.emit('complete')

There's not very clear parent-child relationships between components and everything seems to have bi-directional references

this leads to this strange flow as it gets deeper

query -> run -> path -> run -> worker -> { run, path }

some ideas

• call run.init in run.execute to avoid the code for prevent multiple inits
• replace path.execute with a new path.init and then call run.workerQueue per path inside run.executePaths

this and a few other fixes could help simplify us down to

query -> run -> worker -> path

which feels like a more natural breakdown

[kumavis]

I'm not entirely sure why query and run are broken out. seems like 1 query always has 1 run, and queries are never re-run.

[kumavis]

heres a fun one
A Query's makePath and queryFn are:

• defined at Query instantiation time in KadDHT
• makePath is stored on Query
• makePath is called directly in Run
• queryFn is stored on Path
• queryFn is called directly in WorkerQueue

[vasco-santos]

After the latest changes of the go-libp2p-kad-dht, we intend to rewrite most of the this codebase once we have a spec from the go implementation. Once that is done, we will guarantee this will be get done.