Parallel solver

[smkatz12]

The current parallel solver is on a separate branch and does not work with the newer releases of POMDPs.jl. It would be helpful to have a parallel solver for solving problems with large state spaces.

[mykelk]

Which branch is the best one to start with here?

[smkatz12]

I think the parallel1.0 branch.

[MaximeBouton]

I am unsure if it is parallel1.0 or mb/parallel, I think mb/parallel got almost merged and was an attempt to improve parallel1.0. You can read the discussion here: #18

The main challenge was accessing the states. The implementation is using SharedArrays but it might not be the best option, because storing the whole state space in a sharedarray becomes quickly impractical.

I also used this opt_pmap branch: https://github.com/JuliaPOMDP/DiscreteValueIteration.jl/blob/opt_pmap/src/parallel.jl, where I introduced a new function ind2state to be able to reconstruct the state variable from the index and not having to store the whole state space but just state indices.

[mykelk]

Wow, there is a lot of stuff there. Should we move this into a separate package, e.g., ParallelDiscreteValueIteration.jl to partition much of this complexity?

[zsunberg]

I think making a new package would be reasonable. My strongest advice is to utilize existing optimized parallel/distributed linear algebra as much as possible. Calling `transition` and translating back an forth between indices and state representations is really slow. (see also #44) On the other hand, it might not be possible to fit things in memory.

…

On Thu, Apr 1, 2021 at 12:02 PM Mykel Kochenderfer ***@***.***> wrote: Wow, there is a lot of stuff there. Should we move this into a separate package, e.g., ParallelDiscreteValueIteration.jl to partition much of this complexity? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#46 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABALI2ZELMGT6W3EKGVKWJ3TGSYHTANCNFSM42HKDFVQ> .

[mykelk]

Sounds good. @zsunberg: Do you have a student eager to do this? Or should I find one here?

[zsunberg]

I didn't have anyone in mind; if you have someone, that would be great. It is probably worth having a 10 minute conversation with me before implementing it rather than just diving in blindly.

…

On Thu, Apr 1, 2021 at 12:16 PM Mykel Kochenderfer ***@***.***> wrote: Sounds good. @zsunberg <https://github.com/zsunberg>: Do you have a student eager to do this? Or should I find one here? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#46 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABALI2YYEI47YI72UOUMAGDTGSZ5VANCNFSM42HKDFVQ> .