Just wanted to highlight this approach to a more long term stable planning agent (source code exists and URL is given in the abstract):

https://arxiv.org/abs/2305.14909

Another approach worth experimenting with for the planner agent: https://arxiv.org/pdf/2307.07696