[Question] A question about the cost function of the p3o algorithm

[Liqinyan821]

Required prerequisites

• I have read the documentation https://omnisafe.readthedocs.io.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Questions

Hello Omnisafe team, thank you very much for your contribution.
When I was Learning the p3o algorithm, I found that the def _loss_pi_cost function was not clip, and loss_pi_cost in the P3O Optimization for Safe Reinforcement Learning used clip.

[Gaiejj]

You must be a very meticulous person! In fact, this is a trick we discovered while debugging the algorithm, which makes P3O more suitable for high-dimensional complex environments. Have you tried removing the clip? Do you have any experimental data? If it performs well without it, we will modify this implementation later.

[sunjingyan204-collab]

Hello Omnisafe team：

Thank you for your contributions!

I have another question regarding the implementation of the cost function in the P3O algorithm. In the original paper

However, in the code implementation, a direct summation approach is used in place of the original J_C(π). This results in an excessively large cost policy loss during the training of my 6-DOF robotic arm task.

My question is: Should this aspect be improved to better align with the theoretical formulation?