Decision Transformer Training Option

[oscarthf]

I would like to see if the PPO method is better than the Decision Transformer method of learning to maximize reward.

[sanagno]

Rewards are sparse in our case, so find it difficult to see how we can use this.

[oscarthf]

I imagined fine-tuning the language model to just predict responses given the meta data after the prompt but maybe the sparse labels prevent making a sum of the future reward over multiple responses.
Maybe there will be a growing subset if fully labeled responses though which you could more easily sum