Decode_n_tokens yield eos/eot token

[nlpfollower]

Seems like we should yield the cur_token/next_token when it's an eos or eot so that it is included in the generator_func in chat. Also this way start_pos is incremented to an empty position in the cache. Repro similar to #1462, although there is no visible response degradation.

I've pushed logs from my trace (nlpfollower#2). You can see how without this yield the first id in the input_pos at the beginning of a round overlaps, and overwrites, the last cache entry made in the previous turn.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1474

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit aef0b8b with merge base 42c52bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

Thanks for another fix; I'll try to verify over the weekend

[Jack-Khuu]

My guts says this is legit, I'll give it another look