Remove tokens per sec in aggregate_metrics when jit_compile

[yanbing-j]

This PR is to remove tokens per sec in aggregate_metrics when jit_compile, to make the average of tokens per sec more precise. The tokens per sec is too low when jit_compile, which should not be included in aggregate_metrics.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1378

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit fdc416d with merge base 5da240a ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[yanbing-j]

Hi @Jack-Khuu, could you please take a look? Thanks!

[Jack-Khuu]

Thanks for adding. Looks like the compile numbers got added back in (outside of the if) with 77774d2 due to Nan's

I rebased your PR since d7b681a added a Nan check that might actually make the old Nan problem a non-issue

[Jack-Khuu]

Can you add a print near around line ~1210 that tells the user that the averages were calculated with the compile sample excluded?

[yanbing-j]

the averages were calculated with the compile sample excluded

Updated. Please take a look again. @Jack-Khuu

[Jack-Khuu]

Thanks for the comment

Just one last nit: Let's only print the "excluding compile in calculations" message if the user is actually compiling, to not confuse users

Once that's ready give me a ping and I'll merge this in

[yanbing-j]

@Jack-Khuu Thanks for the merging!