Integration test mega-issue

[ebsmothers]

We currently have a bunch of integration tests, but none of them run (because none of them work) in CI. This issue tracks tests that need to be fixed along with new ones that need to be added and their respective owners.

Existing tests

• test_coder.py @allenwang28
• test_vllm_policy_correctness.py @joecummings
• test_policy_update.py @pbontrager
• test_titan_fwd_vs_hf_fwd.py @joecummings

Tests to be added

• E2E GRPO run on Qwen 0.6B model @JenniferWang @Jack-Khuu
• E2E SFT run on Qwen 0.6B model @daniellepintz
  ... (please edit the task to add more that I've missed)

[JenniferWang]

I am looking at `test_vllm_policy_correctness.py

I am confused why do you think it's not working in CI?

[ebsmothers]

@JenniferWang at least when I run on my local I get the error

Failed: async def functions are not natively supported.

So maybe we just need something like @pytest.mark.asyncio on those tests?

Update: OK yeah it was exactly that. Since I was already 90% of the way there just testing the hypothesis went ahead and opened the PR too: #416

[JenniferWang]

Re-thinking test_policy_update.py

That test, by itself, is not as useful in CI -- the main functionality is sanity check if the weight tensors sharding is correct given a config, which is not useful for a tiny model on CI.

We can cover the integration part in an E2E GRPO Qwen 0.6B model.