eval doc does not pass test

[mikekgfb]

🐛 Describe the bug

#1383 enables run-docs evaluation to extract a test script from eval documentation,
to run evaluation script. In turn, this extracts the command

python3 torchchat.py eval stories15M --tasks wikitext --limit 10

from the eval doc as a test to ensure that the doc is in fact correct. This appears to be a correct use of eval to me, yet it fails when running as follows:

https://hud.pytorch.org/pr/pytorch/torchchat/1383#33154706429

2024-11-18T18:13:35.1710781Z + python3 torchchat.py eval stories15M --tasks wikitext --limit 10
2024-11-18T18:13:35.1711201Z NumExpr defaulting to 16 threads.
2024-11-18T18:13:35.1711531Z PyTorch version 2.6.0.dev20241002+cu121 available.
2024-11-18T18:13:35.1711768Z 
2024-11-18T18:13:35.1711939Z Downloading builder script:   0% 0.00/5.67k [00:00<?, ?B/s]
2024-11-18T18:13:35.1712401Z Downloading builder script: 100% 5.67k/5.67k [00:00<00:00, 37.1MB/s]
2024-11-18T18:13:35.1712808Z Traceback (most recent call last):
2024-11-18T18:13:35.1713182Z   File "/pytorch/torchchat/torchchat.py", line 100, in <module>
2024-11-18T18:13:35.1713552Z     eval_main(args)
2024-11-18T18:13:35.1713905Z   File "/pytorch/torchchat/torchchat/usages/eval.py", line 238, in main
2024-11-18T18:13:35.1714340Z     builder_args = BuilderArgs.from_args(args)
2024-11-18T18:13:35.1714667Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-18T18:13:35.1715101Z   File "/pytorch/torchchat/torchchat/cli/builder.py", line 169, in from_args
2024-11-18T18:13:35.1715520Z     return cls(
2024-11-18T18:13:35.1715827Z     run_cmd_or_die(f"docker exec -t {container_name} /exec")
2024-11-18T18:13:35.1716580Z   File "/home/ec2-user/actions-runner/_work/torchchat/torchchat/test-infra/.github/scripts/run_with_env_secrets.py", line 39, in run_cmd_or_die
2024-11-18T18:13:35.1717388Z     raise RuntimeError(f"Command {cmd} failed with exit code {exit_code}")
2024-11-18T18:13:35.1718153Z RuntimeError: Command docker exec -t c2e4cff2805edb5848301b09ed712578d726414222642162007e0e16e7c48ba1 /exec failed with exit code 1
2024-11-18T18:13:35.1718786Z            ^^^^
2024-11-18T18:13:35.1719026Z   File "<string>", line 24, in __init__
2024-11-18T18:13:35.1719475Z   File "/pytorch/torchchat/torchchat/cli/builder.py", line 76, in __post_init__
2024-11-18T18:13:35.1719926Z     raise RuntimeError(
2024-11-18T18:13:35.1720431Z RuntimeError: need to specified a valid checkpoint path, checkpoint dir, gguf path, DSO path, or PTE path

Versions

github runner, environment as configured by pytorch test infra

[Jack-Khuu]

@vmpuri Can you take a look?

Maybe the doc is outdated

[mikekgfb]

Resolved by landing #1383