Add an interface for LLM runner

[kirklandsign]

In case we have custom LLM runners other than llama runner, we want to have a uniform interface

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6356

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit bc1f420 with merge base 2c43190 ():

NEW FAILURES - The following jobs have failed:

• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 548447d3d2b595a2f10ad522a1d448b50c54f51e32b6de3958b553f2cefb7860 /exec failed with exit code 127
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 1fe6fcfdc88492b6661989f861692117fc33877ae6d1d486fdc5987301f0e379 /exec failed with exit code 127
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 9fd09de3699fa65c48e89af7e6525740b845a1adf8c380eb39c62d58fd5777a2 /exec failed with exit code 127

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[larryliu0820]

Maybe call it IRunner? Also what is the actual usage of having this interface? Can you also include the changes you want to make, to reduce duplicate code?

[kirklandsign]

Maybe call it IRunner
Works for me

Actual usage: https://github.com/pytorch/executorch/pull/6208/files#diff-344496b5b78dc04421b5eddcb0f261269c3150e122a9be63676da565955cb4f7R123-R131

That code is not ready yet

[larryliu0820]

This can have a default implementation right?

[kirklandsign]

Maybe not.

executorch/examples/models/llama/runner/runner.cpp

Lines 62 to 65 in 516a417

	bool Runner::is_loaded() const {
	return module_->is_loaded() && tokenizer_ && text_decoder_runner_ &&
	text_prefiller_ && text_token_generator_;
	}

it's quite impl detail

[larryliu0820]

Please fix the CI failures

[facebook-github-bot]

@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@kirklandsign merged this pull request in 4d7b294.