[etLLM][Config, Part1] Convert Args to DictConfig #9450
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9450
Note: Links to docs will display an error until the docs builds have been completed. ❌ 4 New FailuresAs of commit bccc2e3 with merge base 76ae537 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "topic: not user facing" |
facebook-github-bot
added
the
CLA Signed
|
@iseeyuan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
iseeyuan
force-pushed
the
dictconfig
branch
2 times, most recently
from
6df85c1 to
311e925
Compare
|
@iseeyuan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@iseeyuan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| config.eval = OmegaConf.create() | ||
| config.eval.tasks = args.tasks | ||
| config.eval.limit = args.limit | ||
| config.eval.num_fewshot = args.num_fewshot | ||
| config.eval.pte = args.pte | ||
| config.eval.tokenizer_bin = args.tokenizer_bin | ||
| config.eval.output_eager_checkpoint_file = args.output_eager_checkpoint_file |
There was a problem hiding this comment.
Sorry is there a definition of the config?
There was a problem hiding this comment.
This PR is just the first step: to convert args to config to unblock internal work, where the configs can be used standalone, without cli args and yaml file. More context in #9449
|
@iseeyuan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| "output_dir": args.output_dir, | ||
| "checkpoint": args.checkpoint, | ||
| "checkpoint_dir": args.checkpoint_dir, | ||
| "tokenizer_path": args.tokenizer_path, |
There was a problem hiding this comment.
This is only needed during export for quant calibration, should go in args_dict["calibration"]. Or since these args are also used in different runners, I'd prefer to have args_dict["tokenizer"], we can combine it with tokenizer_config_path which the eager runner uses
| args_dict["kv_cache"] = { | ||
| "use_kv_cache": args.use_kv_cache, | ||
| "quantize_kv_cache": args.quantize_kv_cache, | ||
| "use_sdpa_with_kv_cache": args.use_sdpa_with_kv_cache, |
There was a problem hiding this comment.
This arg is a bit poorly named now that the custom sdpa op is now decoupled from the kv cache, should move this to arcs_dict["misc"]. We should rename this arg @kimishpatel
Summary: 1. Add structure to the complicated args 2. Convert args to DictConfig, to decouple the cli args 3. Pass needed sub configs to functions (instead of args) Test Plan: ``` python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp -E 8,64 ``` Reviewed By: larryliu0820 Differential Revision: D71557301 Pulled By: jackzhxng
Summary: Pull Request resolved: #9717 1. Add structure to the complicated args 2. Convert args to DictConfig, to decouple the cli args 3. Pass needed sub configs to functions (instead of args) Pull Request resolved: #9450 Test Plan: ``` python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp -E 8,64 ``` Reviewed By: larryliu0820 Differential Revision: D71557301 Pulled By: jackzhxng
Summary: 1. Add structure to the complicated args 2. Convert args to DictConfig, to decouple the cli args 3. Pass needed sub configs to functions (instead of args) Test Plan: ``` python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp -E 8,64 ``` Pulled By: jackzhxng jackzhxng Differential Revision: D71557301
Summary: 1. Add structure to the complicated args 2. Convert args to DictConfig, to decouple the cli args 3. Pass needed sub configs to functions (instead of args) Test Plan: ``` python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp -E 8,64 ``` Pulled By: jackzhxng jackzhxng Differential Revision: D71557301
Summary: 1. Add structure to the complicated args 2. Convert args to DictConfig, to decouple the cli args 3. Pass needed sub configs to functions (instead of args) Test Plan: ``` python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp -E 8,64 ``` Pulled By: jackzhxng jackzhxng Differential Revision: D71557301
Summary: 1. Add structure to the complicated args 2. Convert args to DictConfig, to decouple the cli args 3. Pass needed sub configs to functions (instead of args) Test Plan: ``` python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp -E 8,64 ``` Reviewed By: larryliu0820 Pulled By: jackzhxng jackzhxng Differential Revision: D71557301
|
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
|
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
Summary
Test plan