discussion feedback

Author: andy-neuma

README.md

@@ -2,15 +2,13 @@
 
 This repository contains configurations for model validation.
 
-The `accuracy` folder contains YAML files for each model that configures information needed for the model to be validated through the [llm-eval-test](https://github.com/openshift-psap/llm-eval-test). There are 4 config files for each model:
+The configs are organized by model. Each top level folder should be named after the model stub and contain `accuracy` and `performance` sub-folders. Both folders contain configs specific to the activity. Shared configs, e.g. `storage.yml` should be in the common area.
 
+The `accuracy` folder contains YAML configs needed for the model to be validated through the [llm-eval-test](https://github.com/openshift-psap/llm-eval-test). There are 3 config files for each model:
 * server.yml: contains settings to start a vllm server with the model
 * client.yml: contains settings for the llm-eval-test harness for the model
 * accuracy.yml: contains evaluation tasks and accuracy expectations for the model
-* storage.yml: specifies where mode and dataset is located
-
-The `benchmark` folder contains YAML files for each model that configures information needed for the model to be validated through the [guidellm](https://github.com/neuralmagic/guidellm). There are  config files for each model:
 
+The `benchmark` folder contains YAML configs needed for the model to be validated through the [guidellm](https://github.com/neuralmagic/guidellm). There are 3 config files for each model:
 * server.yml: contains settings to start a vllm server with the model
 * client.yml: contains settings for the guidellm for the model
-* storage.yml: specifies where mode and dataset is located

meta-llama/Llama-3.1-8B-Instruct/storage.yml

@@ -1,3 +1,3 @@
-# storage config for Meta-Llama-3.1-8B-Instruct
-model: meta-llama/Meta-Llama-3.1-8B-Instruct
+# storage configs for https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct
+model: hf
 data: hf