Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .ci/scripts/run-docs
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,6 @@ if [ "$1" == "advanced" ]; then
fi

if [ "$1" == "evaluation" ]; then

exit 0

echo "::group::Create script to run evaluation"
python3 torchchat/utils/scripts/updown.py --file torchchat/utils/docs/evaluation.md --replace 'llama3:stories15M,-l 3:-l 2' --suppress huggingface-cli,HF_TOKEN > ./run-evaluation.sh
# for good measure, if something happened to updown processor,
Expand Down
7 changes: 6 additions & 1 deletion torchchat/utils/docs/evaluation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,13 @@

# Evaluation Features

<!--

[shell default]: ./install/install_requirements.sh

[shell default]: TORCHCHAT_ROOT=${PWD} ./torchchat/utils/scripts/install_et.sh

-->

Torchchat provides evaluation functionality for your language model on
a variety of tasks using the
Expand All @@ -14,7 +19,7 @@ library.

## Usage

The evaluation mode of `torchchat.py` script can be used to evaluate your language model on various tasks available in the `lm_eval` library such as "wikitext". You can specify the task(s) you want to evaluate using the `--tasks` option, and limit the evaluation using the `--limit` option. If no task is specified, it will default to evaluating on "wikitext".
The evaluation mode of `torchchat.py` script can be used to evaluate your language model on various tasks available in the `lm_eval` library such as "wikitext". You can specify the task(s) you want to evaluate using the `--tasks` option, and limit the evaluation using the `--limit` option. If no task is specified, the task will default to evaluating on "wikitext".

**Examples**

Expand Down
Loading