Hi! We’re running HELMET via eval.py and hit a crash before evaluation starts due to an argparse attribute mismatch.
Repro:
- Checkout commit a62ef7d (may also affect nearby commits).
- Run:
python eval.py --config configs/cite.yaml --model_name_or_path meta-llama/Llama-3.2-1B --use_vllm_serving --endpoint_url http://127.0.0.1:8000/v1/ --api_key EMPTY --overwrite --no_cuda
Observed:
eval.py parses stop_new_line (so the Namespace has stop_new_line), but model_utils.py calls load_LLM(args) and tries to access args.stop_newline, raising:
AttributeError: 'Namespace' object has no attribute 'stop_newline'. Did you mean: 'stop_new_line'?
Expected:
Evaluation should run without crashing.