Line break missing in Jinja Template

[abse4411]

I noticed that the prompt Template of the paper contains a line break at the end,

but the actual inference code will remove the line break by default when using the Jinja Template:

# obtain prompt
def get_prompt(data_i):
    return Template(system_prompts).render(problem = data_i["question"])

This can make a difference in the reproducibility of the experiment.
So I set keep_trailing_newline=True in the code to prevent this problem:

# obtain prompt
def get_prompt(data_i):
    return Template(system_prompts, keep_trailing_newline=True).render(problem = data_i["question"])

Here's my experimental results on the MATH500 dataset (using TraDo-8B-Instruct) :

Whether to remove the newline character	ACC
No	75.46666666666667
Yes	74.86666666667

The experimental results with end-of-line breaks (75.46...) are very close to the reported value (75.5) ! I have created a PR to fix it： #39

[yinjjiew]

Good catch! I will check it soon~