Questions about vicuna_v1.1_reverse/seed/aug

[riddlevv]

Hello! When I'm running your code, I found that the template vicuna_v1.1_reverse/seed/aug haven't been provided in fastchat. So I write them by myself, I'm not sure whether they are right and whether the seperator I defined in the template are compatible with the code used to mask the input token, or should change some code when using SeparatorStyle.ADD_COLON_TWO?

This is my vicuna_v1.1_seed:

register_conv_template(
    Conversation(
        name="vicuna_v1.1",
        system_message="**Answer in the style of an AI Assistant**"
        roles=("human", "gpt"),
        messages=(),
        offset=0,
        sep_style=SeparatorStyle.ADD_COLON_TWO,
        sep="\n",
        sep2="</s>",
    )
)

Can you provide your template(or you have provided but I didn't find? Sorry). I would really appreciate that coz it would help me a lot. And you definitely have a better understanding for the code than me.

Thank u very much.

Best regards.
Sean

[riddlevv]

Oh I've found them🤣 Sorry, just ignore this issue.

[riddlevv]

Here's a problem, the template for augmentation data should be "with knowledge from web search", the template for aug and seed should be replaced with each other.

register_conv_template(
    Conversation(
        name="vicuna_v1.1_aug",
        system_message=(
            "This is a chat between a curious user and a helpful artificial intelligence assistant. "
            "Given the assistant's reponse, please predict the user's instruction in the style of an AI Assistant."
        ),
        roles=("USER", "ASSISTANT"),
        messages=(),
        offset=0,
        sep_style=SeparatorStyle.ADD_COLON_TWO,
        sep=" ",
        sep2="</s>",
    )
)

register_conv_template(
    Conversation(
        name="vicuna_v1.1_seed",
        system_message=(
            "This is a chat between a curious user and a helpful artificial intelligence assistant. "
            "The assistant gives helpful, detailed, and polite answers to the user's questions with knowledge from web search."
        ),
        roles=("USER", "ASSISTANT"),
        messages=(),
        offset=0,
        sep_style=SeparatorStyle.ADD_COLON_TWO,
        sep=" ",
        sep2="</s>",
    )
)

[Spico197]

Hi there, sorry for the late response.

❤️ Yes you are right, sorry for the mismatched system prompts. I've swapped them in the latest commit.

[stamina1121]

您好，很抱歉回复晚了。

❤️ 是的，您是对的，很抱歉系统提示不匹配。我已经在最新的提交中交换了它们。

您好！请问这样现在交换模板的话，对最后实验的精度有影响吗？ 还想问下您是用什么库评估的模型再各个下游任务的精度呢，是llm_eval库吗。 期待您的回复！