模型微调后生成效果不理想

[minmie]

我正在微调模型，希望模型能理解工具返回的内容，并且生成正确的答案。

假设用户提问：平台事业部有人收过礼品吗？
工具返回的数据如下（是一个json）:

{
          "code": 0,
          "data": {
            "content": [
              {
                "receiverDepartment": "平台事业部",
                "approvalStatus": 1,
                "allAmount": 1,
                "receiverName": "张丽婷",
                "approvalStatusName": "审批中",
                "giftName": "苹果、奔驰",
                "giftIds": [],
                "receiverNo": "28154149"
              },
              {
                "receiverName": "张三丰",
                "giftIds": [],
                "allAmount": 1,
                "approvalStatusName": "审批中",
                "receiverDepartment": "平台事业部",
                "giftName": "张三、现金、小陈、月饼",
                "approvalStatus": 1,
                "receiverNo": "99329002"
              },
              {
                "approvalStatus": 1,
                "giftIds": [],
                "receiverName": "张丽雨",
                "receiverDepartment": "平台事业部",
                "giftName": "月光、asdfm、gucci",
                "allAmount": 1,
                "approvalStatusName": "审批中",
                "receiverNo": "69997368"
              }
            ],
            "page": 0,
            "size": 3,
            "totalPage": 9,
            "totalSize": 26
          }
        }

期望的回答（训练时使用的正确答案数据格式）

是的，平台事业部有人收过礼品，部分礼金记录如下：
- 张丽婷(28154149) 礼品信息:苹果、奔驰
- 张三丰(99329002) 礼品信息:张三、现金、小陈、月饼
- 张丽雨(69997368) 礼品信息:月光、asdfm、gucci

以上只列出了部礼金记录，更多详细信息请参考<holder:url>

实际训练好后，模型生成的答案如下：

首先生成的格式就不对，而且内容也经常出现错误。
请问是哪里出问题了，有没有优化的方法？

[minmie]

训练数据500条，训练了3个epoch。学习率2e-2
下面是一条完整的训练数据

{
    "conversations": [
      {
        "role": "user",
        "content": "平台事业部有人收过礼品吗？"
      },
      {
        "role": "tool",
        "name": "get_list",
        "parameters": {
          "department": "平台事业部"
        },
        "observation": {
          "code": 0,
          "data": {
            "content": [
              {
                "receiverDepartment": "平台事业部",
                "approvalStatus": 1,
                "allAmount": 1,
                "receiverName": "张丽婷",
                "approvalStatusName": "审批中",
                "giftName": "苹果、奔驰",
                "giftIds": [],
                "receiverNo": "28154149"
              },
              {
                "receiverName": "张三丰",
                "giftIds": [],
                "allAmount": 1,
                "approvalStatusName": "审批中",
                "receiverDepartment": "平台事业部",
                "giftName": "张三、现金、小陈、月饼",
                "approvalStatus": 1,
                "receiverNo": "99329002"
              },
              {
                "approvalStatus": 1,
                "giftIds": [],
                "receiverName": "张丽雨",
                "receiverDepartment": "平台事业部",
                "giftName": "月光、asdfm、gucci",
                "allAmount": 1,
                "approvalStatusName": "审批中",
                "receiverNo": "69997368"
              }
            ],
            "page": 0,
            "size": 3,
            "totalPage": 9,
            "totalSize": 26
          }
        }
      },
      {
        "role": "assistant",
        "content": "是的，平台事业部有人收过礼品，部分礼金记录如下：\n- 张丽婷(28154149) 礼品信息:苹果、奔驰\n- 张三丰(99329002) 礼品信息:张三、现金、小陈、月饼\n- 张丽雨(69997368) 礼品信息:月光、asdfm、gucci\n。\n以上只列出了部礼金记录，更多详细信息请参考<holder:url>"
      }
    ],
    "tools": [
      {
        "name": "query_clue",
        "description": "查询线索列表",
        "params": []
      },
      {
        "name": "get_case_list",
        "description": "查询案件列表",
        "params": []
      },
      {
        "name": "get_list",
        "description": "查询礼金列表",
        "params": [
          {
            "name": "name",
            "description": "收礼人姓名",
            "type": "str",
            "required": false
          },
          {
            "name": "department",
            "description": "收礼人所在部门",
            "type": "str",
            "required": false
          }
        ]
      }
    ]
  }

[zRzRzRzRzRzRzR]

在#253中提出