Skip to content

Commit eb05ff0

Browse files
chencyudelchencyudel
committed
typo
1 parent 368a858 commit eb05ff0

File tree

2 files changed

+9
-9
lines changed

2 files changed

+9
-9
lines changed

‎README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@
4141

4242

4343
## News
44-
🔥🔥 [2024/01/17] We released MFTCoder v0.3.0, mainly for MFTCoder-accelerate. It now supports new models like Mixtral(MoE), Deepseek-coder, chatglm3. It supports FSDP as an option. It also supports Self-paced Loss as a solution for convergence balance in Multitask Fine-tuning.
44+
🔥🔥 [2024/01/17] We released MFTCoder v0.3.0, mainly for MFTCoder-accelerate. It now supports new models like Mixtral(MoE), DeepSeek-coder, chatglm3. It supports FSDP as an option. It also supports Self-paced Loss as a solution for convergence balance in Multitask Fine-tuning.
4545

46-
🔥🔥 [2024/01/17] [CodeFuse-Deepseek-33B](https://huggingface.co/codefuse-ai/CodeFuse-Deepseek-33B) has been released, achieving a pass@1 (greedy decoding) score of 78.7% on HumanEval. It achieves top1 win-rate on Bigcode Leardboard.
46+
🔥🔥 [2024/01/17] [CodeFuse-DeepSeek-33B](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) has been released, achieving a pass@1 (greedy decoding) score of 78.7% on HumanEval. It achieves top1 win-rate on Bigcode Leardboard.
4747

4848
🔥🔥 [2024/01/17] [CodeFuse-Mixtral-8x7B](https://huggingface.co/codefuse-ai/CodeFuse-Mixtral-8X7B) has been released, achieving a pass@1 (greedy decoding) score of 56.1% on HumanEval.
4949

@@ -62,7 +62,7 @@
6262
### HumanEval Performance
6363
| Model | HumanEval(Pass@1) | Date |
6464
|:----------------------------|:-----------------:|:-------:|
65-
| **CodeFuse-Deepseek-33B** | **78.7%** | 2024/01 |
65+
| **CodeFuse-DeepSeek-33B** | **78.7%** | 2024/01 |
6666
| **CodeFuse-Mixtral-8x7B** | **56.1%** | 2024/01 |
6767
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023/09 |
6868
| **CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023/09 |
@@ -146,7 +146,7 @@ We are excited to release the following two CodeLLMs trained by MFTCoder, now av
146146

147147
| Model | | | Base Model | Num of examples trained | Batch Size | Seq Length |
148148
|--------------------------------------|------------------------|---|----------------------|------|------------|------------|
149-
| 🔥🔥 CodeFuse-Deepseek-33B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-Deepseek-33B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-Deepseek-33B) | Deepseek-coder-33B | 60万 | 80 | 4096 |
149+
| 🔥🔥 CodeFuse-DeepSeek-33B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-DeepSeek-33B) | DeepSeek-coder-33B | 60万 | 80 | 4096 |
150150
| 🔥🔥 CodeFuse-Mixtral-8x7B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-Mixtral-8x7B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-Mixtral-8x7B) | Mixtral-8x7B | 60万 | 80 | 4096 |
151151
| 🔥🔥 CodeFuse-CodeLlama-34B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-CodeLlama-34B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B) | CodeLlama-34b-Python | 60万 | 80 | 4096 |
152152
| 🔥🔥 CodeFuse-CodeLlama-34B-4bits |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-CodeLlama-34B-4bits) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits) | CodeLlama-34b-Python | | | 4096 |

‎README_cn.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@
3939

4040

4141
## 新闻
42-
🔥🔥 [2024/01/17] MFTCoder新版v0.3.0发布。新增对Mixtral(MoE), deepseek等模型的支持；新增支持FSDP(Fully Sharded Data Parallel)；新增Self-paced Loss, 支持多任务收敛均衡。 感兴趣详见微信公众号CodeFuse[文章](https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
42+
🔥🔥 [2024/01/17] MFTCoder新版v0.3.0发布。新增对Mixtral(MoE), DeepSeek等模型的支持；新增支持FSDP(Fully Sharded Data Parallel)；新增Self-paced Loss, 支持多任务收敛均衡。 感兴趣详见微信公众号CodeFuse[文章](https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
4343

44-
🔥🔥 [2024/01/17] 开源了[CodeFuse-Deepseek-33B](https://huggingface.co/codefuse-ai/CodeFuse-Deepseek-33B)模型，在HumanEval pass@1(greedy decoding)上可以达到78.7%。感兴趣详见微信公众号CodeFuse[文章](https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
44+
🔥🔥 [2024/01/17] 开源了[CodeFuse-DeepSeek-33B](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B)模型，在HumanEval pass@1(greedy decoding)上可以达到78.7%。感兴趣详见微信公众号CodeFuse[文章](https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
4545

4646
🔥🔥 [2024/01/17] 开源了[CodeFuse-Mixtral-8x7B](https://huggingface.co/codefuse-ai/CodeFuse-Mixtral-8x7B)模型，在HumanEval pass@1(greedy decoding)上可以达到56.1%。感兴趣详见微信公众号CodeFuse[文章](https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
4747

@@ -60,7 +60,7 @@
6060
### HumanEval表现
6161
| 模型 | HumanEval(Pass@1) | 日期 |
6262
|:---------------------------------|:-----------------:|:-------:|
63-
| **CodeFuse-Deepseek-33B** | **78.7%** | 2024/01 |
63+
| **CodeFuse-DeepSeek-33B** | **78.7%** | 2024/01 |
6464
| **CodeFuse-Mixtral-8x7B** | **56.1%** | 2024/01 |
6565
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023/09 |
6666
| **CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023/09 |
@@ -104,7 +104,7 @@
104104
本项目主要内容如下：
105105
- 同时支持单任务SFT(Supervised FineTuning)和MFT(Multi-task FineTuning), 当前开源支持数据均衡，未来将持续开源难易均衡， 收敛均衡等
106106
- 支持QLoRA低成本高效指令微调、LoRA高效指令微调、全量参数高精度微调。
107-
- 支持绝大部分主流的开源大模型，重点关注代码能力优秀的开源大模型，如Deepseek-coder, Mistral, Mistral(MoE), Chatglm3, Qwen, GPT-Neox, Starcoder, Codegeex2, Code-LLaMA等。
107+
- 支持绝大部分主流的开源大模型，重点关注代码能力优秀的开源大模型，如DeepSeek-coder, Mistral, Mistral(MoE), Chatglm3, Qwen, GPT-Neox, Starcoder, Codegeex2, Code-LLaMA等。
108108
- 支持lora与base model进行权重合并，推理更便捷。
109109
- 整理并开源2个指令微调数据集：[Evol-instruction-66k](https://huggingface.co/datasets/codefuse-ai/Evol-instruction-66k)和[CodeExercise-Python-27k](https://huggingface.co/datasets/codefuse-ai/CodeExercise-Python-27k)。
110110
- 开源多个[Codefuse系列指令微调模型权重]，具体参见我们的huggingface组织和modelscope组织下的模型：[codefuse-ai huggingface](https://huggingface.co/codefuse-ai) or [codefuse-ai 魔搭](https://modelscope.cn/organization/codefuse-ai)。
@@ -136,7 +136,7 @@ sh init_env.sh
136136

137137
| 模型 | HuggingFace links | ModelScope links | 基座模型 | 训练数据 | Batch Size | Seq Length |
138138
|--------------------------------------|------------------------|---|----------------------|------|------------|------------|
139-
| 🔥🔥🔥 CodeFuse-Deepseek-33B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-Deepseek-33B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-Deepseek-33B) | Deepseek-coder-33B | 60万 | 80 | 4096 |
139+
| 🔥🔥🔥 CodeFuse-DeepSeek-33B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-DeepSeek-33B) | DeepSeek-coder-33B | 60万 | 80 | 4096 |
140140
| 🔥🔥🔥 CodeFuse-Mixtral-8x7B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-Mixtral-8x7B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-Mixtral-8x7B) | Mixtral-8x7B | 60万 | 80 | 4096 |
141141
| 🔥🔥🔥 CodeFuse-CodeLlama-34B |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-CodeLlama-34B) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B) | CodeLlama-34b-Python | 60万 | 80 | 4096 |
142142
| 🔥🔥🔥 CodeFuse-CodeLlama-34B-4bits |[h-link](https://huggingface.co/codefuse-ai/CodeFuse-CodeLlama-34B-4bits) |[m-link](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits) | CodeLlama-34b-Python | | | 4096 |

0 commit comments

Comments
 (0)