Skip to content

Commit c19372b

Browse files
author
gongenlei
authored
[CodeGen] Add CodeGen model for code generation (#2641)
* feat: add CodeGen model * update copyright
1 parent 4231112 commit c19372b

File tree

7 files changed

+733
-0
lines changed

7 files changed

+733
-0
lines changed

paddlenlp/transformers/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,8 @@
114114
from .optimization import *
115115
from .auto.modeling import *
116116
from .auto.tokenizer import *
117+
from .codegen.modeling import *
118+
from .codegen.tokenizer import *
117119

118120
# For faster tokenizer
119121
from ..utils.import_utils import is_faster_tokenizer_available

paddlenlp/transformers/auto/modeling.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@
7878
("Bert", "bert"),
7979
("Bart", "bart"),
8080
("GAUAlpha", "gau_alpha"),
81+
("CodeGen", "codegen"),
8182
])
8283

8384
MAPPING_TASKS = OrderedDict([

paddlenlp/transformers/auto/tokenizer.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@
7777
("BertTokenizer", "bert"),
7878
("BartTokenizer", "bart"),
7979
("GAUAlphaTokenizer", "gau_alpha"),
80+
("CodeGenTokenizer", "codegen"),
8081
])
8182

8283
FASTER_TOKENIZER_MAPPING_NAMES = OrderedDict([("BertFasterTokenizer", "bert"),

paddlenlp/transformers/codegen/__init__.py

Whitespace-only changes.

0 commit comments

Comments
 (0)