Skip to content

Commit 1f86e82

Browse files
KingZeyuChen
andauthored
optimization for read_func of lcqmc (#553)
* #530 add parameter verification * skip illegal lines in dataset likes paws-x-zh Co-authored-by: Zeyu Chen <[email protected]>
1 parent 97086d8 commit 1f86e82

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

paddlenlp/datasets/lcqmc.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,9 +64,11 @@ def _read(self, filename):
6464
if len(data) == 3:
6565
query, title, label = data
6666
yield {"query": query, "title": title, "label": label}
67-
else:
67+
elif len(data) == 2:
6868
query, title = data
6969
yield {"query": query, "title": title, "label": ''}
70+
else:
71+
continue
7072

7173
def get_labels(self):
7274
"""

0 commit comments

Comments
 (0)