Skip to content

Commit cd1ea22

Browse files
authored
Fix dssm data coding problems (#241)
1 parent 1bdde2b commit cd1ea22

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

models/match/multiview-simnet/data/preprocess.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,12 @@
1515

1616
import os
1717
import sys
18+
import io
1819
import jieba
1920
import numpy as np
2021
import random
2122

22-
f = open("./raw_data.txt", "r")
23+
f = io.open("./raw_data.txt", mode="r", encoding='utf-8')
2324
lines = f.readlines()
2425
f.close()
2526

@@ -35,7 +36,7 @@
3536
else:
3637
word_dict[word] = len(word_dict) + 1
3738

38-
f = open("./raw_data.txt", "r")
39+
f = io.open("./raw_data.txt", mode="r", encoding='utf-8')
3940
lines = f.readlines()
4041
f.close()
4142

0 commit comments

Comments
 (0)