Skip to content

Commit 4193640

Browse files
committed
feat: add quote stripping to text cleaning functions
This change makes the model better at predicting.
1 parent 4ebd8b4 commit 4193640

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

app/utils.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,10 @@ def remove_emoji(text: str) -> str:
2828
return text
2929

3030

31+
def strip_quotes(text: str) -> str:
32+
return text.strip(" \"'")
33+
34+
3135
def clear_text(text: str) -> str:
3236
if not isinstance(text, str):
3337
logger.warning(
@@ -37,7 +41,7 @@ def clear_text(text: str) -> str:
3741
)
3842
return text
3943

40-
return remove_emoji(text)
44+
return remove_emoji(strip_quotes(text))
4145

4246

4347
Result = TypeVar("Result")

0 commit comments

Comments
 (0)