Skip to content

Commit 09a29ea

Browse files
committed
style: moved comments above code instead of inline
1 parent ccd457b commit 09a29ea

File tree

1 file changed

+8
-4
lines changed

1 file changed

+8
-4
lines changed

src/semchunk/semchunk.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,12 @@
2323
"""A map of token counters to their memoized versions."""
2424

2525
_NON_WHITESPACE_SEMANTIC_SPLITTERS = (
26+
# Sentence terminators.
2627
".",
2728
"?",
2829
"!",
29-
"*", # Sentence terminators.
30+
"*",
31+
# Clause separators.
3032
";",
3133
",",
3234
"(",
@@ -39,15 +41,17 @@
3941
"’",
4042
"'",
4143
'"',
42-
"`", # Clause separators.
44+
"`",
45+
# Sentence interrupters.
4346
":",
4447
"—",
45-
"…", # Sentence interrupters.
48+
"…",
49+
# Word joiners.
4650
"/",
4751
"\\",
4852
"–",
4953
"&",
50-
"-", # Word joiners.
54+
"-",
5155
)
5256
"""A tuple of semantically meaningful non-whitespace splitters that may be used to chunk texts, ordered from most desirable to least desirable."""
5357

0 commit comments

Comments
 (0)