You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -127,7 +127,7 @@ This function returns a list of chunks up to `chunk_size`-tokens-long, with any
127
127
1. Recursively splits the resulting chunks until a set of chunks equal to or less than the specified chunk size is produced;
128
128
1. Merges any chunks that are under the chunk size back together until the chunk size is reached;
129
129
1. Reattaches any non-whitespace splitters back to the ends of chunks barring the final chunk if doing so does not bring chunks over the chunk size, otherwise adds non-whitespace splitters as their own chunks; and
130
-
1. Since version XXXX, excludes chunks consisting entirely of whitespace characters.
130
+
1. Since version 3.0.0, excludes chunks consisting entirely of whitespace characters.
131
131
132
132
To ensure that chunks are as semantically meaningful as possible, `semchunk` uses the following splitters, in order of precedence:
133
133
1. The largest sequence of newlines (`\n`) and/or carriage returns (`\r`);
0 commit comments