Skip to content

Commit b0ca50c

Browse files
authored
Merge pull request #88 from are-ces/update-readme
Add important detail about --auto-chunking
2 parents 40ccb00 + f658910 commit b0ca50c

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,12 @@ exact same path** that was specified during database creation.
231231
232232
### Llama-Stack Faiss
233233
234+
> [!IMPORTANT]
235+
> When using the `--auto-chunking` flag, chunking happens within llama-stack using the
236+
> OpenAI-compatible Files API. This makes vector stores significantly larger than manual
237+
> chunking because the Files API stores a redundant copy of the embeddings.
238+
> Manual chunking results in smaller database files.
239+
234240
The process is basically the same as in the
235241
[Llama-Index Faiss Vector Store](#faiss-vector-store) but passing the
236242
`--vector-store-type` parameter; so you generate the documentation using the

0 commit comments

Comments
 (0)