Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/docs/ops/sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,19 @@ The `LocalFile` source imports files from a local file system.

The spec takes the following fields:
* `path` (type: `str`, required): full path of the root directory to import files from
* `binary` (type: `bool`, default: `False`): whether reading files as binary (instead of text)
* `binary` (type: `bool`, optional): whether reading files as binary (instead of text)
* `included_patterns` (type: `list[str]`, optional): a list of glob patterns to include files, e.g. `["*.txt", "docs/**/*.md"]`.
If not specified, all files will be included.
* `excluded_patterns` (type: `list[str]`, optional): a list of glob patterns to exclude files, e.g. `["tmp", "**/node_modules"]`.
Any file or directory matching these patterns will be excluded even if they match `included_patterns`.
If not specified, no files will be excluded.

:::info

`included_patterns` and `excluded_patterns` are using Unix-style glob syntax. See [globset syntax](https://docs.rs/globset/latest/globset/index.html#syntax) for the details.

:::


The output is a table with the following sub fields:
* `filename` (key, type: `str`): the filename of the file, including the path, relative to the root directory, e.g. `"dir1/file1.md"`
Expand Down
4 changes: 2 additions & 2 deletions python/cocoindex/sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ class LocalFile(op.SourceSpec):
binary: bool = False

# If provided, only files matching these patterns will be included.
# See https://docs.rs/globset/latest/globset/index.html for the syntax of the patterns.
# See https://docs.rs/globset/latest/globset/index.html#syntax for the syntax of the patterns.
included_patterns: list[str] | None = None

# If provided, files matching these patterns will be excluded.
# See https://docs.rs/globset/latest/globset/index.html for the syntax of the patterns.
# See https://docs.rs/globset/latest/globset/index.html#syntax for the syntax of the patterns.
excluded_patterns: list[str] | None = None