Skip to content

Commit 3f8b08b

Browse files
committed
Further update docs for incremental updates / change capture. (#269)
1 parent 7024b1d commit 3f8b08b

File tree

2 files changed

+5
-2
lines changed

2 files changed

+5
-2
lines changed

docs/docs/core/flow_methods.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,10 @@ CLI equivalence: `cocoindex update -L`
9393
A data source may enable one or multiple *change capture mechanisms*:
9494

9595
* Configured with a [refresh interval](flow_def#refresh-interval), which is generally applicable to all data sources.
96+
9697
* Specific data sources also provide their specific change capture mechanisms.
97-
See documentations for specific data sources for details.
98+
For example, [`GoogleDrive` source](../ops/sources#google-drive) allows polling recent modified files.
99+
See documentations for specific data sources.
98100

99101
Change capture mechanisms enables CocoIndex to continuously capture changes from the source data and update the target data accordingly, under live update mode.
100102

docs/docs/ops/sources.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,8 @@ The spec takes the following fields:
7171
If the file is deleted (or the current account no longer has access to), this change will not be detected by this change stream.
7272

7373
So when a `GoogleDrive` source enabled `recent_changes_poll_interval`, it's still recommended to set a `refresh_interval`, with a larger value.
74-
So that most changes can be covered by polling recent changes (with low latency), and remaining changes (files no longer exist or accessible) will still be covered (with a higher latency).
74+
So that most changes can be covered by polling recent changes (with low latency, like 10 seconds), and remaining changes (files no longer exist or accessible) will still be covered (with a higher latency, like 5 minutes, and should be larger if you have a huge number of files like 1M).
75+
In reality, configure them based on your requirement: how freshness do you need to target index to be?
7576

7677
:::
7778

0 commit comments

Comments
 (0)