Update docs for incremental update / change capturing to clarify things. (#268)

Author: badmonster0

docs/docs/core/basics.md

@@ -70,15 +70,17 @@ An indexing flow, once set up, maintains a long-lived relationship between data
     on certain pace, according to the update mode:
 
     *   **One time update**: Once triggered, CocoIndex updates the target data to reflect the version of source data up to the current moment.
-    *   **Live update**: CocoIndex continuously watches the source data and updates the target data accordingly.
+    *   **Live update**: CocoIndex continuously reacts to changes of source data and updates the target data accordingly, based on various **change capture mechanisms** for the source.
 
     See more details in the [build / update target data](flow_methods#build--update-target-data) section.
 
-3.  CocoIndex intelligently manages these updates by:
+3.  CocoIndex intelligently reprocesses to propagate source changes to target by:
+
     *   Determining which parts of the target data need to be recomputed
     *   Reusing existing computations where possible
     *   Only reprocessing the minimum necessary data
 
+    This is known as **incremental processing**.
 
 You can think of an indexing flow similar to formulas in a spreadsheet:
 

docs/docs/core/flow_methods.mdx

@@ -38,21 +38,21 @@ It creates a `demo_flow` object in `cocoindex.Flow` type.
 ## Build / update target data
 
 The major goal of a flow is to perform the transformations on source data and build / update data in the target storage (the index).
-This action has two flavors:
+This action has two modes:
 
 *   **One time update.**
     It builds/update the target data based on source data up to the current moment.
     After the target data is at least as fresh as the source data when update starts, it's done.
     It fits into situations that you need to access the fresh target data at certain time points.
 
 *   **Live update.**
-    It continuously watches the source data and updates the target data accordingly.
+    It continuously captures changes from the source data and updates the target data accordingly.
     It's long-running and only stops when being aborted explicitly.
     It fits into situations that you need to access the fresh target data continuously in most of the time.
 
 :::info
 
-For both flavors, CocoIndex is performing updates incrementally.
+For both modes, CocoIndex is performing *incremental processing*,
 i.e. we only performs computations and storage mutations on source data that are changed, or the flow has changed.
 This is to achieve best efficiency.
 
@@ -90,22 +90,26 @@ CLI equivalence: `cocoindex update -L`
 
 :::
 
-Live update is *eligible* for certain data sources, including:
+A data source may enable one or multiple *change capture mechanisms*:
 
-*   Data sources configured with a [refresh interval](flow_def#refresh-interval).
-*   Data sources provides a **change stream**.
+*   Configured with a [refresh interval](flow_def#refresh-interval), which is generally applicable to all data sources.
+*   Specific data sources also provide their specific change capture mechanisms.
+    See documentations for specific data sources for details.
+
+Change capture mechanisms enables CocoIndex to continuously capture changes from the source data and update the target data accordingly, under live update mode.
 
 To perform live update, you need to create a `cocoindex.FlowLiveUpdater` object using the `cocoindex.Flow` object.
 It takes an optional `cocoindex.FlowLiveUpdaterOptions` option, with the following fields:
 
 *   `live_mode` (type: `bool`, default: `True`):
-     Whether to perform live update for eligible data sources.
+     Whether to perform live update for data sources with change capture mechanisms.
+     It has no effect for data sources without any change capture mechanism.
 
 *   `print_stats` (type: `bool`, default: `False`): Whether to print stats during update.
 
-For data sources ineligible for live updates, or when the `live_mode` is `False`,
-the `FlowLiveUpdater` only performs a one-time update, i.e. similar to the one-time update (`update()` method) above,
-under a unified interface.
+Note that `cocoindex.FlowLiveUpdater` provides a unified interface for both one-time update and live update.
+It only performs live update when `live_mode` is `True`, and only for sources with change capture mechanisms enabled.
+If a source has multiple change capture mechanisms enabled, all will take effect to trigger updates.
 
 <Tabs>
 <TabItem value="python" label="Python" default>
@@ -126,7 +130,7 @@ A `FlowLiveUpdater` object supports the following methods:
 *   `wait()` (async): Wait for the updater to finish. It only unblocks in one of the following cases:
     *   The updater was aborted.
     *   A one time update is done, and live update is not enabled:
-        either `live_mode` is `False`, or all data sources are ineligible for live updates.
+        either `live_mode` is `False`, or all data sources have no change capture mechanisms enabled.
 *   `update_stats()`: It returns the stats of the updater.
 
 <Tabs>

docs/docs/ops/sources.md

@@ -59,7 +59,7 @@ The spec takes the following fields:
 *   `service_account_credential_path` (type: `str`, required): full path to the service account credential file in JSON format.
 *   `root_folder_ids` (type: `list[str]`, required): a list of Google Drive folder IDs to import files from.
 *   `binary` (type: `bool`, optional): whether reading files as binary (instead of text).
-*   `recent_changes_poll_interval` (type: `datetime.timedelta`, optional): when set, this source provides a *change stream* by polling Google Drive for recent modified files periodically.
+*   `recent_changes_poll_interval` (type: `datetime.timedelta`, optional): when set, this source provides a *change capture mechanism* by polling Google Drive for recent modified files periodically.
 
     :::info
 
@@ -70,8 +70,8 @@ The spec takes the following fields:
     On the other hand, this only detects changes for files still exists.
     If the file is deleted (or the current account no longer has access to), this change will not be detected by this change stream.
 
-    So when a source is configured with a change stream, it's still recommended to set a `refresh_interval`, with a larger value.
-    So for most changes can be covered by the change stream (with low latency), and remaining changes (files no longer exist or accessible) will still be covered (with a higher latency).
+    So when a `GoogleDrive` source enabled `recent_changes_poll_interval`, it's still recommended to set a `refresh_interval`, with a larger value.
+    So that most changes can be covered by polling recent changes (with low latency), and remaining changes (files no longer exist or accessible) will still be covered (with a higher latency).
 
     :::