Skip to content

Commit a25f6b1

Browse files
committed
docs: add example for setting up live updates with refresh interval
1 parent 482686c commit a25f6b1

File tree

2 files changed

+34
-13
lines changed

2 files changed

+34
-13
lines changed

docs/docs/tutorials/live_updates.md

Lines changed: 31 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,21 @@ Live updates in CocoIndex can be triggered in two main ways:
1919

2020
When a change is detected, CocoIndex performs an **incremental update**. This means it only re-processes the data that has been affected by the change, without having to re-index your entire dataset. This makes the update process fast and efficient.
2121

22+
Here's an example of how to set up a source with a `refresh_interval`:
23+
24+
```python
25+
@cocoindex.flow_def(name="LiveUpdateExample")
26+
def live_update_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
27+
# Source: local files in the 'data' directory
28+
data_scope["documents"] = flow_builder.add_source(
29+
cocoindex.sources.LocalFile(path="data"),
30+
refresh_interval=cocoindex.timedelta(seconds=5),
31+
)
32+
# ...
33+
```
34+
35+
By setting `refresh_interval` to 5 seconds, we're telling CocoIndex to check for changes in the `data` directory every 5 seconds.
36+
2237
## Implementing Live Updates
2338

2439
You can enable live updates using either the CocoIndex CLI or the Python library.
@@ -44,11 +59,8 @@ Here's how you can use `FlowLiveUpdater` to start and manage a live update proce
4459
```python
4560
import cocoindex
4661

47-
# Assume you have a flow defined as 'my_flow'
48-
# from my_flows import my_flow
49-
5062
# Create a FlowLiveUpdater instance
51-
with cocoindex.FlowLiveUpdater(my_flow, cocoindex.FlowLiveUpdaterOptions(print_stats=True)) as updater:
63+
with cocoindex.FlowLiveUpdater(live_update_flow, cocoindex.FlowLiveUpdaterOptions(print_stats=True)) as updater:
5264
print("Live updater started. Press Ctrl+C to stop.")
5365
# The updater runs in the background.
5466
# The wait() method blocks until the updater is stopped.
@@ -64,10 +76,7 @@ You can also get status updates from the `FlowLiveUpdater` to monitor the update
6476
```python
6577
import cocoindex
6678

67-
# Assume you have a flow defined as 'my_flow'
68-
# from my_flows import my_flow
69-
70-
updater = cocoindex.FlowLiveUpdater(my_flow)
79+
updater = cocoindex.FlowLiveUpdater(live_update_flow)
7180
updater.start()
7281

7382
while True:
@@ -89,7 +98,7 @@ This allows you to react to updates in your application, for example, by notifyi
8998

9099
Let's walk through an example of how to set up a live update flow. For the complete, runnable code, see the [live updates example](https://github.com/cocoindex-io/cocoindex/tree/main/examples/live_updates) in the CocoIndex repository.
91100

92-
### Setting up the Source
101+
### 1. Setting up the Source
93102

94103
The first step is to define a source and configure a `refresh_interval`. In this example, we'll use a `LocalFile` source to monitor a directory named `data`.
95104

@@ -101,12 +110,23 @@ def live_update_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.
101110
cocoindex.sources.LocalFile(path="data"),
102111
refresh_interval=cocoindex.timedelta(seconds=5),
103112
)
104-
# ...
113+
114+
# Collector
115+
collector = data_scope.add_collector()
116+
with data_scope["documents"].row() as doc:
117+
collector.collect(filename=doc["filename"], content=doc["content"])
118+
119+
# Target: Postgres database
120+
collector.export(
121+
"documents_index",
122+
cocoindex.targets.Postgres(),
123+
primary_key_fields=["filename"]
124+
)
105125
```
106126

107127
By setting `refresh_interval` to 5 seconds, we're telling CocoIndex to check for changes in the `data` directory every 5 seconds.
108128

109-
### Running the Live Updater
129+
### 2. Running the Live Updater
110130

111131
Once the flow is defined, you can use the `FlowLiveUpdater` to start the live update process.
112132

examples/live_updates/main.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@
44
from dotenv import load_dotenv
55

66

7+
# Define the flow
78
@cocoindex.flow_def(name="LiveUpdates")
89
def live_update_flow(
910
flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope
10-
):
11+
) -> None:
1112
# Source: local files in the 'data' directory
1213
data_scope["documents"] = flow_builder.add_source(
1314
cocoindex.sources.LocalFile(path="data"),
@@ -30,7 +31,7 @@ def live_update_flow(
3031
)
3132

3233

33-
def main():
34+
def main() -> None:
3435
# Setup the flow
3536
live_update_flow.setup(report_to_stdout=True)
3637

0 commit comments

Comments
 (0)