Skip to content

Commit 4265077

Browse files
committed
example: add the tutorial of performing live updates
1 parent ff0ec4b commit 4265077

File tree

6 files changed

+169
-0
lines changed

6 files changed

+169
-0
lines changed

examples/live_updates/.env

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
COCOINDEX_DATABASE_URL=postgres://cocoindex:cocoindex@localhost/cocoindex

examples/live_updates/README.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Applying Live Updates to CocoIndex Flow Example
2+
[![GitHub](https://img.shields.io/github/stars/cocoindex-io/cocoindex?color=5B5BD6)](https://github.com/cocoindex-io/cocoindex)
3+
4+
We appreciate a star ⭐ at [CocoIndex Github](https://github.com/cocoindex-io/cocoindex) if this is helpful.
5+
6+
This example demonstrates how to use CocoIndex's live update feature to keep an index synchronized with a local directory.
7+
8+
## How it Works
9+
10+
The `main.py` script defines a CocoIndex flow that:
11+
12+
1. **Sources** data from a local directory named `data`. It uses a `refresh_interval` of 5 seconds to check for changes.
13+
2. **Collects** the `filename` and `content` of each file.
14+
3. **Exports** the collected data to a Postgres database table.
15+
16+
The script then starts a `FlowLiveUpdater`, which runs in the background and continuously monitors the `data` directory for changes.
17+
18+
## Running the Example
19+
20+
1. [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one.
21+
22+
2. **Install the dependencies:**
23+
24+
```bash
25+
pip install -e .
26+
```
27+
28+
3. **Run the example:**
29+
30+
You can run the live update example in two ways:
31+
32+
**Option 1: Using the Python script**
33+
34+
This method uses CocoIndex [Library API](https://cocoindex.io/docs/core/flow_methods#library-api-2) to perform live updates.
35+
36+
```bash
37+
python main.py
38+
```
39+
40+
**Option 2: Using the CocoIndex CLI**
41+
42+
This method is useful for managing your indexes from the command line, through CocoIndex [CLI](https://cocoindex.io/docs/core/flow_methods#cli-2).
43+
44+
```bash
45+
cocoindex update main.py -L --setup
46+
```
47+
48+
4. **Test the live updates:**
49+
50+
While the script is running, you can try adding, modifying, or deleting files in the `data` directory. You will see the changes reflected in the logs as CocoIndex updates the index.
51+
52+
## Cleaning Up
53+
54+
To remove the database table created by this example, you can run:
55+
56+
```bash
57+
cocoindex drop main.py
58+
```
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
In the spirit of Project Zeta’s innovative chaos, here’s a collection of absurdly true facts about the weirdest animals you’ve never heard of:
2+
3+
1. **Tardigrade (Water Bear)**: This microscopic beast can survive outer space, radiation, and being boiled alive. It once crashed a team meeting by stowing away in Bob’s coffee mug and demanding admin access to the server.
4+
5+
2. **Aye-Aye**: A Madagascar primate with a creepy long finger it uses to tap trees for grubs. It tried to “debug” our codebase by tapping the keyboard, resulting in 47 nested for-loops.
6+
7+
3. **Saiga Antelope**: This goofy-nosed critter looks like it’s auditioning for a sci-fi flick. Its sneezes are so powerful they once blew out the office Wi-Fi during a sprint review.
8+
9+
4. **Glaucus Atlanticus (Blue Dragon Sea Slug)**: This tiny ocean dragon steals venom from jellyfish and uses it like a borrowed superpower. It infiltrated our water cooler and left behind a sparkly, toxic trail.
10+
11+
5. **Pink Fairy Armadillo**: A palm-sized digger that looks like a cotton candy tank. It burrowed into the office carpet, mistaking it for a desert, and now we have a “no armadillos” policy.
12+
13+
6. **Dumbo Octopus**: A deep-sea octopus with ear-like fins, flapping around like it’s late for a Zoom call. It once rewired our projector to display memes of itself across the office.
14+
15+
7. **Jerboa**: A hopping desert rodent with kangaroo vibes. It stole the team’s snacks and leaped over three cubicles before anyone noticed, earning the codename "Snack Bandit."
16+
17+
8. **Mantis Shrimp**: This crustacean sees more colors than our graphic designer and punches harder than a failing CI pipeline. It shattered a monitor when we tried to pair-program with it.
18+
19+
9. **Okapi**: A zebra-giraffe hybrid that looks like a Photoshop error. It wandered into our sprint planning and suggested we pivot to a “forest-themed” microservices architecture.
20+
21+
10. **Blobfish**: The ocean’s saddest-looking blob, voted “Most Likely to Crash a Stand-Up” by the team. Its mere presence caused our morale bot to send 200 crying emojis.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Chuck Norris Project Facts
2+
Date: 2025-07-20
3+
Author: Anonymous (because Chuck Norris knows who you are)
4+
5+
Here are some totally true facts about Chuck Norris's involvement in Project Omega:
6+
7+
1. Chuck Norris doesn't write code; he stares at the computer until it writes itself out of fear.
8+
2. The project deadline was yesterday, but time rescheduled itself to accommodate Chuck Norris.
9+
3. Chuck Norris's code never has bugs—just "features" that are too scared to misbehave.
10+
4. When the database crashed, Chuck Norris roundhouse-kicked the server, and it apologized.
11+
5. The team tried to use Agile, but Chuck Norris declared, "I am the only methodology you need."
12+
6. Version control? Chuck Norris is the only version that matters.
13+
7. The project scope expanded because Chuck Norris added "world domination" as a deliverable.
14+
8. When the CI/CD pipeline failed, Chuck Norris rebuilt it with a single grunt.
15+
9. The codebase is 100% documented because no one dares ask Chuck Norris, "What does this do?"
16+
10. Chuck Norris doesn't deploy to production; production deploys to Chuck Norris.
17+
18+
Last updated: 2025-07-20 06:36 AM MST
19+
Note: If you modify this file, Chuck Norris will know... and he’ll find you.

examples/live_updates/main.py

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
import dataclasses
2+
import datetime
3+
4+
import cocoindex
5+
from dotenv import load_dotenv
6+
7+
8+
@dataclasses.dataclass
9+
class Document:
10+
filename: str
11+
content: str
12+
13+
14+
@cocoindex.flow_def(name="LiveUpdates")
15+
def live_update_flow(
16+
flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope
17+
):
18+
# Source: local files in the 'data' directory
19+
data_scope["documents"] = flow_builder.add_source(
20+
cocoindex.sources.LocalFile(path="data"),
21+
refresh_interval=datetime.timedelta(seconds=5),
22+
)
23+
24+
# Collector
25+
collector = data_scope.add_collector()
26+
with data_scope["documents"].row() as doc:
27+
collector.collect(
28+
filename=doc["filename"],
29+
content=doc["content"],
30+
)
31+
32+
# Target: Postgres database
33+
collector.export(
34+
"documents_index",
35+
cocoindex.targets.Postgres(),
36+
primary_key_fields=["filename"],
37+
)
38+
39+
40+
def main():
41+
# Setup the flow
42+
live_update_flow.setup(report_to_stdout=True)
43+
44+
# Start the live updater
45+
print("Starting live updater...")
46+
with cocoindex.FlowLiveUpdater(
47+
live_update_flow, cocoindex.FlowLiveUpdaterOptions(print_stats=True)
48+
) as updater:
49+
print("Live updater started. Watching for changes in the 'data' directory.")
50+
print("Try adding, modifying, or deleting files in the 'data' directory.")
51+
print("Press Ctrl+C to stop.")
52+
updater.wait()
53+
54+
55+
if __name__ == "__main__":
56+
load_dotenv()
57+
cocoindex.init()
58+
main()
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
[project]
2+
name = "live-updates-example"
3+
version = "0.1.0"
4+
description = "Simple example for cocoindex: perform live updates based on local markdown files."
5+
requires-python = ">=3.11"
6+
dependencies = [
7+
"cocoindex>=0.1.70",
8+
"python-dotenv>=1.1.0",
9+
]
10+
11+
[tools.setuptools]
12+
packages = []

0 commit comments

Comments
 (0)