Skip to content

Commit 401c829

Browse files
badmonster0badmonster0
authored
Update flow_methods documents to make CLI/Python separate tabs. (#274)
1 parent 390332c commit 401c829

File tree

1 file changed

+62
-34
lines changed

1 file changed

+62
-34
lines changed

‎docs/docs/core/flow_methods.mdx‎

Lines changed: 62 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -12,25 +12,34 @@ After a flow is defined as discussed in [Flow Definition](/docs/core/flow_def),
1212

1313
It can be achieved in two ways:
1414

15+
* Use [CocoIndex CLI](/docs/core/cli).
16+
1517
* Use APIs provided by the library.
1618
You have a `cocoindex.Flow` object after defining the flow in your code, and you can interact with it later.
1719

18-
* Use [CocoIndex CLI](/docs/core/cli).
19-
20-
We'll focus on the first way in this document.
21-
The following sections assume you have a `demo_flow`:
20+
The following sections assume you have a flow `demo_flow`:
2221

2322
<Tabs>
2423
<TabItem value="python" label="Python" default>
2524

26-
```python
25+
```python title="main.py"
2726
@cocoindex.flow_def(name="DemoFlow")
2827
def demo_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
29-
...
28+
...
3029
```
3130

3231
It creates a `demo_flow` object in `cocoindex.Flow` type.
32+
To enable CLI, you also need to make sure you have a main function decorated with `@cocoindex.main_fn()`:
33+
3334

35+
```python title="main.py"
36+
@cocoindex.main_fn()
37+
def main():
38+
...
39+
40+
if __name__ == "__main__":
41+
main()
42+
```
3443
</TabItem>
3544
</Tabs>
3645

@@ -61,19 +70,24 @@ This is to achieve best efficiency.
6170

6271
### One time update
6372

64-
:::tip
73+
<Tabs>
74+
<TabItem value="shell" label="Shell" default>
6575

66-
CLI equivalence: `cocoindex update`
76+
The `cocoindex update` subcommand creates/updates data in the target storage.
6777

68-
:::
78+
Once it's done, the target data is fresh up to the moment when the function is called.
79+
80+
```sh
81+
python main.py cocoindex update
82+
```
83+
84+
</TabItem>
85+
<TabItem value="python" label="Python">
6986

7087
The `update()` async method creates/updates data in the target storage.
7188

7289
Once the function returns, the target data is fresh up to the moment when the function is called.
7390

74-
<Tabs>
75-
<TabItem value="python" label="Python" default>
76-
7791
```python
7892
stats = await demo_flow.update()
7993
print(stats)
@@ -84,12 +98,6 @@ print(stats)
8498

8599
### Live update
86100

87-
:::tip
88-
89-
CLI equivalence: `cocoindex update -L`
90-
91-
:::
92-
93101
A data source may enable one or multiple *change capture mechanisms*:
94102

95103
* Configured with a [refresh interval](flow_def#refresh-interval), which is generally applicable to all data sources.
@@ -100,6 +108,21 @@ A data source may enable one or multiple *change capture mechanisms*:
100108

101109
Change capture mechanisms enable CocoIndex to continuously capture changes from the source data and update the target data accordingly, under live update mode.
102110

111+
<Tabs>
112+
<TabItem value="shell" label="Shell" default>
113+
114+
To perform live update, run the `cocoindex update` subcommand with `-L` option:
115+
116+
```sh
117+
python main.py cocoindex update -L
118+
```
119+
120+
If there's at least one data source with change capture mechanism enabled, it will keep running until the aborted (e.g. by `Ctrl-C`).
121+
Otherwise, it falls back to the same behavior as one time update, and will finish after a one-time update is done.
122+
123+
</TabItem>
124+
<TabItem value="python" label="Python">
125+
103126
To perform live update, you need to create a `cocoindex.FlowLiveUpdater` object using the `cocoindex.Flow` object.
104127
It takes an optional `cocoindex.FlowLiveUpdaterOptions` option, with the following fields:
105128

@@ -113,19 +136,13 @@ Note that `cocoindex.FlowLiveUpdater` provides a unified interface for both one-
113136
It only performs live update when `live_mode` is `True`, and only for sources with change capture mechanisms enabled.
114137
If a source has multiple change capture mechanisms enabled, all will take effect to trigger updates.
115138

116-
<Tabs>
117-
<TabItem value="python" label="Python" default>
118-
119139
This creates a `cocoindex.FlowLiveUpdater` object, with an optional `cocoindex.FlowLiveUpdaterOptions` option:
120140

121141
```python
122142
my_updater = cocoindex.FlowLiveUpdater(
123143
demo_flow, cocoindex.FlowLiveUpdaterOptions(print_stats=True))
124144
```
125145

126-
</TabItem>
127-
</Tabs>
128-
129146
A `FlowLiveUpdater` object supports the following methods:
130147

131148
* `abort()`: Abort the updater.
@@ -135,9 +152,6 @@ A `FlowLiveUpdater` object supports the following methods:
135152
either `live_mode` is `False`, or all data sources have no change capture mechanisms enabled.
136153
* `update_stats()`: It returns the stats of the updater.
137154

138-
<Tabs>
139-
<TabItem value="python" label="Python" default>
140-
141155
```python
142156
my_updater = cocoindex.FlowLiveUpdater(demo_flow)
143157

@@ -175,23 +189,37 @@ with cocoindex.FlowLiveUpdater(demo_flow) as my_updater:
175189

176190
## Evaluate the flow
177191

178-
:::tip
192+
CocoIndex allows you to run the transformations defined by the flow without updating the target storage.
179193

180-
CLI equivalence: `cocoindex evaluate`
194+
<Tabs>
195+
<TabItem value="shell" label="Shell" default>
181196

182-
:::
197+
The `cocoindex evaluate` subcommand runs the transformation and dumps flow outputs.
198+
It takes the following options:
183199

184-
CocoIndex allows you to run the transformations defined by the flow without updating the target storage.
185-
The `evaluate_and_dump()` method supports this by dumping flow outputs to files.
200+
* `--output-dir` (optional): The directory to dump the result to. If not provided, it will use `eval_{flow_name}_{timestamp}`.
201+
* `--no-cache` (optional): By default, we use already-cached intermediate data if available.
202+
This flag will turn it off.
203+
Note that we only read existing cached data without updating the cache, even if it's turned on.
204+
205+
Example:
206+
207+
```sh
208+
python main.py cocoindex evaluate --output-dir ./eval_output
209+
```
210+
211+
</TabItem>
212+
<TabItem value="python" label="Python">
213+
214+
The `evaluate_and_dump()` method runs the transformation and dumps flow outputs to files.
186215

187216
It takes a `EvaluateAndDumpOptions` dataclass as input to configure, with the following fields:
188217

189218
* `output_dir` (type: `str`, required): The directory to dump the result to.
190219
* `use_cache` (type: `bool`, default: `True`): Use already-cached intermediate data if available.
191220
Note that we only read existing cached data without updating the cache, even if it's turned on.
192221

193-
<Tabs>
194-
<TabItem value="python" label="Python" default>
222+
Example:
195223

196224
```python
197225
demo_flow.evaluate_and_dump(EvaluateAndDumpOptions(output_dir="./eval_output"))

0 commit comments

Comments
 (0)