Skip to content

Commit b5566b1

Browse files
authored
docs: add more code snippets for the custom target doc (#870)
1 parent 098ecd9 commit b5566b1

File tree

1 file changed

+89
-0
lines changed

1 file changed

+89
-0
lines changed

docs/docs/custom_ops/custom_targets.mdx

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,95 @@ def prepare(spec: CustomTarget) -> PreparedCustomTarget:
158158

159159
If not provided, the original spec will be passed directly to `mutate`.
160160

161+
### Complete Example
162+
163+
In this example, we define a custom target that accepts data with the following fields:
164+
- `filename` (key field)
165+
- `author` (value field)
166+
- `html` (value field)
167+
168+
<Tabs>
169+
<TabItem value="python" label="Python" default>
170+
171+
```python
172+
import dataclasses
173+
import cocoindex
174+
175+
# 1. Define the target spec
176+
class MyCustomTarget(cocoindex.op.TargetSpec):
177+
"""Spec of the custom target, to configure the target location etc."""
178+
location: str
179+
180+
# 2. Define the value dataclass for exported data
181+
@dataclasses.dataclass
182+
class LocalFileTargetValues:
183+
"""Represents value fields of exported data."""
184+
author: str
185+
html: str
186+
187+
# 3. Define the target connector
188+
@cocoindex.op.target_connector(spec_cls=MyCustomTarget)
189+
class LocalFileTargetConnector:
190+
@staticmethod
191+
def get_persistent_key(spec: MyCustomTarget, target_name: str) -> str:
192+
return spec.location
193+
194+
@staticmethod
195+
def apply_setup_change(
196+
key: str, previous: MyCustomTarget | None, current: MyCustomTarget | None
197+
) -> None:
198+
# Setup/teardown logic here
199+
...
200+
201+
@staticmethod
202+
def mutate(
203+
*all_mutations: tuple[MyCustomTarget, dict[str, LocalFileTargetValues | None]],
204+
) -> None:
205+
"""Apply data mutations to the target."""
206+
for spec, mutations in all_mutations:
207+
for filename, mutation in mutations.items():
208+
if mutation is None:
209+
# Delete the file
210+
...
211+
else:
212+
# Write the file with author and html content
213+
...
214+
215+
# 4. Usage in a flow
216+
@cocoindex.flow_def(name="ExampleFlow")
217+
def example_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope) -> None:
218+
# Add data source
219+
data_scope["documents"] = flow_builder.add_source(...)
220+
221+
# Create collector
222+
output_data = data_scope.add_collector()
223+
224+
# Collect data
225+
with data_scope["documents"].row() as doc:
226+
# Create the "author" and "fieldname" field
227+
...
228+
229+
# Collect the data
230+
output_data.collect(filename=doc["filename"], author=doc["author"], html=doc["transformed_html"])
231+
232+
# Export to custom target
233+
output_data.export(
234+
"OutputData",
235+
MyCustomTarget(location=...),
236+
primary_key_fields=["filename"],
237+
)
238+
```
239+
240+
In this example, the type for data in `all_mutations` is `dict[str, LocalFileTargetValues | None]`:
241+
- `str` is the `DataKeyType` (the filename)
242+
- `LocalFileTargetValues` is the `DataValueType` (containing `html` and `author` fields)
243+
- The `mutate()` method receives tuples of `(MyCustomTarget, dict[str, LocalFileTargetValues | None])`
244+
245+
For simplicity, the type hints can be omitted and a `dict` will be created instead of a dataclass instance, and `author` and `html` will be the keys of the dict.
246+
247+
</TabItem>
248+
</Tabs>
249+
161250
## Best Practices
162251

163252
### Idempotency of Methods with Side Effects

0 commit comments

Comments
 (0)