Skip to content

Commit 6046c2b

Browse files
authored
Update documentation for Cloud Object Storage source (#19)
* Update documentation. * Clean-up.
1 parent b50f576 commit 6046c2b

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

docs/connectors.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,7 @@ This source manages the receiving of objects from AWS S3 or IBM Cloud Object Sto
131131
- `endpoint` the name of the public endpoint from the bucket configuration qualified by the URI scheme (for example, `https://`);
132132
- `region` (optional) the region of the bucket, if left empty the region will be automatically parsed by Rayvens from the endpoint;
133133
- `move_after_read` (optional) enables moving of any read file from bucket specified by `bucket_name` to new bucket that is the value of this field. If using the Cloud Object Storage, the new bucket has to be created ahead of time. The new bucket name is provided as the value of this field;
134+
- `meta_event_only` (optional) this option only allows for the filename to be propagated;
134135

135136
This source will fetch and delete the files in the bucket unless the `move_after_read` option is enabled in which case the read files are moved to the bucket specified by the `move_after_read` field value.
136137

@@ -153,6 +154,24 @@ source_config = dict(kind='cloud-object-storage-source',
153154
move_after_read="new-bucket-name")
154155
```
155156

157+
The event returned by this source is a JSON string of the form:
158+
159+
```
160+
{"filename": <filename_value>, "body": <body_value>}
161+
```
162+
163+
Convert the event to JSON when handling the event:
164+
165+
```
166+
import json
167+
json_event = json.loads(event)
168+
```
169+
and then access its field like so:
170+
```
171+
json_event['filename']
172+
json_event['body']
173+
```
174+
156175
### `kind="file-source"`
157176

158177
This source reads a file from the file system when the file becomes available. The event is triggered automatically whenever the file becomes available. The file content is read to memory and returned by the source to be processed further by the application. This source has the following options:

0 commit comments

Comments
 (0)