Skip to content

Commit 788f5b3

Browse files
authored
docs: rule and stream update (#3896)
Signed-off-by: Jiyong Huang <huangjy@emqx.io>
1 parent 76c5c53 commit 788f5b3

File tree

19 files changed

+474
-380
lines changed

19 files changed

+474
-380
lines changed

docs/en_US/api/restapi/rules.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -100,9 +100,10 @@ Example response when using slice mode:
100100
}
101101
```
102102

103-
## update a rule
103+
## upsert a rule
104104

105-
The API accepts a JSON content and update a rule.
105+
The API accepts a JSON content and upsert a rule which means if the rule is not existed, create it; otherwise, update
106+
it. If update fails, the original rule will continue running.
106107

107108
```shell
108109
PUT http://localhost:9081/rules/{id}
@@ -132,15 +133,19 @@ DELETE http://localhost:9081/rules/{id}
132133

133134
## start a rule
134135

135-
The API is used to start running the rule.
136+
The API is used to start running the rule. Please note that the command only indicates the successful transmission of
137+
the start instruction. To verify if the rule has completed startup, the rule status must be checked. If the rule is
138+
currently in the process of starting or stopping, the start instruction will be added to the rule's command queue.
136139

137140
```shell
138141
POST http://localhost:9081/rules/{id}/start
139142
```
140143

141144
## stop a rule
142145

143-
The API is used to stop running the rule.
146+
The API is used to stop running the rule. Please note that the command only indicates the successful transmission of the
147+
stop instruction. To verify if the rule has completed startup, the rule status must be checked. If the rule is currently
148+
in the process of starting or stopping, the start instruction will be added to the rule's command queue.
144149

145150
```shell
146151
POST http://localhost:9081/rules/{id}/stop

docs/en_US/api/restapi/schemas.md

Lines changed: 2 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -117,23 +117,8 @@ PUT http://localhost:9081/schemas/protobuf/{name}
117117
The schema can have an optional **version** field. This field is essential for controlling updates and ensuring that new
118118
schemas are correctly applied. When you update a schema, the system compares the new version string to the existing one.
119119
An update is only accepted if the new version is **lexically greater** than the old one. This comparison is a
120-
character-by-character string comparison, not a numerical one.
121-
122-
### Versioning Logic
123-
124-
- **No Version Specified:** If neither the old nor the new schema has a `version` field, the update will proceed. This
125-
behavior aligns with the original, unversioned logic.
126-
- **Versioning Set:** If a `version` field is present in either the old or the new schema, the system will always
127-
perform a version comparison. The presence of any version string triggers the new comparison logic.
128-
- **Lexical Comparison:** Updates are based on a lexical (string) comparison. The new schema's `version` must be
129-
lexicographically greater than the current one for the update to be successful.
130-
- **Smallest Version:** A schema without a `version` field is considered to have the "smallest possible" version. This
131-
means that adding a version field to an existing, unversioned schema will always result in a successful update, as any
132-
new version string will be lexically greater than the non-existent one.
133-
134-
To avoid confusion and ensure correct ordering, it's highly recommended to use a **timestamp** as the version string.
135-
Timestamps, such as Unix epoch time, provide a universally unique and monotonically increasing value that naturally
136-
satisfies the lexical comparison rule.
120+
character-by-character string comparison, not a numerical one. The control logic for all versioned APIs is the same;
121+
please refer to the [Versioning Logic](../../guide/rules/overview.md#versioning-logic) for details.
137122

138123
Below is an example schema request with version:
139124

docs/en_US/concepts/rules.md

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,54 @@ Each rule represents a computing job to run in eKuiper. It defines the continuou
44

55
## Rule Lifecycle
66

7-
Currently, eKuiper only supports stream processing rule, which means that at lease one of the rule source must be a continuous stream. Thus, the rule will run continuously once started and only stopped if the user send stop command explicitly. The rule may stop abnormally for errors or the eKuiper instance exits.
7+
eKuiper currently **only supports Streaming Rules**. These types of rules require at least one Continuous Stream as a
8+
data source.
9+
10+
Once a rule is started, it will run continuously until:
11+
12+
1. The user explicitly sends a stop command.
13+
2. The rule abnormally terminates due to an internal error or the eKuiper instance exiting.
14+
15+
**Asynchronous Rule Startup and Status Management**
16+
17+
The rule startup process is **asynchronous**. When a user sends a start command, eKuiper performs necessary static
18+
checks and then asynchronously executes the rule's startup operation. Therefore:
19+
20+
* The command response received by the user only indicates that eKuiper has accepted the startup request and set the
21+
rule's **Expected Status** to 'started'.
22+
* This does not mean the rule has begun running. Users need to further check the rule's **Runtime Status** to confirm
23+
that the rule has successfully started and is running.
24+
25+
**Rule Update and Error Rollback**
26+
27+
During the rule update process, eKuiper provides **rollback** support. If the updated rule fails to start, the system
28+
will automatically maintain and continue running the original old rule to ensure service stability.
829

930
## Rules Relationship
1031

1132
It is common to run multiple rules simultaneously. As eKuiper is a single instance process, the rules are running in the same memory space. However, there are separated in the runtime and the error in one rule should not affect others. Regarding workload, all rules share the same hardware resource. Each rule can specify the operator buffer to limit the processing rate to avoid taking all resources.
1233

34+
When multiple rules use a **[Shared Stream](../guide/streams/overview.md#share-source-instance-across-rules)**, they
35+
share the upstream source components, including data ingestion and decoding.
36+
37+
In execution, all rules utilizing a shared stream form a single **Directed Acyclic Graph (DAG)** where downstream rules
38+
can be dynamically added or removed.
39+
40+
**Impact of Shared Streams**
41+
42+
Due to this shared structure, rules within the DAG will influence each other. Specifically:
43+
44+
* **Backpressure Propagation:** Backpressure originating from one rule can propagate backward through the shared source
45+
component.
46+
* **Wider Impact:** This backpressure on the shared stream will then affect the performance and processing of **all**
47+
other rules connected to that same shared source.
48+
49+
Besides this, the shared source side ignores checkpoint.
50+
1351
## Rule Pipeline
1452

1553
Multiple rules can form a processing pipeline by specifying a joint point in sink/source. For example, the first rule produce the result to a topic in memory sink and the other rule subscribe to that topic in its memory source. Besides the pair of memory sink/source, users can also use mqtt or other sink/source pair to connect rules.
1654

1755
## More Readings
1856

19-
- [Rule Reference](../guide/rules/overview.md)
57+
* [Rule Reference](../guide/rules/overview.md)

docs/en_US/configuration/global_configurations.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ basic:
4646
rulePatrolInterval: 10s
4747
# cfgStorageType indicates the storage type to store the config, support `file` and `kv`. When `cfgStorageType` is file, it will save configuration into File. When `cfgStorageType` is `kv`, it will save configuration into the storage defined in `store`
4848
cfgStorageType: file
49+
# If it is enabled, each REST API call will print logs
50+
enableRestAuditLog: false
4951
```
5052
5153
for debug option in basic following env is valid `KUIPER__BASIC__DEBUG=true` and if used debug value will be set to true.
@@ -340,7 +342,9 @@ This section configures the portable plugin runtime.
340342

341343
## Ruleset Provision
342344

343-
Support file based stream and rule provisioning on startup. Users can put a [ruleset](../api/restapi/ruleset.md#ruleset-format) file named `init.json` into `data` directory to initialize the ruleset. The ruleset will only be import on the first startup of eKuiper.
345+
Support file based stream and rule provisioning on startup. Users can put
346+
a [ruleset](../api/restapi/ruleset.md#ruleset-format) file named `init.json` into `etc` directory to initialize the
347+
ruleset. The ruleset will only be import on the first startup of eKuiper.
344348

345349
## Configure FoundationDB as storage
346350

0 commit comments

Comments
 (0)