Skip to content

Commit 4cbe48d

Browse files
committed
docs/reference/filebeat: document run_as_cel and cel.* options
Add reference documentation for the run_as_cel option and the cel.program, cel.state, cel.max_executions, cel.regexp, cel.xsd, and cel.redact configuration fields. Includes a field mapping table, a note on cursor state migration, and a worked example.
1 parent 9dd3384 commit 4cbe48d

File tree

1 file changed

+118
-1
lines changed

1 file changed

+118
-1
lines changed

docs/reference/filebeat/filebeat-input-httpjson.md

Lines changed: 118 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ applies_to:
1212

1313
Use the `httpjson` input to read messages from an HTTP API with JSON payloads.
1414

15-
If you are starting development of a new custom HTTP API input, we recommend that you use the [Common Expression Language input](/reference/filebeat/filebeat-input-cel.md) which provides greater flexibility and an improved developer experience.
15+
If you are starting development of a new custom HTTP API input, we recommend that you use the [Common Expression Language input](/reference/filebeat/filebeat-input-cel.md) which provides greater flexibility and an improved developer experience. Existing `httpjson` inputs can be migrated to CEL using the [`run_as_cel`](#run-as-cel) option.
1616

1717
This input supports:
1818

@@ -1728,6 +1728,123 @@ filebeat.inputs:
17281728
```
17291729

17301730

1731+
### `run_as_cel` [run-as-cel]
1732+
1733+
```{applies_to}
1734+
stack: ga 9.4.0
1735+
```
1736+
1737+
When set to `true`, the input is transparently redirected to the [Common Expression Language input](/reference/filebeat/filebeat-input-cel.md). This allows an existing `httpjson` configuration to run under the CEL engine without changing the input `type`. A `cel.program` must be provided when this option is enabled.
1738+
1739+
Shared configuration fields are automatically translated to their CEL equivalents:
1740+
1741+
| httpjson field | CEL field |
1742+
| --- | --- |
1743+
| `interval` | `interval` |
1744+
| `id` | `id` |
1745+
| `request.url` | `resource.url` |
1746+
| `request.timeout` | `resource.timeout` |
1747+
| `request.ssl` | `resource.ssl` |
1748+
| `request.proxy_url` | `resource.proxy_url` |
1749+
| `request.proxy_headers` | `resource.proxy_headers` |
1750+
| `request.proxy_disable` | `resource.proxy_disable` |
1751+
| `request.idle_connection_timeout` | `resource.idle_connection_timeout` |
1752+
| `request.keep_alive` | `resource.keep_alive` |
1753+
| `request.retry` | `resource.retry` |
1754+
| `request.redirect` | `resource.redirect` |
1755+
| `request.tracer` | `resource.tracer` |
1756+
| `auth` | `auth` |
1757+
1758+
Fields that are specific to httpjson (such as `request.transforms`, `response.transforms`, `response.split`, `response.pagination`, and `chain`) are not transferred and have no effect when `run_as_cel` is enabled. The CEL program is responsible for equivalent logic.
1759+
1760+
If the httpjson input has existing cursor state, it is automatically carried over to the CEL input on the first run. On subsequent runs the CEL input uses its own stored cursor. Removing `run_as_cel` restores the original httpjson behaviour with its last cursor intact.
1761+
1762+
Default: `false`.
1763+
1764+
1765+
### `cel.program` [cel-program]
1766+
1767+
```{applies_to}
1768+
stack: ga 9.4.0
1769+
```
1770+
1771+
The CEL program to execute when [`run_as_cel`](#run-as-cel) is enabled. This is the same program format used by the [CEL input's `program` field](/reference/filebeat/filebeat-input-cel.md). Required when `run_as_cel` is `true`.
1772+
1773+
1774+
### `cel.state` [cel-state]
1775+
1776+
```{applies_to}
1777+
stack: ga 9.4.0
1778+
```
1779+
1780+
Initial state for the CEL program, equivalent to the [CEL input's `state` field](/reference/filebeat/filebeat-input-cel.md). May include an initial `cursor` object that is used as the bootstrap value on the first execution when no stored cursor exists.
1781+
1782+
1783+
### `cel.max_executions` [cel-max-executions]
1784+
1785+
```{applies_to}
1786+
stack: ga 9.4.0
1787+
```
1788+
1789+
The maximum number of CEL program executions per interval. Equivalent to the [CEL input's `max_executions` field](/reference/filebeat/filebeat-input-cel.md). Default: `1000`.
1790+
1791+
1792+
### `cel.regexp` [cel-regexp]
1793+
1794+
```{applies_to}
1795+
stack: ga 9.4.0
1796+
```
1797+
1798+
A map of named regular expression patterns available to the CEL program. Equivalent to the [CEL input's `regexp` field](/reference/filebeat/filebeat-input-cel.md).
1799+
1800+
1801+
### `cel.xsd` [cel-xsd]
1802+
1803+
```{applies_to}
1804+
stack: ga 9.4.0
1805+
```
1806+
1807+
A map of named XSD schemas for XML decoding in the CEL program. Equivalent to the [CEL input's `xsd` field](/reference/filebeat/filebeat-input-cel.md).
1808+
1809+
1810+
### `cel.redact` [cel-redact]
1811+
1812+
```{applies_to}
1813+
stack: ga 9.4.0
1814+
```
1815+
1816+
Redaction configuration for the CEL program. Equivalent to the [CEL input's `redact` field](/reference/filebeat/filebeat-input-cel.md).
1817+
1818+
::::{admonition} Example: migrating an httpjson input to CEL
1819+
```yaml
1820+
filebeat.inputs:
1821+
- type: httpjson
1822+
id: my-api-input
1823+
interval: 60s
1824+
run_as_cel: true
1825+
request.url: https://api.example.com/events
1826+
auth.oauth2:
1827+
client.id: my-client-id
1828+
client.secret: my-client-secret
1829+
token_url: https://auth.example.com/oauth2/token
1830+
request.ssl.verification_mode: full
1831+
request.timeout: 30s
1832+
cel.program: |
1833+
state.url.with({
1834+
"Header": {"Accept": ["application/json"]},
1835+
}).as(req, request("GET", req).as(resp,
1836+
bytes(resp.Body).decode_json().as(body, {
1837+
"events": body.items.map(e, {"message": e.encode_json()}),
1838+
"cursor": {"since": body.items[body.items.size()-1].updated_at},
1839+
})
1840+
))
1841+
cel.state:
1842+
cursor:
1843+
since: "2024-01-01T00:00:00Z"
1844+
```
1845+
::::
1846+
1847+
17311848
## Request life cycle [_request_life_cycle]
17321849

17331850
![Request lifecycle](images/input-httpjson-lifecycle.png "")

0 commit comments

Comments
 (0)