Skip to content

Commit f73217f

Browse files
authored
HTTP datasource: document body format, response code and supported methods (#824)
1 parent 0b7fe9f commit f73217f

File tree

1 file changed

+39
-11
lines changed
  • crowdsec-docs/docs/log_processor/data_sources

1 file changed

+39
-11
lines changed

crowdsec-docs/docs/log_processor/data_sources/http.md

Lines changed: 39 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -63,29 +63,61 @@ labels:
6363
type: mytype
6464
```
6565
66-
:::info
67-
If most of cases when the logs are sent in JSON format, you can use the [`transform`](https://docs.crowdsec.net/docs/next/data_sources/intro/#transform) expression to parse the logs.
68-
:::
66+
Look at the `Parameters` section to view all supported options.
67+
68+
## Body format
69+
70+
The datasource expects to receive one or multiple JSON objects.
71+
72+
The datasource will also automatically decompress any request body in `gzip` format, as long as the `Content-Encoding` header is set to `gzip`.
73+
74+
The JSON object can be any format, crowdsec will pass it as-is to the parsers.
75+
76+
If you are sending multiple JSON object in the same request, they must be separated by a newline (NDJSON format):
77+
```json
78+
{"log": "log line 1", "timestamp": "2021-01-01T00:00:00Z"}
79+
{"log": "log line 2", "timestamp": "2021-01-01T00:00:01Z"}
80+
```
81+
82+
The objects will be processed by the parsers one-by-one.
83+
84+
If you send multiple log lines in a single JSON object, you can use a [transform](/docs/log_processor/data_sources/introduction.md#transform) expression to generate multiple events:
6985

70-
For example, if the logs are sent in the following format:
7186
```json
7287
{
7388
"Records": [
7489
{
7590
"message": "test",
7691
"timestamp": "2021-01-01T00:00:00Z"
92+
},
93+
{
94+
"message": "test2",
95+
"timestamp": "2021-01-01T00:00:01Z"
7796
}
7897
]
7998
}
8099
```
81100

82-
the `transform` expression can be:
101+
Using the following `transform` expression will make the datasource generate one event per entry in the array:
83102
```yaml
84-
transform: map(JsonExtractSlice(evt.Line.Raw, "Records"), ToJsonString(#))
103+
transform: |
104+
map(JsonExtractSlice(evt.Line.Raw, "Records"), ToJsonString(#))
85105
```
86106

87107

88-
Look at the `configuration parameters` to view all supported options.
108+
## Status code and supported methods
109+
110+
The HTTP datasource expects to receive logs in a `POST` request, and will return a `200 OK`.
111+
112+
If an invalid body is received (invalid JSON), a `400 Bad Request` code will be returned.
113+
114+
The datasource will return a `200 OK` to `GET` and `HEAD` requests if the credentials provided in the request are valid.
115+
116+
A `405 Method Not Allowed` code will be returned for any other methods.
117+
118+
If the credentials provided are invalid, a `401 Unauthorized` code will be returned.
119+
120+
If the body size is bigger than the configured limit, a `413 Request Entity Too Large` code will be returned.
89121

90122
## Parameters
91123

@@ -106,10 +138,6 @@ At least one of `listen_addr` or `listen_socket` is required.
106138

107139
The endpoint path to listen on.
108140

109-
:::info
110-
The request method is always `POST`.
111-
:::
112-
113141
Optional, default is `/`.
114142

115143
### `auth_type`

0 commit comments

Comments
 (0)