Skip to content

Commit 6a5c9b1

Browse files
yleifermanjpipkin1
andauthored
Update universal-connector-source.md (#5550)
* Update universal-connector-source.md Rephrase the text for variables to make it more user friendly. * Update universal-connector-source.md Fix references * Updates from review * Update universal-connector-source.md * Fix broken links --------- Co-authored-by: John Pipkin <[email protected]>
1 parent 320d3ca commit 6a5c9b1

File tree

1 file changed

+23
-23
lines changed

1 file changed

+23
-23
lines changed

docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ Invalid examples:
125125

126126
#### Request Headers
127127

128-
Include any HTTP request headers required by the vendor API. The key names are static text, but the values can access our [template feature](#template-dynamic-values) to make them dynamic.
128+
Include any HTTP request headers required by the vendor API. The key names are static text, but the values can access our [variables feature](#dynamic-values-variables) to make them dynamic.
129129

130130
| Example Header Key | Example Header Value |
131131
| :----------------- | :--------------------------- |
@@ -135,7 +135,7 @@ Include any HTTP request headers required by the vendor API. The key names are s
135135

136136
#### Request Parameters
137137

138-
Include any URL query parameters required by the vendor API. The key names are static text, but the values can access our [template feature](#template-dynamic-values) to make them dynamic.
138+
Include any URL query parameters required by the vendor API. The key names are static text, but the values can access our [variables feature](#dynamic-values-variables) to make them dynamic.
139139

140140
| Example Header Key | Example Header Value |
141141
| :----------------- | :--------------------------------------------- |
@@ -148,7 +148,7 @@ Examples URL encoded:
148148

149149
#### Request Body
150150

151-
This is optional and only used if the HTTP `POST` method is configured above. You can use this field to include any information in the HTTP request body. The data included in this field can access our [template feature](#template-dynamic-values).
151+
This is optional and only used if the HTTP `POST` method is configured above. You can use this field to include any information in the HTTP request body. The data included in this field can access our [dynamic values](#dynamic-values-variables).
152152

153153
</div>
154154
</details>
@@ -159,9 +159,9 @@ This is optional and only used if the HTTP `POST` method is configured above. Yo
159159

160160
#### Time Window
161161

162-
The source will provide both a start and end timestamp for you to [dynamically](#template-dynamic-values) use in your HTTP request. The window will only move forward if no errors are raised when collecting logs from the vendor API for the current window.
162+
The source will provide both a start and end timestamp for you to [dynamically](#dynamic-values-variables) use in your HTTP request. The window will only move forward if no errors are raised when collecting logs from the vendor API for the current window.
163163

164-
Use the [template feature](#template-dynamic-values) to include the window start and end timestamps within your HTTP request.
164+
Use the [dynamic values](#dynamic-values-variables) to include the window start and end timestamps within your HTTP request.
165165

166166
The start time is inclusive and the end time is exclusive as that is the behavior of most APIs.
167167

@@ -176,7 +176,7 @@ The start time is inclusive and the end time is exclusive as that is the behavio
176176
<details>
177177
<summary>HTTP Response Log Ingest Configuration</summary>
178178
<div>
179-
Select the format of the data returned by the vendor and configure how the source should break down the response into into individual logs with the correct timestamp.
179+
Select the format of the data returned by the vendor and configure how the source should break down the response into individual logs with the correct timestamp.
180180

181181
#### JSON with JPath
182182

@@ -330,7 +330,7 @@ Sources can be configured using UTF-8 encoded JSON files with the Collector Ma
330330
| sourceType | String | `"Universal Connector"` | Yes | Type of source. |
331331
| config | JSON Object | [Configuration object](#configuration-object) | Yes | Source type specific values. |
332332

333-
### Configuration Object
333+
### Configuration object
334334

335335
| Parameter | Type | Required | Default | Description | Example |
336336
| :------------------------- | :---------- | :------- | :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -377,17 +377,17 @@ Sources can be configured using UTF-8 encoded JSON files with the Collector Ma
377377
| clientRateLimitBurst | Integer | Yes | `1000` | The number of requests the source is allowed to burst. | `1000` |
378378
| pollingInterval | String | Yes | `"5m"` | Set how frequently to poll for new data. It must be between 5 minutes and 48 hours. | `"5m"` |
379379

380-
## Template Dynamic Values
380+
## Dynamic values variables
381381

382-
The source has the ability to template in dynamic text into the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API.
382+
The source has the ability to use dynamic values, like the window start time, in the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API.
383383

384384
The following fields values are allowed to access dynamic text from the template functions described in this section:
385385

386386
- HTTP Request Header Values
387387
- HTTP Request Parameter Values
388388
- HTTP Request Body
389389

390-
To start using the template with one of the supported config values listed above, you will need to enclose the template logic inside double curly braces `{{}}`. Any text outside these double curly braces will be treated as normal unmodified text.
390+
To start using the variables with one of the supported config values listed above, you will need to enclose the template logic inside double curly braces `{{}}`. Any text outside these double curly braces will be treated as normal unmodified text.
391391

392392
Here are some syntax examples calling functions with and without arguments:
393393

@@ -397,7 +397,7 @@ Here are some syntax examples calling functions with and without arguments:
397397
{{ .FunctionName "string argument 1" "string argument 2" }}
398398
```
399399

400-
**Available Template Functions**
400+
**Available variables**
401401

402402
- [WindowStartUTC](#windowstartutc)
403403
- [WindowStartLocation](#windowstartlocation)
@@ -406,9 +406,9 @@ Here are some syntax examples calling functions with and without arguments:
406406

407407
### WindowStartUTC
408408

409-
This function will template in the `start timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone.
409+
This variable will get the value in the `start timestamp` of the source window when the source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone.
410410

411-
The syntax for this function requires a timestamp format as a single argument. See the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp.
411+
The syntax for this function requires a timestamp format as a single argument. See the [Timestamp formatting](#timestamp-formatting) section for more information on how to format the timestamp.
412412

413413
```sh
414414
{{ .WindowStartUTC "<timestamp format>" }}
@@ -425,13 +425,13 @@ The syntax for this function requires a timestamp format as a single argument. S
425425

426426
### WindowStartLocation
427427

428-
This function is the same as [WindowStartUTC](#windowstartutc) except it has an additional argument to specify the timezone location.
428+
This variable is the same as [WindowStartUTC](#windowstartutc) except it has an additional argument to specify the timezone location.
429429

430430
:::sumo[Best Practice]
431431
We strongly recommend you always use `WindowStartUTC` instead of `WindowStartLocation`. Most vendors support and expect UTC timestamps when using their APIs.
432432
:::
433433

434-
Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for specifying the time zone in the first argument and refer to the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp.
434+
Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for specifying the time zone in the first argument and refer to the [Timestamp formatting](#timestamp-formatting) section for more information on how to format the timestamp.
435435

436436
```sh
437437
{{ .WindowStartLocation "<time zone location>" "<timestamp format>" }}
@@ -448,9 +448,9 @@ Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_t
448448

449449
### WindowEndUTC
450450

451-
This function will template in the `end timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone.
451+
This variable will get the value in the `end timestamp` of the source window when the source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone.
452452

453-
The syntax for this function requires a timestamp format as a single argument. Refer to the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp.
453+
The syntax for this variable requires a timestamp format as a single argument. Refer to the [Timestamp formatting](#timestamp-formatting) section for more information on how to format the timestamp.
454454

455455
```sh
456456
{{ .WindowEndUTC "<timestamp format>" }}
@@ -465,17 +465,17 @@ The syntax for this function requires a timestamp format as a single argument. R
465465
| `{{ .WindowEndUTC "epoch" }}` | `1709842556` |
466466
| `{{ .WindowEndUTC "epochMilli" }}` | `1709842556000` |
467467
| `lessThan:{{ .WindowEndUTC "2006-01-02T15:04:05.999Z07:00" }}` | `lessThan:2024-03-07T20:15:56.905Z` |
468-
| `lessThan:{{ .WindowEndUTC "yyyy-MM-ddTHH:mm:ss.SSSZ" }}` | `lessThan:2024-03-07T20:15:56.905Z` |
468+
| `{"startTime":"{{ .WindowEndUTC "yyyy-MM-ddTHH:mm:ss.SSSZ" }}"}` | `{"startTime":"{{ .WindowEndUTC "2024-03-07T20:15:56.905Z" }}"` |
469469

470470
### WindowEndLocation
471471

472-
This function is the same as [WindowEndUTC](#windowendutc) except it has an additional argument to specify the timezone location.
472+
This variable is the same as [WindowEndUTC](#windowendutc) except it has an additional argument to specify the timezone location.
473473

474474
:::sumo[Best Practice]
475475
We strongly recommend you always use `WindowEndUTC` instead of `WindowEndLocation`. Most vendors support and expect UTC timestamps when using their APIs.
476476
:::
477477

478-
Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for specifying the time zone in the first argument and refer to the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp.
478+
Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for specifying the time zone in the first argument and refer to the [Timestamp formatting](#timestamp-formatting) section for more information on how to format the timestamp.
479479

480480
```sh
481481
{{ .WindowEndLocation "<time zone location>" "<timestamp format>" }}
@@ -490,15 +490,15 @@ Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_t
490490
| `lessThan:{{ .WindowEndLocation "Europe/Berlin" "2006-01-02T15:04:05.999Z07:00" }}` | `lessThan:2024-03-07T21:15:56.905+01:00` |
491491
| `lessThan:{{ .WindowEndLocation "Europe/Berlin" "yyyy-MM-ddTHH:mm:ss.SSSZ" }}` | `lessThan:2024-03-07T21:15:56.905+01:00` |
492492

493-
## Timestamp Formatting
493+
## Timestamp formatting
494494

495495
The source uses the [Go programming language timestamp formatting](https://go.dev/src/time/format.go) and the Human-readable timestamp formatting. See the table below for references and examples.
496496

497497
:::sumo[Best Practice]
498498
We recommend using [this code snippet](https://goplay.tools/snippet/WTFe5ZLU9PO) as a quick way to locally test timestamp parsing with a format before configuring the source.
499499
:::
500500

501-
### Format Reference
501+
### Format reference
502502

503503
| Date Format | Reference Value | Human Readable Referencce Value |
504504
| :------------------------------------------ | :-------------------------------------------------------------------- | :--------------------------------------------------------------------- |
@@ -527,7 +527,7 @@ We recommend using [this code snippet](https://goplay.tools/snippet/WTFe5ZLU9PO)
527527
| Timezone Offset with Colon | `-07:00` | `-HH:mm` |
528528
| Timezone Abbreviated Name | `MST` | `zzz` |
529529

530-
### Format Examples
530+
### Format examples
531531

532532
| Standard | Timestamp in Log | Timestamp Format |
533533
| :-------------------- | :------------------------------- | :------------------------------------ |

0 commit comments

Comments
 (0)