From 17cd09ea965518be1c13ed54a12ff952f288a658 Mon Sep 17 00:00:00 2001 From: Yair Leiferman <85251538+yleiferman@users.noreply.github.com> Date: Thu, 3 Jul 2025 15:45:23 -0500 Subject: [PATCH 1/5] Update universal-connector-source.md Rephrase the text for variables to make it more user friendly. --- .../universal-connector-source.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md b/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md index de5a9e6267..2dd50d6da0 100644 --- a/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md +++ b/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md @@ -377,9 +377,9 @@ Sources can be configured using UTF-8 encoded JSON files with the Collector Ma | clientRateLimitBurst | Integer | Yes | `1000` | The number of requests the source is allowed to burst. | `1000` | | pollingInterval | String | Yes | `"5m"` | Set how frequently to poll for new data. It must be between 5 minutes and 48 hours. | `"5m"` | -## Template Dynamic Values +## Dynamic Values Variables -The source has the ability to template in dynamic text into the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API. +The source has the ability to use dynamic values, like the window start time, into the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API. The following fields values are allowed to access dynamic text from the template functions described in this section: @@ -387,7 +387,7 @@ The following fields values are allowed to access dynamic text from the template - HTTP Request Parameter Values - HTTP Request Body -To start using the template with one of the supported config values listed above, you will need to enclose the template logic inside double curly braces `{{}}`. Any text outside these double curly braces will be treated as normal unmodified text. +To start using the variables with one of the supported config values listed above, you will need to enclose the template logic inside double curly braces `{{}}`. Any text outside these double curly braces will be treated as normal unmodified text. Here are some syntax examples calling functions with and without arguments: @@ -397,7 +397,7 @@ Here are some syntax examples calling functions with and without arguments: {{ .FunctionName "string argument 1" "string argument 2" }} ``` -**Available Template Functions** +**Available Variables** - [WindowStartUTC](#windowstartutc) - [WindowStartLocation](#windowstartlocation) @@ -406,7 +406,7 @@ Here are some syntax examples calling functions with and without arguments: ### WindowStartUTC -This function will template in the `start timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone. +This variable will get the value in the `start timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone. The syntax for this function requires a timestamp format as a single argument. See the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp. @@ -425,7 +425,7 @@ The syntax for this function requires a timestamp format as a single argument. S ### WindowStartLocation -This function is the same as [WindowStartUTC](#windowstartutc) except it has an additional argument to specify the timezone location. +This variable is the same as [WindowStartUTC](#windowstartutc) except it has an additional argument to specify the timezone location. :::sumo[Best Practice] We strongly recommend you always use `WindowStartUTC` instead of `WindowStartLocation`. Most vendors support and expect UTC timestamps when using their APIs. @@ -448,9 +448,9 @@ Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_t ### WindowEndUTC -This function will template in the `end timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone. +This variable will get the value in the `end timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone. -The syntax for this function requires a timestamp format as a single argument. Refer to the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp. +The syntax for this variable requires a timestamp format as a single argument. Refer to the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp. ```sh {{ .WindowEndUTC "" }} @@ -465,11 +465,11 @@ The syntax for this function requires a timestamp format as a single argument. R | `{{ .WindowEndUTC "epoch" }}` | `1709842556` | | `{{ .WindowEndUTC "epochMilli" }}` | `1709842556000` | | `lessThan:{{ .WindowEndUTC "2006-01-02T15:04:05.999Z07:00" }}` | `lessThan:2024-03-07T20:15:56.905Z` | -| `lessThan:{{ .WindowEndUTC "yyyy-MM-ddTHH:mm:ss.SSSZ" }}` | `lessThan:2024-03-07T20:15:56.905Z` | +| `{"startTime":"{{ .WindowEndUTC "yyyy-MM-ddTHH:mm:ss.SSSZ" }}"}` | `{"startTime":"{{ .WindowEndUTC "2024-03-07T20:15:56.905Z" }}"` | ### WindowEndLocation -This function is the same as [WindowEndUTC](#windowendutc) except it has an additional argument to specify the timezone location. +This variable is the same as [WindowEndUTC](#windowendutc) except it has an additional argument to specify the timezone location. :::sumo[Best Practice] We strongly recommend you always use `WindowEndUTC` instead of `WindowEndLocation`. Most vendors support and expect UTC timestamps when using their APIs. From 966f0eeaac8be0f81cbd455cc6df3b1991cd2356 Mon Sep 17 00:00:00 2001 From: Yair Leiferman <85251538+yleiferman@users.noreply.github.com> Date: Thu, 3 Jul 2025 15:57:00 -0500 Subject: [PATCH 2/5] Update universal-connector-source.md Fix references --- .../universal-connector-source.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md b/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md index 2dd50d6da0..43092a8091 100644 --- a/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md +++ b/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md @@ -125,7 +125,7 @@ Invalid examples: #### Request Headers -Include any HTTP request headers required by the vendor API. The key names are static text, but the values can access our [template feature](#template-dynamic-values) to make them dynamic. +Include any HTTP request headers required by the vendor API. The key names are static text, but the values can access our [variables feature](#dynamic-values) to make them dynamic. | Example Header Key | Example Header Value | | :----------------- | :--------------------------- | @@ -135,7 +135,7 @@ Include any HTTP request headers required by the vendor API. The key names are s #### Request Parameters -Include any URL query parameters required by the vendor API. The key names are static text, but the values can access our [template feature](#template-dynamic-values) to make them dynamic. +Include any URL query parameters required by the vendor API. The key names are static text, but the values can access our [varables feature](#dynamic-values) to make them dynamic. | Example Header Key | Example Header Value | | :----------------- | :--------------------------------------------- | @@ -148,7 +148,7 @@ Examples URL encoded: #### Request Body -This is optional and only used if the HTTP `POST` method is configured above. You can use this field to include any information in the HTTP request body. The data included in this field can access our [template feature](#template-dynamic-values). +This is optional and only used if the HTTP `POST` method is configured above. You can use this field to include any information in the HTTP request body. The data included in this field can access our [dynamic values](#dynamic-values). @@ -159,9 +159,9 @@ This is optional and only used if the HTTP `POST` method is configured above. Yo #### Time Window -The source will provide both a start and end timestamp for you to [dynamically](#template-dynamic-values) use in your HTTP request. The window will only move forward if no errors are raised when collecting logs from the vendor API for the current window. +The source will provide both a start and end timestamp for you to [dynamically](#dynamic-values) use in your HTTP request. The window will only move forward if no errors are raised when collecting logs from the vendor API for the current window. -Use the [template feature](#template-dynamic-values) to include the window start and end timestamps within your HTTP request. +Use the [dynamic values](#dynamic-values) to include the window start and end timestamps within your HTTP request. The start time is inclusive and the end time is exclusive as that is the behavior of most APIs. @@ -377,7 +377,7 @@ Sources can be configured using UTF-8 encoded JSON files with the Collector Ma | clientRateLimitBurst | Integer | Yes | `1000` | The number of requests the source is allowed to burst. | `1000` | | pollingInterval | String | Yes | `"5m"` | Set how frequently to poll for new data. It must be between 5 minutes and 48 hours. | `"5m"` | -## Dynamic Values Variables +## Dynamic Values The source has the ability to use dynamic values, like the window start time, into the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API. From 4a773f42d8f0cbc76c73b2e2ebf2574940965d11 Mon Sep 17 00:00:00 2001 From: John Pipkin Date: Thu, 3 Jul 2025 16:02:16 -0500 Subject: [PATCH 3/5] Updates from review --- .../universal-connector-source.md | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md b/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md index 2dd50d6da0..3488138837 100644 --- a/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md +++ b/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/universal-connector-source.md @@ -176,7 +176,7 @@ The start time is inclusive and the end time is exclusive as that is the behavio
HTTP Response Log Ingest Configuration
- Select the format of the data returned by the vendor and configure how the source should break down the response into into individual logs with the correct timestamp. + Select the format of the data returned by the vendor and configure how the source should break down the response into individual logs with the correct timestamp. #### JSON with JPath @@ -330,7 +330,7 @@ Sources can be configured using UTF-8 encoded JSON files with the Collector Ma | sourceType | String | `"Universal Connector"` | Yes | Type of source. | | config | JSON Object | [Configuration object](#configuration-object) | Yes | Source type specific values. | -### Configuration Object +### Configuration object | Parameter | Type | Required | Default | Description | Example | | :------------------------- | :---------- | :------- | :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -377,9 +377,9 @@ Sources can be configured using UTF-8 encoded JSON files with the Collector Ma | clientRateLimitBurst | Integer | Yes | `1000` | The number of requests the source is allowed to burst. | `1000` | | pollingInterval | String | Yes | `"5m"` | Set how frequently to poll for new data. It must be between 5 minutes and 48 hours. | `"5m"` | -## Dynamic Values Variables +## Dynamic values variables -The source has the ability to use dynamic values, like the window start time, into the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API. +The source has the ability to use dynamic values, like the window start time, in the values of certain fields providing flexibility in crafting the HTTP requests sent to the vendor API. The following fields values are allowed to access dynamic text from the template functions described in this section: @@ -397,7 +397,7 @@ Here are some syntax examples calling functions with and without arguments: {{ .FunctionName "string argument 1" "string argument 2" }} ``` -**Available Variables** +**Available variables** - [WindowStartUTC](#windowstartutc) - [WindowStartLocation](#windowstartlocation) @@ -406,9 +406,9 @@ Here are some syntax examples calling functions with and without arguments: ### WindowStartUTC -This variable will get the value in the `start timestamp` of the source window when source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone. +This variable will get the value in the `start timestamp` of the source window when the source is configured to use the `Time Window` progression. The timestamp will always use `UTC` time and never adjust for a specific timezone. -The syntax for this function requires a timestamp format as a single argument. See the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp. +The syntax for this function requires a timestamp format as a single argument. See the [Timestamp formatting](#timestamp-formatting) section for more information on how to format the timestamp. ```sh {{ .WindowStartUTC "" }} @@ -431,7 +431,7 @@ This variable is the same as [WindowStartUTC](#windowstartutc) except it has an We strongly recommend you always use `WindowStartUTC` instead of `WindowStartLocation`. Most vendors support and expect UTC timestamps when using their APIs. ::: -Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for specifying the time zone in the first argument and refer to the [Timestamp Formatting](#timestamp-formatting) section for more information on how to format the timestamp. +Refer to the [TZ identifier](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for specifying the time zone in the first argument and refer to the [Timestamp formatting](#timestamp-formatting) section for more information on how to format the timestamp. ```sh {{ .WindowStartLocation "
@@ -159,9 +159,9 @@ This is optional and only used if the HTTP `POST` method is configured above. Yo #### Time Window -The source will provide both a start and end timestamp for you to [dynamically](#dynamic-values) use in your HTTP request. The window will only move forward if no errors are raised when collecting logs from the vendor API for the current window. +The source will provide both a start and end timestamp for you to [dynamically](#dynamic-values-variables) use in your HTTP request. The window will only move forward if no errors are raised when collecting logs from the vendor API for the current window. -Use the [dynamic values](#dynamic-values) to include the window start and end timestamps within your HTTP request. +Use the [dynamic values](#dynamic-values-variables) to include the window start and end timestamps within your HTTP request. The start time is inclusive and the end time is exclusive as that is the behavior of most APIs.