Skip to content

Commit f676253

Browse files
kimsaucejpipkin1
andauthored
Replace otelcol timestamp /reuse reference (#5566)
* Replace otelcol timestamp /reuse reference * cross-references * Update docs/send-data/opentelemetry-collector/remote-management/source-templates/postgresql/index.md * Update docs/send-data/opentelemetry-collector/remote-management/source-templates/rabbitmq/index.md * Update docs/send-data/opentelemetry-collector/remote-management/source-templates/redis/index.md * Update docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/reuse/apps/opentelemetry/timestamp-parsing.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> --------- Co-authored-by: John Pipkin (Sumo Logic) <[email protected]>
1 parent 44723f5 commit f676253

File tree

15 files changed

+45
-27
lines changed

15 files changed

+45
-27
lines changed

docs/reuse/apps/opentelemetry/logs-advance-option-otel.md

Lines changed: 0 additions & 5 deletions
This file was deleted.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
**Timestamp Parsing**. You can configure timestamp parsing for logs ingested using this source template. For more information, see [Timestamps, Time Zones, Time Ranges, and Date Formats for the OpenTelemetry Collector](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference).

docs/send-data/opentelemetry-collector/remote-management/source-templates/apache/index.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,9 @@ In this step, you will configure the yaml required for Apache Collection. Below
7777
- **Error file log path**. Enter the path to the error log file for your Apache instance.
7878
- **Fields/Metadata**. You can provide any customer fields to be tagged with the data collected. By default, sumo tags `_sourceCategory` with the value otel/apache user needs to provide the value for `webengine.cluster.name`.
7979

80-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
80+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
81+
82+
<TimestampParsing/>
8183

8284
**Processing Rules**. You can add **processing rules** for logs/metrics collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
8385

docs/send-data/opentelemetry-collector/remote-management/source-templates/docker/index.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,9 @@ In this step, you will configure the yaml required for Docker Collection. Below
7777
- **Excluded Image List**. A list of strings, [regexes](https://golang.org/pkg/regexp/), or [globs](https://github.com/gobwas/glob) whose referent container image names will not be among the queried containers for scrapping metrics. Learn more about [*excluded_images*](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/dockerstatsreceiver/README.md#configuration).
7878
- **Fields/Metadata**. You can provide any customer fields to be tagged with the data collected. By default, Sumo Logic tags `_sourceCategory` with the value otel/docker.
7979

80-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
80+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
81+
82+
<TimestampParsing/>
8183

8284
**Processing Rules**. You can add **processing rules** for logs/metrics collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
8385

docs/send-data/opentelemetry-collector/remote-management/source-templates/elasticsearch/index.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,15 +71,16 @@ import CollectorInstallation from '../../../../../reuse/apps/opentelemetry/colle
7171
In this step, you will configure the yaml required for Elasticsearch collection. Below are the inputs required for configuration:
7272

7373
- **Name**. Name of the source template.
74-
- **Description**. Description for the source template.
74+
- **Description**. Description for the source template.
7575
- **Log Filepath**. Location where the Elasticsearch logs are logged. Please refer to your elasticsearch.conf file.
7676
- **Endpoint**. Enter the url of the server you need to monitor. (default: `localhost:9200`).
7777
- **Username**. Enter the Elasticsearch username.
7878
- **Password Environment Variable Name**. Enter the Elasticsearch password environment variable name.
7979
- **Fields/Metadata**. You can provide any customer fields to be tagged with the data collected. By default, Sumo Logic tags `_sourceCategory` with the value otel/elasticsearch user needs to provide the value for `db.cluster.name`.
8080

81-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
81+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
8282

83+
<TimestampParsing/>
8384

8485
**Processing Rules**. You can add **processing rules** for logs/metrics collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
8586

@@ -91,4 +92,4 @@ import DataConfiguration from '../../../../../reuse/apps/opentelemetry/data-conf
9192

9293
:::info
9394
Refer to the [changelog](changelog.md) for information on periodic updates to this source template.
94-
:::
95+
:::

docs/send-data/opentelemetry-collector/remote-management/source-templates/kafka/index.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,9 @@ In this step, you will configure the yaml required for Kafka collection. Below a
6868
- **Endpoint**. The URL of the broker endpoint (default: `localhost:9092`).
6969
- **Fields/Metadata**. You can provide any customer fields to be tagged with the data collected. By default, Sumo Logic tags `_sourceCategory` with the value otel/kafka user needs to provide the value for `webengine.cluster.name`.
7070

71-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
71+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
7272

73+
<TimestampParsing/>
7374

7475
**Processing Rules**. You can add **processing rules** for logs/metrics collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
7576

docs/send-data/opentelemetry-collector/remote-management/source-templates/localfile/index.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,9 @@ In this step, you will configure the yaml required for Local File collection. Be
5555
- **Collection should begin from**. Defines where will the collection of the logs start from. Possible values are "End of File" and "Beginning of File".
5656
- **Detect messages spanning multiple lines**. You can enable this option when dealing with logs which span over multiple lines. On enabling this option you will need to specify **Boundary regex location** where you can specify if the expression defines end or start of the log line and **Expression to match message boundary** where you will define the expression.
5757

58-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
58+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
5959

60+
<TimestampParsing/>
6061

6162
**Processing Rules**. You can add processing rules for logs collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
6263

docs/send-data/opentelemetry-collector/remote-management/source-templates/mysql/index.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -80,16 +80,17 @@ import CollectorInstallation from '../../../../../reuse/apps/opentelemetry/colle
8080
In this step, you will configure the yaml required for MySQL collection. Below are the inputs required for configuration:
8181

8282
- **Name**. Name of the source template.
83-
- **Description**. Description for the source template.
83+
- **Description**. Description for the source template.
8484
- **Error log path**. Location where the SQL Errors are logged. Please refer to your my.cnf file.
8585
- **Slow Transaction log file path (optional)**. Location where the Slow SQL transactions are logged. Please refer to your my.cnf file.
8686
- **Endpoint**. The URL of the MySQL endpoint (default: `localhost:3306`).
8787
- **Username**. Enter the MySQL username.
8888
- **Password Environment Variable Name**. Enter the MySQL password environment variable name.
8989
- **Fields/Metadata**. You can provide any customer fields to be tagged with the data collected. By default, Sumo Logic tags `_sourceCategory` with the value otel/mysql user needs to provide the value for `db.cluster.name`.
9090

91-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
91+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
9292

93+
<TimestampParsing/>
9394

9495
**Processing Rules**. You can add **processing rules** for logs/metrics collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
9596

@@ -101,4 +102,4 @@ import DataConfiguration from '../../../../../reuse/apps/opentelemetry/data-conf
101102

102103
:::info
103104
Refer to the [changelog](changelog.md) for information on periodic updates to this source template.
104-
:::
105+
:::

docs/send-data/opentelemetry-collector/remote-management/source-templates/nginx/index.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,9 @@ In this step, you will configure the yaml required for Nginx collection. Below a
7676
- **Path to Nginx error Log file**. Enter the path to the error log file for your Nginx instance.
7777
- **Fields/Metadata**. You can provide any customer fields to be tagged with the data collected. By default sumo tags `_sourceCategory` with the value otel/nginx user needs to provide the value for `webengine.cluster.name`.
7878

79-
**Timestamp Parsing**. You can do timestamp parsing for log ingested using this source template. For more details, refer to [Timestamps, Time Zones, and Date Formats](/docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference) for OpenTelemetry collector.
79+
import TimestampParsing from '../../../../../reuse/apps/opentelemetry/timestamp-parsing.md';
80+
81+
<TimestampParsing/>
8082

8183
**Processing Rules**. You can add **processing rules** for logs/metrics collected. To learn more, refer to [Processing Rules](../../processing-rules/index.md).
8284

docs/send-data/opentelemetry-collector/remote-management/source-templates/otrm-time-reference.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,29 @@
11
---
22
id: otrm-time-reference
3-
title: Timestamps, Time Zones, and Date Formats
3+
title: Timestamps, Time Zones, Time Ranges, and Date Formats for OpenTelemetry Remote Management
44
description: Learn how Sumo Logic manages timestamps, time zones, and dates, and the configuration options that are available with OTRM source templates.
55
keywords:
66
- time
77
- time reference
8-
- time zone
8+
- timezone
99
- time zone
1010
---
1111

1212
import Tabs from '@theme/Tabs';
1313
import TabItem from '@theme/TabItem';
1414
import useBaseUrl from '@docusaurus/useBaseUrl';
1515

16-
We support several options for timestamps, time zones, time ranges, and dates. When collecting log data, the timestamp attached to messages is vital, both for the integrity of the data in your account, and for accurate query results.
16+
We support several options for handling timestamps, time zones, and date formats in logs ingested through our OpenTelemetry Remote Management (OTRM) source templates.
17+
18+
This guide covers timestamp parsing behavior, configuration, and troubleshooting specific to OTRM. If you're using traditional Sumo Logic sources (not OTRM), refer to the general [Time Reference documentation](/docs/send-data/reference-information/time-reference/).
1719

18-
Because of the importance of timestamps, Sumo Logic indexes the timestamp of each message, making sure that data relevant to a query’s time range is returned properly in search results, which allows you to reconstruct a correct event timeline.
20+
When collecting log data, the timestamp attached to messages is critical for data integrity and accurate search results. Sumo Logic indexes the timestamp of each message to ensure results fall within the query’s time range, allowing you to reconstruct event timelines reliably.
1921

2022
## Timestamps
2123

2224
Timestamp is the part of a log message that marks the time that an event occurred. During ingestion, we can detect the message timestamp, convert it to Unix epoch time (the number of milliseconds since midnight, January 1, 1970 UTC), and index it. The timestamp is parsed either using the default timestamp parsing settings, or a custom format that you specify, including the time zone.
2325

24-
When configuring a source template, specify a custom format to parse timestamps in your log messages. 
26+
When configuring a source template, you can specify a custom format to parse timestamps in your log messages. 
2527

2628
:::note
2729
Currently, only `strptime` timestamps are supported in the source templates.
@@ -42,7 +44,7 @@ OpenTelemetry Collectors can automatically parse most timestamps without any is
4244

4345
1. Perform one of the following steps:
4446
* If you're configuring a new Source template, proceed to Step 2. Or,
45-
* To edit the timestamp settings for an existing Source template, navigate to the source template. Then click on **Edit**, to the right of the Source name and go to Step 2.<br/><img src={useBaseUrl('img/send-data/source-template-edit.png')} alt="Screenshot showing the editing interface for a source template in Sumo Logic, highlighting the section for editing advanced options including timestamp settings" style={{border: '1px solid gray'}} width="400"/>
47+
* To edit the timestamp settings for an existing Source template, navigate to the source template. Then click **Edit** to the right of the Source name and go to Step 2.<br/><img src={useBaseUrl('img/send-data/source-template-edit.png')} alt="Screenshot showing the editing interface for a source template in Sumo Logic, highlighting the section for editing advanced options including timestamp settings" style={{border: '1px solid gray'}} width="400"/>
4648
1. Navigate to the **Timestamp Parsing** section and select **Specify the format** and enter the below details:
4749
1. **Select Timezone**. Define the geographic location (time zone) to use while parsing a timestamp that does not include a time zone. The available locations depend on the local IANA Time Zone database. For example, `America/New_York`. For more examples, refer to the [List of tz database time zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
4850
1. **Format**. Specify the exact layout of the timestamp to be parsed. For example, `- %Y-%m-%dT%H:%M:%S.%LZ`. To learn more about the formatting rules, refer to [this guide](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/internal/coreinternal/timeutils/internal/ctimefmt/ctimefmt.go#L68).
@@ -142,7 +144,7 @@ The following conventions are some examples of the supported formats for `strpti
142144

143145
The following considerations apply to time zones:
144146

145-
We highly recommend that the time zone be set explicitly on any source template where the logs does not have a time zone available. Sumo Logic always attempts to determine the time zone for the Source. However, if that is not possible, the time zone will revert to UTC. In these cases, the time zone will be incorrect, and that could significantly affect forensic analysis and reporting.
147+
We highly recommend that the time zone be set explicitly on any source template where the logs do not have a time zone available. Sumo Logic always attempts to determine the time zone for the Source. However, if that is not possible, the time zone will revert to UTC. In these cases, the time zone will be incorrect, and that could significantly affect forensic analysis and reporting.
146148

147149
### Default time zone
148150

0 commit comments

Comments
 (0)