From 9b3343c344aeb78673aaea0d843b453b4968727c Mon Sep 17 00:00:00 2001 From: Amee Lepcha Date: Tue, 8 Apr 2025 16:37:14 +0530 Subject: [PATCH 1/3] Update Timestamp Autocorrection and Historical Data Ingestion Documentation --- docs/send-data/collector-faq.md | 6 ++++++ docs/send-data/reference-information/time-reference.md | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/send-data/collector-faq.md b/docs/send-data/collector-faq.md index e4b497b020..1445b8f413 100644 --- a/docs/send-data/collector-faq.md +++ b/docs/send-data/collector-faq.md @@ -423,6 +423,12 @@ See [using _format for troubleshooting](/docs/send-data/reference-information/ti Sumo Logic assumes that all log message times fall within a window of -1 year through +2 days compared to the current time. Any log messages with a parsed timestamp outside of that window is automatically re-stamped with the current time. +* To ingest historical data older than 30 days but within 365 days, you must specify a `timestamp` field using a regex locator and a valid date format. + +* Data older than 365 days can still be ingested. However, even if a custom timestamp is provided, it will be autocorrected to the current time unless technical support disables this function at the organization level. + +* To ingest data older than 365 days with the original timestamp intact, you must contact technical support to disable the autocorrection function at the org level. + #### Assumption: Data from a source will have similar timestamps Sumo Logic assumes that all log messages coming from a particular source will have timestamps that are close together. If a message comes through that appears to be more than one day earlier or later than recent messages from that source, it will be auto-corrected to match the current time. diff --git a/docs/send-data/reference-information/time-reference.md b/docs/send-data/reference-information/time-reference.md index 80f7540ccf..f77deb04db 100644 --- a/docs/send-data/reference-information/time-reference.md +++ b/docs/send-data/reference-information/time-reference.md @@ -33,7 +33,7 @@ If your log messages from a Source contain multiple timestamps, timestamps in un The Collector assumes that all log messages coming from a particular Source will have timestamps that are close together. If a message comes through that appears to be more than one day earlier or later than recent messages from that Source it will be auto-corrected to match the current time. You can stop this auto-correction by explicitly configuring a custom timestamp format on your Source. -The Collector also assumes that all log messages coming from a particular Source will have timestamps that are within a window of -1 year through +2 days compared to the current time. Any log message with a parsed timestamp outside of that window is automatically re-stamped with the current time. You must contact [Sumo Logic Support](https://support.sumologic.com/) to adjust this auto-correction behavior. See [How to ingest old or historical data](/docs/send-data/collector-faq#how-to-ingest-old-or-historical-data) for further details. +The Collector also assumes that all log messages coming from a particular Source will have timestamps that are within a window of -1 year through +2 days compared to the current time. Any log messages with a timestamp older than 30 days is automatically set to the current time. You must contact [Sumo Logic Support](https://support.sumologic.com/) to adjust this auto-correction behavior. See [How to ingest old or historical data](/docs/send-data/collector-faq#how-to-ingest-old-or-historical-data) for further details. ### Automated timestamp parsing From e667433c7e5d5e830b37ce5932ef910c196d22ce Mon Sep 17 00:00:00 2001 From: Amee Lepcha Date: Tue, 8 Apr 2025 16:51:02 +0530 Subject: [PATCH 2/3] Update collector-faq.md --- docs/send-data/collector-faq.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/send-data/collector-faq.md b/docs/send-data/collector-faq.md index 1445b8f413..fe79a826c0 100644 --- a/docs/send-data/collector-faq.md +++ b/docs/send-data/collector-faq.md @@ -419,14 +419,14 @@ This article describes the assumptions that Sumo makes about customer data, tips See [using _format for troubleshooting](/docs/send-data/reference-information/time-reference.md) timestamps. -#### Assumption: Data is less than 365 days old - -Sumo Logic assumes that all log message times fall within a window of -1 year through +2 days compared to the current time. Any log messages with a parsed timestamp outside of that window is automatically re-stamped with the current time. +#### Assumption: Data is less than 30 days but within 365 days * To ingest historical data older than 30 days but within 365 days, you must specify a `timestamp` field using a regex locator and a valid date format. -* Data older than 365 days can still be ingested. However, even if a custom timestamp is provided, it will be autocorrected to the current time unless technical support disables this function at the organization level. +#### Assumption: Data is older than 365 days +Sumo Logic assumes that all log message times fall within a window of -1 year through +2 days compared to the current time. Any log messages with a parsed timestamp outside of that window are automatically re-stamped with the current time. +* Data older than 365 days can still be ingested. However, even if a custom timestamp is provided, it will be autocorrected to the current time unless technical support disables this function at the organization level. * To ingest data older than 365 days with the original timestamp intact, you must contact technical support to disable the autocorrection function at the org level. #### Assumption: Data from a source will have similar timestamps From 58cb57d31c2acef9b22f094d267055d2eccde60f Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 24 Apr 2025 12:02:00 +0530 Subject: [PATCH 3/3] Update docs/send-data/collector-faq.md Co-authored-by: Kim (Sumo Logic) <56411016+kimsauce@users.noreply.github.com> --- docs/send-data/collector-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/send-data/collector-faq.md b/docs/send-data/collector-faq.md index fe79a826c0..f6e7b0d0ba 100644 --- a/docs/send-data/collector-faq.md +++ b/docs/send-data/collector-faq.md @@ -427,7 +427,7 @@ See [using _format for troubleshooting](/docs/send-data/reference-information/ti Sumo Logic assumes that all log message times fall within a window of -1 year through +2 days compared to the current time. Any log messages with a parsed timestamp outside of that window are automatically re-stamped with the current time. * Data older than 365 days can still be ingested. However, even if a custom timestamp is provided, it will be autocorrected to the current time unless technical support disables this function at the organization level. -* To ingest data older than 365 days with the original timestamp intact, you must contact technical support to disable the autocorrection function at the org level. +* To ingest data older than 365 days with the original timestamp intact, you'll need to contact [Support](https://support.sumologic.com/support/s) to disable the autocorrection function at the org level. #### Assumption: Data from a source will have similar timestamps