diff --git a/CHANGELOG.md b/CHANGELOG.md index c41dd858cb..83d7df9ab9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ * General * BUGFIX: Correct `glean.database.rkv_load_error`'s category and `glean.upload.discarded_exceeding_pings_size`'s name ([bug 2009475](https://bugzilla.mozilla.org/show_bug.cgi?id=2009475)) + * Event timestamps are now always clamped to the range of a signed 64-bit integer. + An overflow is recorded in the new metric `glean.error.event_timestamp_clamped` in case this happens ([#3308](https://github.com/mozilla/glean/pull/3308)). # v66.3.0 (2025-12-19) diff --git a/docs/user/user/collected-metrics/metrics.md b/docs/user/user/collected-metrics/metrics.md index 9880dce71f..d139e238fd 100644 --- a/docs/user/user/collected-metrics/metrics.md +++ b/docs/user/user/collected-metrics/metrics.md @@ -4,7 +4,7 @@ > If you are looking for the metrics collected by Glean.js, > refer to the documentation over on the [`@mozilla/glean.js`](https://github.com/mozilla/glean.js/blob/main/docs/reference/metrics.md) repository. - + # Metrics @@ -17,7 +17,7 @@ This means you might have to go searching through the dependency tree to get a f - [all-pings](#all-pings) - [baseline](#baseline) - [deletion-request](#deletion-request) -- [glean_internal_info](#glean_internal_info) +- [glean_client_info](#glean_client_info) - [health](#health) - [metrics](#metrics) @@ -121,7 +121,7 @@ All Glean pings contain built-in metrics in the [`ping_info`](https://mozilla.gi This ping contains no metrics. -## glean_internal_info +## glean_client_info All Glean pings contain built-in metrics in the [`ping_info`](https://mozilla.github.io/glean/book/user/pings/index.html#the-ping_info-section) and [`client_info`](https://mozilla.github.io/glean/book/user/pings/index.html#the-client_info-section) sections. @@ -143,6 +143,9 @@ metric information. The `health` ping is automatically sent when the application calls Glean initialize before any operations are done on the data path with a reason of `pre_init`. +Previously a second ping with reason `post_init` was sent. +This reason has been removed in Glean v66.2.0. + This ping includes the [client id](https://mozilla.github.io/glean/book/user/pings/index.html#the-client_info-section). @@ -170,6 +173,7 @@ In addition to those built-in metrics, the following metrics are added to the pi | glean.database.rkv_load_error |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |If there was an error loading the RKV database, record it. |[Bug 1815253](https://bugzilla.mozilla.org/show_bug.cgi?id=1815253)||never |1 | | glean.database.size |[memory_distribution](https://mozilla.github.io/glean/book/user/metrics/memory_distribution.html) |The size of the database file at startup. |[Bug 1656589](https://bugzilla.mozilla.org/show_bug.cgi?id=1656589#c7)||never |1 | | glean.database.write_time |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The time it takes for a write-commit for the Glean database. |[Bug 1896193](https://bugzilla.mozilla.org/show_bug.cgi?id=1896193#c4)||never |1 | +| glean.error.event_timestamp_clamped |[counter](https://mozilla.github.io/glean/book/user/metrics/counter.html) |The number of times we had to clamp an event timestamp for exceeding the range of a signed 64-bit integer (9223372036854775807). |[Bug 1873482](https://bugzilla.mozilla.org/show_bug.cgi?id=1873482)||2026-06-30 |1 | | glean.error.io |[counter](https://mozilla.github.io/glean/book/user/metrics/counter.html) |The number of times we encountered an IO error when writing a pending ping to disk. |[Bug 1686233](https://bugzilla.mozilla.org/show_bug.cgi?id=1686233#c2)||never |1 | | glean.error.preinit_tasks_overflow |[counter](https://mozilla.github.io/glean/book/user/metrics/counter.html) |The number of tasks that overflowed the pre-initialization buffer. Only sent if the buffer ever overflows. In Version 0 this reported the total number of tasks enqueued. |[Bug 1609482](https://bugzilla.mozilla.org/show_bug.cgi?id=1609482#c3)||never |1 | | glean.health.data_directory_info |[object](https://mozilla.github.io/glean/book/user/metrics/object.html) |Information about the data directories and files used by FOG. Structure is an array of objects, each containing the following properties: - `dir_name`: The name of the directory. This is the subdirectory name relative to the FOG data directory and should only include "db", "events", and "pending_pings". - `dir_exists`: Whether the directory exists. This should only be false on the first run of FOG, or if the directory was deleted. - `dir_created`: The creation time of the directory, in seconds since the unix epoch. If the directory does not exist, this will be `null` and if the time cannot be determined, it will default to `0`. - `dir_modified`: The last modification time of the directory, in seconds since the unix epoch. If the directory does not exist, this will be `null` and if the time cannot be determined, it will default to `0`. - `file_count`: The number of files in the directory. If the directory does not exist, this will be `0`. - `error_message`: If there was an error accessing the directory, this will contain a brief description of the error. If there was no error, this will be null. - `files`: An array of objects, each containing: - `file_name`: The name of the file. Could be `data.safe.bin`, `events.safe.bin`, or A UUID representing the doc-id of a pending ping. - `file_created`: The creation time of the file, in seconds since the epoch. If the file does not exist, this will be `null` and if the time cannot be determined, it will default to `0`. - `file_modified`: The last modification time of the file, in seconds since the epoch. If the file does not exist, this will be `null` and if the time cannot be determined, it will default to `0`. - `file_size`: The size of the file in bytes. This can be just about any size but a `0` value indicates the file is empty. - `error_message`: If there was an error accessing the file, this will contain a brief description of the error. If there was no error, this will be null. |[Bug 1982711](https://bugzilla.mozilla.org/show_bug.cgi?id=1982711#c3)||never |1 | @@ -255,5 +259,5 @@ In addition to those built-in metrics, the following metrics are added to the pi Data categories are [defined here](https://wiki.mozilla.org/Firefox/Data_Collection). - + diff --git a/glean-core/metrics.yaml b/glean-core/metrics.yaml index 432c47d824..69785b2e5d 100644 --- a/glean-core/metrics.yaml +++ b/glean-core/metrics.yaml @@ -649,6 +649,24 @@ glean.error: - jrediger@mozilla.com expires: never + event_timestamp_clamped: + type: counter + description: | + The number of times we had to clamp an event timestamp + for exceeding the range of a signed 64-bit integer (9223372036854775807). + send_in_pings: + - health + bugs: + - https://bugzilla.mozilla.org/show_bug.cgi?id=1873482 + data_reviews: + - https://bugzilla.mozilla.org/show_bug.cgi?id=1873482 + data_sensitivity: + - technical + notification_emails: + - glean-team@mozilla.com + - jrediger@mozilla.com + expires: 2026-06-30 + glean.upload: ping_upload_failure: type: labeled_counter diff --git a/glean-core/src/event_database/mod.rs b/glean-core/src/event_database/mod.rs index 108bc3d465..413855e3cf 100644 --- a/glean-core/src/event_database/mod.rs +++ b/glean-core/src/event_database/mod.rs @@ -560,6 +560,22 @@ impl EventDatabase { // Let's fix cur_ec up and hope this isn't a sign something big is broken. cur_ec = execution_counter; } + + // event timestamp is a `u64`, but BigQuery uses `i64` (signed!) everywhere. Let's clamp the value to make + // sure we stay within bounds. + if event.event.timestamp > i64::MAX as u64 { + glean + .additional_metrics + .event_timestamp_clamped + .add_sync(glean, 1); + log::warn!( + "Calculated event timestamp was too high. Got: {}, max: {}", + event.event.timestamp, + i64::MAX, + ); + event.event.timestamp = event.event.timestamp.clamp(0, i64::MAX as u64); + } + if highest_ts > event.event.timestamp { // Even though we sorted everything, something in the // execution_counter or glean.startup.date math went awry. @@ -1357,4 +1373,50 @@ mod test { ) .is_err()); } + + #[test] + fn normalize_store_clamps_timestamp() { + let (glean, _dir) = new_glean(None); + + let store_name = "store-name"; + let event = RecordedEvent { + category: "category".into(), + name: "name".into(), + ..Default::default() + }; + + let timestamps = [ + 0, + (i64::MAX / 2) as u64, + i64::MAX as _, + (i64::MAX as u64) + 1, + ]; + let mut store = timestamps + .into_iter() + .map(|timestamp| StoredEvent { + event: RecordedEvent { + timestamp, + ..event.clone() + }, + execution_counter: None, + }) + .collect(); + + let glean_start_time = glean.start_time(); + glean + .event_storage() + .normalize_store(&glean, store_name, &mut store, glean_start_time); + assert_eq!(4, store.len()); + + assert_eq!(0, store[0].event.timestamp); + assert_eq!((i64::MAX / 2) as u64, store[1].event.timestamp); + assert_eq!((i64::MAX as u64), store[2].event.timestamp); + assert_eq!((i64::MAX as u64), store[3].event.timestamp); + + let error_count = glean + .additional_metrics + .event_timestamp_clamped + .get_value(&glean, "health"); + assert_eq!(Some(1), error_count); + } } diff --git a/glean-core/src/internal_metrics.rs b/glean-core/src/internal_metrics.rs index 005cdd16fc..a0a3ddb791 100644 --- a/glean-core/src/internal_metrics.rs +++ b/glean-core/src/internal_metrics.rs @@ -40,6 +40,10 @@ pub struct AdditionalMetrics { /// An experimentation identifier derived and provided by the application /// for the purpose of experimentation enrollment. pub experimentation_id: StringMetric, + + /// The number of times we had to clamp an event timestamp + /// for exceeding the range of a signed 64-bit integer (9223372036854775807). + pub event_timestamp_clamped: CounterMetric, } impl CoreMetrics { @@ -198,6 +202,15 @@ impl AdditionalMetrics { disabled: false, dynamic_label: None, }), + + event_timestamp_clamped: CounterMetric::new(CommonMetricData { + name: "event_timestamp_clamped".into(), + category: "glean.error".into(), + send_in_pings: vec!["health".into()], + lifetime: Lifetime::Ping, + disabled: false, + dynamic_label: None, + }), } } }