-
Notifications
You must be signed in to change notification settings - Fork 58
New logging output #4424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
New logging output #4424
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
3c7b861
Introduce a task-level log context
sandhose bfc92f5
Create a few basic logging contexts
sandhose 9075ed7
Provide log context stats in a separate structure
sandhose 4820b75
Roll our own event formatter
sandhose 2b4eaf5
Add context to some log messages
sandhose 78ccdf7
Log on every HTTP response
sandhose 62c598f
Macro to record an HTTP response error with the Sentry event ID attached
sandhose 6fbb072
Exclude the HTTP server response events from Sentry
sandhose 2959ce6
handlers::admin: don't rely on #[instrument(err)] for logging errors
sandhose 34bd5b3
handlers::compat: don't rely on #[instrument(err)] for logging errors
sandhose b14e4cb
handlers::oauth2::token: Way better error logging on the token endpoint
sandhose c32557b
Better logging of client cretentials verification errors
sandhose fb0bd8f
Better errors for the introspection endpoint
sandhose 10a7113
Make the FancyError type log the error when being transformed into a …
sandhose af9dd97
Fix Sentry creating transactions for every request
sandhose 0f24789
handlers::oauth2: don't rely on #[instrument(err)] for error logging
sandhose a8ed39d
handlers::upstream_oauth2: don't rely on #[instrument(err)] to captur…
sandhose 2b46f98
handlers::views: don't rely on #[instrument(err)] to capture errors
sandhose d2806ed
Create log contexts for accepted connections & log errors in them
sandhose eb03522
Make the error wrapper log errors
sandhose d748f18
Replace most remaining #[instrument(err)] annotations
sandhose 9aa9485
Record the job result from within the job LogContext
sandhose 76adf18
tasks: don't rely on #[instrument(err)] for logging errors
sandhose 0c74ecd
Suggestions from code review:
sandhose 5fc74f7
Merge remote-tracking branch 'origin/main' into quenting/better-logging
sandhose 295436a
Format code
sandhose b9ae522
Merge branch 'main' into quenting/better-logging
sandhose File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
[package] | ||
name = "mas-context" | ||
version.workspace = true | ||
authors.workspace = true | ||
edition.workspace = true | ||
license.workspace = true | ||
homepage.workspace = true | ||
repository.workspace = true | ||
publish = false | ||
|
||
[lints] | ||
workspace = true | ||
|
||
[dependencies] | ||
pin-project-lite.workspace = true | ||
quanta.workspace = true | ||
tokio.workspace = true | ||
tower-service.workspace = true | ||
tower-layer.workspace = true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
// Copyright 2025 New Vector Ltd. | ||
// | ||
// SPDX-License-Identifier: AGPL-3.0-only | ||
// Please see LICENSE in the repository root for full details. | ||
|
||
use std::{ | ||
pin::Pin, | ||
sync::atomic::Ordering, | ||
task::{Context, Poll}, | ||
}; | ||
|
||
use quanta::Instant; | ||
use tokio::task::futures::TaskLocalFuture; | ||
|
||
use crate::LogContext; | ||
|
||
pub type LogContextFuture<F> = TaskLocalFuture<crate::LogContext, PollRecordingFuture<F>>; | ||
|
||
impl LogContext { | ||
/// Wrap a future with the given log context | ||
pub(crate) fn wrap_future<F: Future>(&self, future: F) -> LogContextFuture<F> { | ||
let future = PollRecordingFuture::new(future); | ||
crate::CURRENT_LOG_CONTEXT.scope(self.clone(), future) | ||
} | ||
} | ||
|
||
pin_project_lite::pin_project! { | ||
/// A future which records the elapsed time and the number of polls in the | ||
/// active log context | ||
pub struct PollRecordingFuture<F> { | ||
#[pin] | ||
inner: F, | ||
} | ||
} | ||
|
||
impl<F: Future> PollRecordingFuture<F> { | ||
pub(crate) fn new(inner: F) -> Self { | ||
Self { inner } | ||
} | ||
} | ||
|
||
impl<F: Future> Future for PollRecordingFuture<F> { | ||
type Output = F::Output; | ||
|
||
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> { | ||
let start = Instant::now(); | ||
let this = self.project(); | ||
let result = this.inner.poll(cx); | ||
|
||
// Record the number of polls and the time we spent polling the future | ||
let elapsed = start.elapsed().as_nanos().try_into().unwrap_or(u64::MAX); | ||
let _ = crate::CURRENT_LOG_CONTEXT.try_with(|c| { | ||
c.inner.polls.fetch_add(1, Ordering::Relaxed); | ||
c.inner.cpu_time.fetch_add(elapsed, Ordering::Relaxed); | ||
}); | ||
|
||
result | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
// Copyright 2025 New Vector Ltd. | ||
// | ||
// SPDX-License-Identifier: AGPL-3.0-only | ||
// Please see LICENSE in the repository root for full details. | ||
|
||
use std::borrow::Cow; | ||
|
||
use tower_layer::Layer; | ||
use tower_service::Service; | ||
|
||
use crate::LogContextService; | ||
|
||
/// A layer which creates a log context for each request. | ||
pub struct LogContextLayer<R> { | ||
tagger: fn(&R) -> Cow<'static, str>, | ||
} | ||
|
||
impl<R> Clone for LogContextLayer<R> { | ||
fn clone(&self) -> Self { | ||
Self { | ||
tagger: self.tagger, | ||
} | ||
} | ||
} | ||
|
||
impl<R> LogContextLayer<R> { | ||
pub fn new(tagger: fn(&R) -> Cow<'static, str>) -> Self { | ||
Self { tagger } | ||
} | ||
} | ||
|
||
impl<S, R> Layer<S> for LogContextLayer<R> | ||
where | ||
S: Service<R>, | ||
{ | ||
type Service = LogContextService<S, R>; | ||
|
||
fn layer(&self, inner: S) -> Self::Service { | ||
LogContextService::new(inner, self.tagger) | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
// Copyright 2025 New Vector Ltd. | ||
// | ||
// SPDX-License-Identifier: AGPL-3.0-only | ||
// Please see LICENSE in the repository root for full details. | ||
|
||
mod future; | ||
mod layer; | ||
mod service; | ||
|
||
use std::{ | ||
borrow::Cow, | ||
sync::{ | ||
Arc, | ||
atomic::{AtomicU64, Ordering}, | ||
}, | ||
}; | ||
|
||
use quanta::Instant; | ||
use tokio::task_local; | ||
|
||
pub use self::{ | ||
future::{LogContextFuture, PollRecordingFuture}, | ||
layer::LogContextLayer, | ||
service::LogContextService, | ||
}; | ||
|
||
/// A counter which increments each time we create a new log context | ||
/// It will wrap around if we create more than [`u64::MAX`] contexts | ||
static LOG_CONTEXT_INDEX: AtomicU64 = AtomicU64::new(0); | ||
task_local! { | ||
pub static CURRENT_LOG_CONTEXT: LogContext; | ||
} | ||
|
||
/// A log context saves informations about the current task, such as the | ||
/// elapsed time, the number of polls, and the poll time. | ||
#[derive(Clone)] | ||
pub struct LogContext { | ||
inner: Arc<LogContextInner>, | ||
} | ||
|
||
struct LogContextInner { | ||
/// A user-defined tag for the log context | ||
tag: Cow<'static, str>, | ||
|
||
/// A unique index for the log context | ||
index: u64, | ||
|
||
/// The time when the context was created | ||
start: Instant, | ||
|
||
/// The number of [`Future::poll`] recorded | ||
polls: AtomicU64, | ||
|
||
/// An approximation of the total CPU time spent in the context | ||
cpu_time: AtomicU64, | ||
} | ||
|
||
impl LogContext { | ||
/// Create a new log context with the given tag | ||
pub fn new(tag: impl Into<Cow<'static, str>>) -> Self { | ||
let tag = tag.into(); | ||
let inner = LogContextInner { | ||
tag, | ||
index: LOG_CONTEXT_INDEX.fetch_add(1, Ordering::Relaxed), | ||
start: Instant::now(), | ||
polls: AtomicU64::new(0), | ||
cpu_time: AtomicU64::new(0), | ||
}; | ||
|
||
Self { | ||
inner: Arc::new(inner), | ||
} | ||
} | ||
|
||
/// Get a copy of the current log context, if any | ||
pub fn current() -> Option<Self> { | ||
CURRENT_LOG_CONTEXT.try_with(Self::clone).ok() | ||
} | ||
|
||
/// Run the async function `f` with the given log context. It will wrap the | ||
/// output future to record poll and CPU statistics. | ||
pub fn run<F: FnOnce() -> Fut, Fut: Future>(&self, f: F) -> LogContextFuture<Fut> { | ||
let future = self.run_sync(f); | ||
self.wrap_future(future) | ||
} | ||
|
||
/// Run the sync function `f` with the given log context, recording the CPU | ||
/// time spent. | ||
pub fn run_sync<F: FnOnce() -> R, R>(&self, f: F) -> R { | ||
let start = Instant::now(); | ||
let result = CURRENT_LOG_CONTEXT.sync_scope(self.clone(), f); | ||
let elapsed = start.elapsed().as_nanos().try_into().unwrap_or(u64::MAX); | ||
self.inner.cpu_time.fetch_add(elapsed, Ordering::Relaxed); | ||
result | ||
} | ||
} | ||
|
||
impl std::fmt::Display for LogContext { | ||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | ||
#[expect(clippy::cast_precision_loss)] | ||
let elapsed = self.inner.start.elapsed().as_nanos() as f64 / 1_000_000.; | ||
|
||
#[expect(clippy::cast_precision_loss)] | ||
let cpu_time_ms = self.inner.cpu_time.load(Ordering::Relaxed) as f64 / 1_000_000.; | ||
|
||
let polls = self.inner.polls.load(Ordering::Relaxed); | ||
let tag = &self.inner.tag; | ||
let index = self.inner.index; | ||
write!( | ||
f, | ||
"{tag}-{index} ({polls} polls, CPU: {cpu_time_ms:.3} ms, total: {elapsed:.3} ms)" | ||
sandhose marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
) | ||
} | ||
} | ||
|
||
/// A helper which implements `Display` for printing the current log context | ||
#[derive(Debug, Clone, Copy)] | ||
pub struct CurrentLogContext; | ||
|
||
impl std::fmt::Display for CurrentLogContext { | ||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | ||
CURRENT_LOG_CONTEXT | ||
.try_with(|c| c.fmt(f)) | ||
.unwrap_or_else(|_| "<no context>".fmt(f)) | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
// Copyright 2025 New Vector Ltd. | ||
// | ||
// SPDX-License-Identifier: AGPL-3.0-only | ||
// Please see LICENSE in the repository root for full details. | ||
|
||
use std::{ | ||
borrow::Cow, | ||
task::{Context, Poll}, | ||
}; | ||
|
||
use tower_service::Service; | ||
|
||
use crate::{LogContext, LogContextFuture}; | ||
|
||
/// A service which wraps another service and creates a log context for | ||
/// each request. | ||
pub struct LogContextService<S, R> { | ||
inner: S, | ||
tagger: fn(&R) -> Cow<'static, str>, | ||
} | ||
|
||
impl<S: Clone, R> Clone for LogContextService<S, R> { | ||
fn clone(&self) -> Self { | ||
Self { | ||
inner: self.inner.clone(), | ||
tagger: self.tagger, | ||
} | ||
} | ||
} | ||
|
||
impl<S, R> LogContextService<S, R> { | ||
pub fn new(inner: S, tagger: fn(&R) -> Cow<'static, str>) -> Self { | ||
Self { inner, tagger } | ||
} | ||
} | ||
|
||
impl<S, R> Service<R> for LogContextService<S, R> | ||
where | ||
S: Service<R>, | ||
{ | ||
type Response = S::Response; | ||
type Error = S::Error; | ||
type Future = LogContextFuture<S::Future>; | ||
|
||
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> { | ||
self.inner.poll_ready(cx) | ||
} | ||
|
||
fn call(&mut self, req: R) -> Self::Future { | ||
let tag = (self.tagger)(&req); | ||
let log_context = LogContext::new(tag); | ||
log_context.run(|| self.inner.call(req)) | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.