Add requester process's PID to Filesystem logs #1718
Conversation
mansi153
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
mansi153
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
There was a problem hiding this comment.
I wouldn't touch -fuser unless we are really confident its required. The goal is to minimize the divergence from the upstream crate.
There was a problem hiding this comment.
Ah, should I PR to the upstream then? I feel it is important to track and make sense of incoming requests especially during situations with multiple (potentially) conflicting operations happening on an inode at the same time.
There was a problem hiding this comment.
Do we really need it? We are already instrumenting all the methods in mountpoint-s3-fs::fuse::S3FuseFilesystem with PID?
There was a problem hiding this comment.
Ok, I have undone the changes to -fuser.
Instead, added a "new request" debug log at the point when the Fuse request reaches the -fs implementations so we (hopefully) don't have to wait for a log later in the request path to deduce the order of executions.
mountpoint-s3-fs/src/fuse.rs
Outdated
There was a problem hiding this comment.
Why add name here, we don't ever emit it?
What's the consequence of adding name when not set?
There was a problem hiding this comment.
My understanding is we emit the name in a similar format across other FS methods too, so I tried to make it consistent across the board. It would default to a blank string if not set, but I haven't looked into where it gets set. I'll check and remove the name from the log if it's not.
There was a problem hiding this comment.
Removed the name field in readdir/readidrplus since it seems a bit unnecessary to scope in fixing logging in specific methods as part of this PR - although I feel it would be nice to reach a state of consistent logging format for our FS requests.
The commit also does some minor refactoring to name unused method parameters more consistent and adhering to Rust guidelines. Signed-off-by: Mansi Pandey <[email protected]>
Signed-off-by: Mansi Pandey <[email protected]>
mansi153
requested a deployment
to
PR integration tests
GitHub Actions
Waiting
mansi153
changed the title
Signed-off-by: Mansi Pandey <[email protected]>
mansi153
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
passaro
left a comment
passaro
left a comment
There was a problem hiding this comment.
The change looks good. It does need a new version and a short entry in the changelog.
| #[instrument(level="warn", skip_all, fields(req=req.unique(), ino=parent, name=?name))] | ||
| #[instrument(level="warn", skip_all, fields(req=req.unique(), ino=parent, name=?name, pid=req.pid()))] | ||
| fn lookup(&self, req: &Request<'_>, parent: InodeNo, name: &OsStr, reply: ReplyEntry) { | ||
| debug!("New request"); |
There was a problem hiding this comment.
Have you considered using something like this instead? https://docs.rs/tracing-subscriber/latest/tracing_subscriber/fmt/struct.Layer.html#method.with_span_events
3 participants
Add requester's PID to Filesystem logs.
It also adds an entry point "New request" log to all the (supported) filesystem methods as a proxy for tracking incoming requests at FUSER.
This makes it easier to trace requests dispatched to Mountpoint, especially during workflows using multiple customer processes to make requests concurrently for the same inode(s).
Additionally, the commit adds/re-orders some other fields in the logs (for a few FS methods) to reattain a consistent order of logging request parameters.
Sample log:
The commit also does some minor refactoring to name unused method parameters more consistent and adhering to Rust guidelines.
Does this change impact existing behavior?
No, only (warn-level and higher) logging change.
No breaking changes.
Does this change need a changelog entry? Does it require a version change?
No, and no.
Logging change only.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).