-
Notifications
You must be signed in to change notification settings - Fork 140
CBG-4928: Update log messages on changes feed being cancelled #7880
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR updates the logging approach for changes feed cancellation scenarios. Currently, these cancellations are logged as warnings with "error" in the messages, but they are not necessarily errors. The changes downgrade the log level from WarnfCtx to InfofCtx, add the base.KeyChanges log key for better filtering, and update the error messages to be more descriptive and accurate.
Key Changes
- Downgraded log level from warning to info for changes feed cancellation messages
- Added
base.KeyChangeslog key to enable targeted log filtering - Updated error messages to be more descriptive (e.g., "Error retrieving changes" → "Could not retrieve changes")
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| db/channel_cache_single.go | Simplified error message when changes feed is cancelled while waiting for view lock |
| db/changes.go | Downgraded two log statements from warning to info level and updated messages to be more descriptive |
| base/error.go | Updated ErrChannelFeed message from "Error while building channel feed" to "Failed to build channel feed" |
| changes, err := singleChannelCache.GetChanges(ctx, paginationOptions) | ||
| if err != nil { | ||
| base.WarnfCtx(ctx, "Error retrieving changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | ||
| base.InfofCtx(ctx, base.KeyChanges, "Could not retrieve changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) |
Copilot
AI
Nov 14, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message 'Could not retrieve changes' is vague and doesn't clearly indicate that the changes feed was cancelled. Consider using a more specific message like 'Changes feed cancelled for channel %q: %v' to better reflect the actual cause.
| base.InfofCtx(ctx, base.KeyChanges, "Could not retrieve changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | |
| if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { | |
| base.InfofCtx(ctx, base.KeyChanges, "Changes feed cancelled for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | |
| } else { | |
| base.InfofCtx(ctx, base.KeyChanges, "Could not retrieve changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | |
| } |
| // On feed error, send the error and exit changes processing | ||
| if current[i].Err == base.ErrChannelFeed { | ||
| base.WarnfCtx(ctx, "MultiChangesFeed got error reading changes feed: %v", current[i].Err) | ||
| base.InfofCtx(ctx, base.KeyChanges, "Could not read changes feed: %v", current[i].Err) |
Copilot
AI
Nov 14, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message 'Could not read changes feed' is ambiguous and doesn't clearly indicate a cancellation scenario. Consider using 'Changes feed cancelled: %v' or 'Changes feed operation cancelled: %v' for clarity.
| base.InfofCtx(ctx, base.KeyChanges, "Could not read changes feed: %v", current[i].Err) | |
| base.InfofCtx(ctx, base.KeyChanges, "Changes feed cancelled: %v", current[i].Err) |
| ErrUpdateCancel = &sgError{"Cancel update"} | ||
| ErrImportCancelledPurged = HTTPErrorf(http.StatusNotFound, "Import Cancelled Due to Purge") | ||
| ErrChannelFeed = &sgError{"Error while building channel feed"} | ||
| ErrChannelFeed = &sgError{"Failed to build channel feed"} |
Copilot
AI
Nov 14, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message 'Failed to build channel feed' still implies a failure rather than a cancellation, which contradicts the PR's goal of treating these as non-error scenarios. Consider renaming to 'Channel feed cancelled' or similar to better reflect the intended semantics.
| ErrChannelFeed = &sgError{"Failed to build channel feed"} | |
| ErrChannelFeed = &sgError{"Channel feed cancelled"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can drop this, because I think this error wouldn't show up if we avoid logging it at all.
| // overhead in that case (and prevent feedback loop on query backlog) | ||
| if options.ChangesCtx.Err() != nil { | ||
| return nil, fmt.Errorf("Changes feed cancelled while waiting for view lock") | ||
| return nil, fmt.Errorf("Changes feed cancelled") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| return nil, fmt.Errorf("Changes feed cancelled") | |
| return nil, fmt.Errorf("Changes feed cancelled %w", options.ChangesCtx.Err()) |
This would tell us in downstream while it was canceled, particularly if we are able to determine a cause, this will make the above easier to reason about I think.
| changes, err := singleChannelCache.GetChanges(ctx, paginationOptions) | ||
| if err != nil { | ||
| base.WarnfCtx(ctx, "Error retrieving changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | ||
| base.InfofCtx(ctx, base.KeyChanges, "Could not retrieve changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming the context cancellation is passed back in the error message:
if !errors.Is(err, context.Canceled) && errors.Is(err, context.DeadlineExceeded) {
base.WarnfCtx(ctx, "Error retrieving changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err)
}
However, I think to match the behavior of below if we don't always wrap the context error message:
| base.InfofCtx(ctx, base.KeyChanges, "Could not retrieve changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | |
| if options.ChangesCtx.Err() != nil { | |
| base.WarnfCtx(ctx, "Error retrieving changes for channel %q: %v", base.UD(singleChannelCache.ChannelID().Name), err) | |
| } |
| if current[i].Err == base.ErrChannelFeed { | ||
| base.WarnfCtx(ctx, "MultiChangesFeed got error reading changes feed: %v", current[i].Err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if current[i].Err == base.ErrChannelFeed { | |
| base.WarnfCtx(ctx, "MultiChangesFeed got error reading changes feed: %v", current[i].Err) | |
| if !options.ChangesCtx.Err() { | |
| base.WarnfCtx(ctx, "MultiChangesFeed got error reading changes feed: %v", current[i].Err) | |
| } |
In this case, it's not clear whether that err from current[i].Err is better than the error returned.
| ErrUpdateCancel = &sgError{"Cancel update"} | ||
| ErrImportCancelledPurged = HTTPErrorf(http.StatusNotFound, "Import Cancelled Due to Purge") | ||
| ErrChannelFeed = &sgError{"Error while building channel feed"} | ||
| ErrChannelFeed = &sgError{"Failed to build channel feed"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can drop this, because I think this error wouldn't show up if we avoid logging it at all.
| // Check whether the changes process has been terminated while we waited for the view lock, to avoid the view | ||
| // overhead in that case (and prevent feedback loop on query backlog) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // Check whether the changes process has been terminated while we waited for the view lock, to avoid the view | |
| // overhead in that case (and prevent feedback loop on query backlog) | |
| // Check whether the changes process has been terminated before running a query |
CBG-4928
Describe your PR here...
Pre-review checklist
fmt.Print,log.Print, ...)base.UD(docID),base.MD(dbName))docs/apiDependencies (if applicable)
Integration Tests
GSI=true,xattrs=truehttps://jenkins.sgwdev.com/job/SyncGatewayIntegration/0000/