Skip to content

Conversation

teskje
Copy link
Contributor

@teskje teskje commented Oct 3, 2025

This PR downgrades the "Applying configuration update" logs from info to debug. They have become extremely noisy over time as we have accumulated more config options, and especially with the introduction of the dyncfgs.

Motivation

  • This PR fixes a recognized bug.

Fixes https://github.com/MaterializeInc/database-issues/issues/9660

Tips for reviewer

I would prefer to either (a) log only actual config changes or (b) send only actual config changes from envd to clusterd. But I didn't find a good way to implement (a) with our current {Compute,Storage}Parameters definition and (b) requires a refactor of adapter code I'm not very familiar with.

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

This commit downgrades the "Applying configuration update" logs from
`info` to `debug`. They have become extremely noisy over time as we have
accumulated more config options, and especially with the introduction of
the dyncfgs.
@teskje teskje marked this pull request as ready for review October 6, 2025 07:48
@teskje teskje requested a review from a team as a code owner October 6, 2025 07:48
Copy link
Contributor

@petrosagg petrosagg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! It was about time

Copy link
Member

@antiguru antiguru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the sentiment, but just last week this was crucial to me to debug an issue. I think we should do something else here: only print the configuration that has changed. (This was to understand 0dt behavior, so we couldn't even apply a new debug level without restarting envd.)

@petrosagg
Copy link
Contributor

but just last week this was crucial to me to debug an issue

But how often have you had such a debugging task? And, was the problem related to the mechanism of propagating configuration to clusters or you just wanted to see what was the value of a particular config at a particular time? The latter need can be served by our configuration service (LD) which tracks the history of each variable.

In general, I think we need a higher bar for having things be logged at info level, leaning towards things that are:

  1. High value
  2. Low traffic
  3. Can't be deduced after the fact

I don't think the LD configs meet that bar so I'm strongly in favor of merging this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants