[unstable] Add log level override

[devendra-lohar]

This PR adds support to override the log level defined in the application config from the cli.

[codecov]

Codecov Report

Attention: Patch coverage is 76.92308% with 3 lines in your changes missing coverage. Please review.

Project coverage is 78.96%. Comparing base (308828c) to head (50512b0).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
cognite/extractorutils/unstable/core/base.py	83.33%	2 Missing ⚠️
cognite/extractorutils/unstable/core/runtime.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #452      +/-   ##
==========================================
+ Coverage   78.02%   78.96%   +0.93%     
==========================================
  Files          42       42              
  Lines        3627     3636       +9     
==========================================
+ Hits         2830     2871      +41     
+ Misses        797      765      -32     

Files with missing lines	Coverage Δ
cognite/extractorutils/unstable/core/runtime.py	47.36% <0.00%> (-0.32%)	⬇️
cognite/extractorutils/unstable/core/base.py	77.09% <83.33%> (+18.61%)	⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Hmnt39]

Is min and max level for the log should be same in case log level override? Or is it configurable too?

[devendra-lohar]

I think when we use the --log-level override from the command line, the intended behavior should be to force all log handlers to a single, specific level unlike the config where there can be multiple log-handlers and hence multiple log-levels.

[toondaey]

Nice work. But I believe there's another _setup_logging in runtime.py that you may need to override.

[devendra-lohar]

Nice work. But I believe there's another _setup_logging in runtime.py that you may need to override.

The _setup_logging method in the Runtime class configures a very basic logger for the runtime process itself and is hardcoded to the INFO level to ensure initial startup messages are always visible.

My initial thought was to keep them separate to guarantee that critical startup messages are always visible. Let me know if I should adjust if a unified approach is preferred.

[toondaey]

Nice work. But I believe there's another _setup_logging in runtime.py that you may need to override.

The _setup_logging method in the Runtime class configures a very basic logger for the runtime process itself and is hardcoded to the INFO level to ensure initial startup messages are always visible.

My initial thought was to keep them separate to guarantee that critical startup messages are always visible. Let me know if I should adjust if a unified approach is preferred.

You're right. The ticket does also say just log level in the extractor config.

[toondaey]

I just realized that we need an entry in the changelog for this.

[toondaey]

Hopefully, no conflicts.

[rsjr]

risk review ok

[Hmnt39]

Looks good

[rsjr]

risk review ok