Console formatter improvements for developers (#6391)

[sgryphon]

Fixes #6391
Design discussion issue #none

Changes

• Adds multiple formatters to the console exporter
• Existing verbose 'Detail' formatter is retained (for learning about OpenTelemetry / development of OTel), and kept as the default
• New developer-focused 'Compact' formatter added:
  • Simple, compact format, written in a single line
  • Coloured, to easily identify log levels
  • Suitable for an application developer to use for local app dev, running in a console, as a replacement for the Microsoft Console logger provider.

Merge requirement checklist

• CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
• Unit tests added/updated
• Appropriate CHANGELOG.md files updated for non-trivial changes
• Changes in public API reviewed (if applicable)

NOTE: There is minimal changes to the public API, just adding a small number of configuration options.
Most code is internal, allowing easy future improvement.

Notes

This was originally a Contrib submission, as a completely separate exporter, but it was requested
to change to be an update to the default console logger.

This is not meant to address every console logger requirement, but has a specific focus on something
application development friendly. It is not meant for production performance, or to be flexible, or output
to files.

It is meant to be something that an application developer can use during application development, running
in a local console. The output format is compact (2 lines), uses colour, and much less verbose than the previous
default console logger.

Other considerations (e.g. performance) are no worse than the existing KeyValue formatter; only the formatting
has been improved (simplified).

I have kept the previous logger 'Detail' formatter as the default, for the users who want to learn
(it is a lot more verbose), but for details of the full attributes of any telemetry a developer could
also use an appropriate local tool, such as Loki (for logs) or Jaeger (for traces), etc.

'Detail' format is still useful for learning about OpenTelemetry, and for development of the OpenTelemetry library itself.

Output format bikeshedding

The format used for the output could be debated ad-infinitum. Rather than spend time on this,
I'd like to get a basic console logger in with any reasonable format.

If the senior project lead on OpenTelemetry .NET (a project should not be a democracy) has a specific format /
change to the format, I'm happy to make it to get things merged.

What I don't want is a half dozen people suggesting different minor tweaks as you could forever be debating
colours, abbreviations, order of the attributes, etc.

The longer term plans are to have a customisable format anyway, so the sooner we get this in, the sooner
someone can start on that.

The chosen format

Originally my Contrib submission was very similar to the default .NET Console Logger, so it could operate
as mostly a drop in replacement with no visible difference to the user except that it would include
Trace ID when available.

This version leans more towards OpenTelemetry standards.

• It uses OpenTelemetry standard Severity (Range) Names (instead of .NET names)
• Colours are taken from .NET console log, with extra colours for SPAN and METRIC.
• Includes the EventName (instead of ID). This is automatically set if using the recommended source-generator LoggerMessage high performance pattern, so works out quite well.
• Includes the TraceId and SpanId.
• Defaults the timestamp to on with HH:mm:ss, because a local run you usually know the date and just want to know how recently events happened.
• Actities (Spans) are output with their end timestamp (so the timestamp is in order), and duration in ms.
• Metrics are similarly output with their end time, and the period they cover in seconds, along with tags and values.

Resource details are not output by default, as usually you are running one application in the console on your local machine.

The main thing missing from .NET logs is the CategoryName, which doesn't have a direct equivalent in OpenTelemetry. Also, the output is written in a single line (rather than 2), and by default the output includes a timestamp (.NET default is empty).

[codecov]

Codecov Report

❌ Patch coverage is 86.54618% with 67 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.79%. Comparing base (518fcf1) to head (c549c15).
⚠️ Report is 18 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...Console/Formatting/Detail/DetailMetricFormatter.cs	87.40%	17 Missing ⚠️
...sole/Formatting/Detail/DetailLogRecordFormatter.cs	80.88%	13 Missing ⚠️
...nsole/Formatting/Detail/DetailActivityFormatter.cs	85.29%	10 Missing ⚠️
...ole/Formatting/Compact/CompactActivityFormatter.cs	80.64%	6 Missing ⚠️
...rc/OpenTelemetry.Exporter.Console/SystemConsole.cs	25.00%	6 Missing ⚠️
...nsole/Formatting/Compact/CompactMetricFormatter.cs	90.69%	4 Missing ⚠️
...r.Console/Formatting/Detail/DetailFormatterBase.cs	76.47%	4 Missing ⚠️
...ry.Exporter.Console/Formatting/FormatterFactory.cs	71.42%	4 Missing ⚠️
...le/Formatting/Compact/CompactLogRecordFormatter.cs	95.58%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6801      +/-   ##
==========================================
+ Coverage   86.91%   88.79%   +1.88%     
==========================================
  Files         262      277      +15     
  Lines       12355    12613     +258     
==========================================
+ Hits        10738    11200     +462     
+ Misses       1617     1413     -204     

Flag	Coverage Δ
unittests-Project-Experimental	88.78% <86.54%> (+1.99%)	⬆️
unittests-Project-Stable	88.88% <86.54%> (+2.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...emetry.Exporter.Console/ConsoleActivityExporter.cs	100.00% <100.00%> (+52.94%)	⬆️
.../OpenTelemetry.Exporter.Console/ConsoleExporter.cs	80.00% <100.00%> (+13.33%)	⬆️
...lemetry.Exporter.Console/ConsoleExporterOptions.cs	100.00% <100.00%> (ø)
...penTelemetry.Exporter.Console/ConsoleExtensions.cs	100.00% <100.00%> (ø)
...metry.Exporter.Console/ConsoleLogRecordExporter.cs	90.90% <100.00%> (+90.90%)	⬆️
...elemetry.Exporter.Console/ConsoleMetricExporter.cs	100.00% <100.00%> (+100.00%)	⬆️
...Console/Formatting/Compact/CompactFormatterBase.cs	100.00% <100.00%> (ø)
...rter.Console/Formatting/CompactFormatterFactory.cs	100.00% <100.00%> (ø)
...rter.Console/Formatting/ConsoleFormatterContext.cs	100.00% <100.00%> (ø)
...rter.Console/Formatting/Detail/ConsoleTagWriter.cs	56.66% <ø> (ø)
... and 10 more

... and 8 files with indirect coverage changes

[cijothomas]

Simple vs KeyValue - it is not obvious for the reader what they refer to.
Also, not a good idea to rely on magic strings.

[sgryphon]

Happy to change the names to whatever works.

Originally I used "Simple", based on the .NET ILogger/ConsoleLoggerProvider, because it was heavily based on that format, but that is no longer the case.

Maybe "Compact" for the new format?

Not sure what to call the older format as a description, maybe "Detailed". It's almost a dump of everything in the entity, although it's not, and it almost follows a standard format (INI or YAML like).

Is that more descriptive? Compact vs Detailed?

It is one of the largest differences between the two formats, and the reason for the change (a console logger that puts out 10+ lines per message is not useful for application development -- but it is useful for understanding OTel -- versus 1-2 lines per message is a reasonable logger).

[sgryphon]

I have changed the names to 'Compact' and 'Detail', which better describe the differences.

(this also helps because the term 'simple' is used elsewhere, e.g. as a simple (vs batch) processor, so changing the name to 'Compact' is good to make them separate).

The main code is changed and branch pushed. I'm still working on updating tests & documentation, so move the PR back to Draft (but peeps can still review the actual code changes).

[cijothomas]

console logger that puts out 10+ lines per message is not useful for application development -- but it is useful for understanding OTel

The intent of console logger in OTel is just for understanding/learning OTel purposes (and sometimes to debug Otel itself.) Are you planning to use this for anything beyond that?

[sgryphon]

Changes are complete, and PR ready for further full review.

I also fixed some of the build issues (in the test and example projects), so it should have less errors now.

[sgryphon]

The intent of console logger in OTel is just for understanding/learning OTel purposes (and sometimes to debug Otel itself.) Are you planning to use this for anything beyond that?

Yes.

Briefly, the existing console logger is useful for understanding OTel, but not usable for application development. This means to currently use OTel you need to have two configurations -- use OTel for production (and maybe a local tool like Loki), plus the Microsoft ConsoleLoggerProvider to output usable logs to your console.

I'd like to be able to use OTel-only for app development, and that means providing a compact exporter, similar in detail to ConsoleLoggerProvider (or any other logging framework like Serilog, NLog, or log4net).

Originally this was submitted as a Contrib pull request, so that there would be separate console loggers for development of OTel (verbose), and usable for application development (compact).

That was rejected with the instruction that the project didn't want two completely separate console loggers, and that it should be integrated with the main project. The original PR is here : open-telemetry/opentelemetry-dotnet-contrib#2887

I documented the feature details here, along with an implementation plan, if you would like more details, or see future ideas : #6391

[sgryphon]

In short, my goal is to get new developers using OpenTelemetry from day 1, without barriers. Hello World web app -> Add Open Telemetry -> Go.

[sgryphon]

I have updated with additional tests to improve code coverage, and for some build/test issues, if I could please get the workflow run again. Thanks.

[rajkumar-rangaraj]

While I appreciate the usability improvements here, I believe this constitutes a breaking change that should target the next major version rather than a minor/patch release.

This PR changes the default output format significantly:

• Multi-line structured output → single-line compact output
• Field labels (LogRecord.Timestamp:, LogRecord.Severity:) removed
• Severity text changed (Information → INFO)
• ANSI color codes now embedded in output
• Attributes, scopes, and resource information no longer emitted by default

Why this is breaking:

Although the public API remains source-compatible, the observable behavior changes in ways that break existing consumers:

1. Log scraping pipelines - Organizations using custom parsers to ingest console exporter output will experience silent failures
2. CI/CD integrations - Build systems parsing console output for diagnostics will break
3. Container logging - Kubernetes/Docker logging drivers capturing stdout expect consistent formats
4. Regex-based filters - Any pattern matching on the previous format stops working

Per SemVer, breaking changes in observable behavior, not just API signatures require a major version bump. Users upgrading from 1.14 to 1.15 don't expect their logging infrastructure to break.

Suggested path forward:

1. Option A: Hold for 2.0 where breaking changes are expected
2. Option B: Ship with Formatter = "Detail" as default (preserving current behavior), let users opt-in to "Compact"

I'd advocate for Option B - it delivers the new functionality without breaking existing consumers, and we can flip the default in 2.0 with proper migration documentation.

Thoughts?

[cijothomas]

Although the public API remains source-compatible, the observable behavior changes in ways that break existing consumers

The spec has made it abundantly clear that console exporter output format is not guaranteed to retain any structure, and hence there is no need to worry about breaking change. However, I am more worried about misuse of this exporter - it is explicitly created for learning purposes, not for any application monitoring at all.

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/sdk_exporters/stdout.md

[sgryphon]

Thanks for the feedback @rajkumar-rangaraj. As @cijothomas points out the output format is clearly marked as may change ... but I think your option B is a good compromise.

It means no impact on existing Otel devs, and is still a very simple introduction to OpenTelemetry for general application developers.

EDIT: Fix pushed to branch.

[sgryphon]

@cijothomas what misuse of this exporter are you worried about, and what makes it different from the current exporter? Are you worried the ability to send colour codes could be a security risk, or that I am using an internal experimental API to get the severity level?

I am genuinely interested in my goal ito solve the significant barrier to getting started with OTel, and have a usable OTel-only applications.

Right now the usual scenario is:

1. Create a new .NET app, e.g. web app, see it running in a default console.
2. Add open telemetry
3. Go yuck, that is horrible; and turn it off.
4. They go and install some alternative, like Serilog
5. Never touch Otel again.

Even when I convince people that using Otel is good, and to invest the time in setting up some infrastructure (like a local Loki service / Jaeger / Seq / etc), they still have to use a non-Otel system for logging to console.

[sgryphon]

@cijothomas at first I thought to keep it completely separate as the 'compact' format and 'detail' format have different goals.

• The existing detail format is created for learning purposes and not meant for application logging.
• The compact format is designed to remove the initial barrier for application developers to use Otel.

I thought the Contrib project would be an appropriate location, to keep them separate, so I submitted it there.

But the feedback from the project team was they thought two console loggers would be confusing and that when I had time I should contribute it to the main project with the existing logger. So I did that (after putting in some detailed plans).

The other alternative is simply a separate project. I did similar projects for the original System.Diagnostics (https://github.com/sgryphon/essential-diagnostics) and then Microsoft.Logging (https://github.com/sgryphon/essential-logging -- ElasticSearch component of which later got adopted by the official ElasticSearch.NET)

Please let me know what direction the project team would now prefer.

[cijothomas]

@cijothomas what misuse of this exporter are you worried about, and what makes it different from the current exporter?

@sgryphon Thanks for the detailed explanation and for putting so much thought into improving the developer experience with OpenTelemetry!

My main concern is about the intended scope of the console exporter. It's fundamentally designed for learning and debugging OpenTelemetry itself, rather than serving as an application logging solution. When we start adding sophisticated formatting options, it suggests people might use it beyond this scope, which could lead us down a path of maintaining what becomes essentially a full-featured logging framework. I can see future requests for more options like JSON etc.

That said, I completely understand the pain point you're addressing - the current verbose output really isn't suitable for day-to-day development work.

I have two suggestions to consider:

1. Leverage tools that are specifically designed for this purpose. For example, .NET developers can use the .NET Aspire Dashboard in standalone mode, which provides an excellent local experience for viewing telemetry data. There are other tools as well, which can run with one-liners, and offers much better viewing experience than console.No matter how much formatting we do, it's very inconvenient to make sense of metrics and span relationships in a console, so these tools are genuinely superior alternatives. (Unfortunately, OpenTelemetry doesn't have an official tool here, though there are ongoing discussions about that.)

2. For this PR: Consider a simple enum like OutputMode with values like Verbose (current behavior) and Compact without additional configuration knobs. This would address the immediate usability concern while keeping the scope limited and avoiding the slippery slope of feature creep.

Let me know what you think. Also, please wait for maintainers of the repo to give a weigh in. (I am no longer a maintainer of this repo)

[sgryphon]

Let me know what you think.

1. Yes, Aspire provides more detail, as does every other logging system (e.g. Seq), and console will always be limited. But they all require additional setup. Adding some packages and a few lines of code is easy (and applies for the whole project), vs every developer installing new software (although not too hard if you are already running a dev container).

And even then, you still need an additional system for console logs. As a developer I use console logs every day, and I'm sure it's the same for most of the team. Maybe I just see an error and immediately check Seq, but I wouldn't turn them off.

Best case you get Microsoft ConsoleLoggerProvider + OpenTelemetry. Otherwise they switch to another system (e.g. Serilog) and OTel never gets a look in.

2. If the project team never wants to expand the console logger, then perhaps a separate project is the better option. There are a number of existing issues requesting various improvements and the plan I submitted included looking at those, after meeting my basic goal, and see how they would fit in to improve the console logger.

Do you know who the maintainers are? (because when I hover over your name it says code owner of 46 files in this PR; although that is a bit weird as there are only 45 files)

[github-actions]

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or pushing will instruct the bot to automatically remove the label. This bot runs once per day.

[sgryphon]

Bumping for stale. Tests need to be re-run.

The test failure appears unrelated to the pull request.

It is a time out just after running Passed OpenTelemetry.Internal.Tests.WildcardHelperTests.Verify_ContainsWildcard(pattern: "a", expected: False) [< 1 ms]

The Exporter.Console tests (this pull request), started on line 333, completed on line 734 (in 9 seconds), well before this.

Note: If there is no desire for this / no feedback (i.e. it goes stale again), then I am happy with the alternative of publishing it as a separate project.