Skip to content

Commit 8aee3e1

Browse files
philwebbphilwebb
committed
Allow structure logging JSON to be customized
Introduce a new `StructureLoggingJsonMembersCustomizer` interface as well as additional properties that can be used to customize the JSON produced with structured logging. Closes gh-42486
1 parent 27c59b8 commit 8aee3e1

File tree

31 files changed

+673
-51
lines changed

31 files changed

+673
-51
lines changed

‎spring-boot-project/spring-boot-docs/src/docs/antora/modules/reference/pages/features/logging.adoc

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -572,8 +572,50 @@ If you add https://www.slf4j.org/api/org/slf4j/Marker.html[markers], these will
572572

573573

574574

575-
[[features.logging.structured.custom-format]]
576-
=== Custom Structured Logging formats
575+
[[features.logging.structured.customizing-json]]
576+
=== Customizing Structured Logging JSON
577+
578+
Spring Boot attempts to pick sensible defaults for the JSON names and values output for structured logging.
579+
Sometimes, however, you may want to make small adjustments to the JSON for your own needs.
580+
For example, it's possible that you might want to change some of the names to match the expectations of your log ingestion system.
581+
You might also want to filter out certain members since you don't find them useful.
582+
583+
The following properties allow you to change the way that structured logging JSON is written:
584+
585+
|===
586+
| Property | Description
587+
588+
| configprop:logging.structured.json.include[] & configprop:logging.structured.json.exclude[]
589+
| Filters specific paths from the JSON
590+
591+
| configprop:logging.structured.json.rename[]
592+
| Renames a specific member in the JSON
593+
594+
| configprop:logging.structured.json.add[]
595+
| Adds additional members to the JSON
596+
|===
597+
598+
For example, the following will exclude `log.level`, rename `process.id` to `procid` and add a fixed `corpname` field:
599+
600+
[configprops,yaml]
601+
----
602+
logging:
603+
structured:
604+
json:
605+
exclude: log.level
606+
rename:
607+
process.id: procid
608+
add:
609+
corpname: mycorp
610+
----
611+
612+
TIP: For more advanced customizations, you can write your own class that implements the javadoc:org.springframework.boot.logging.structured.StructuredLogFormatter[] interface and declare it using the configprop:logging.structured.json.customizer[] property.
613+
You can also declare implementations by listing them in a `META-INF/spring.factories` file.
614+
615+
616+
617+
[[features.logging.structured.other-formats]]
618+
=== Supporting Other Structured Logging Formats
577619

578620
The structured logging support in Spring Boot is extensible, allowing you to define your own custom format.
579621
To do this, implement the `StructuredLoggingFormatter` interface. The generic type argument has to be `ILoggingEvent` when using Logback and `LogEvent` when using Log4j2 (that means your implementation is tied to a specific logging system).

‎spring-boot-project/spring-boot-docs/src/main/java/org/springframework/boot/docs/features/logging/structured/customformat/MyCustomFormat.java renamed to ‎spring-boot-project/spring-boot-docs/src/main/java/org/springframework/boot/docs/features/logging/structured/otherformats/MyCustomFormat.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
* limitations under the License.
1515
*/
1616

17-
package org.springframework.boot.docs.features.logging.structured.customformat;
17+
package org.springframework.boot.docs.features.logging.structured.otherformats;
1818

1919
import ch.qos.logback.classic.spi.ILoggingEvent;
2020

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/log4j2/ElasticCommonSchemaStructuredLogFormatter.java

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
2929
import org.springframework.boot.logging.structured.ElasticCommonSchemaProperties;
3030
import org.springframework.boot.logging.structured.JsonWriterStructuredLogFormatter;
31+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
3132
import org.springframework.boot.logging.structured.StructuredLogFormatter;
3233
import org.springframework.core.env.Environment;
3334
import org.springframework.util.ObjectUtils;
@@ -41,8 +42,9 @@
4142
*/
4243
class ElasticCommonSchemaStructuredLogFormatter extends JsonWriterStructuredLogFormatter<LogEvent> {
4344

44-
ElasticCommonSchemaStructuredLogFormatter(Environment environment) {
45-
super((members) -> jsonMembers(environment, members));
45+
ElasticCommonSchemaStructuredLogFormatter(Environment environment,
46+
StructureLoggingJsonMembersCustomizer<?> customizer) {
47+
super((members) -> jsonMembers(environment, members), customizer);
4648
}
4749

4850
private static void jsonMembers(Environment environment, JsonWriter.Members<LogEvent> members) {

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/log4j2/GraylogExtendedLogFormatStructuredLogFormatter.java

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
3838
import org.springframework.boot.logging.structured.GraylogExtendedLogFormatProperties;
3939
import org.springframework.boot.logging.structured.JsonWriterStructuredLogFormatter;
40+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
4041
import org.springframework.boot.logging.structured.StructuredLogFormatter;
4142
import org.springframework.core.env.Environment;
4243
import org.springframework.core.log.LogMessage;
@@ -68,8 +69,9 @@ class GraylogExtendedLogFormatStructuredLogFormatter extends JsonWriterStructure
6869
*/
6970
private static final Set<String> ADDITIONAL_FIELD_ILLEGAL_KEYS = Set.of("id", "_id");
7071

71-
GraylogExtendedLogFormatStructuredLogFormatter(Environment environment) {
72-
super((members) -> jsonMembers(environment, members));
72+
GraylogExtendedLogFormatStructuredLogFormatter(Environment environment,
73+
StructureLoggingJsonMembersCustomizer<?> customizer) {
74+
super((members) -> jsonMembers(environment, members), customizer);
7375
}
7476

7577
private static void jsonMembers(Environment environment, JsonWriter.Members<LogEvent> members) {

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/log4j2/LogstashStructuredLogFormatter.java

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
import org.springframework.boot.json.JsonWriter;
3333
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
3434
import org.springframework.boot.logging.structured.JsonWriterStructuredLogFormatter;
35+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
3536
import org.springframework.boot.logging.structured.StructuredLogFormatter;
3637
import org.springframework.util.CollectionUtils;
3738

@@ -43,8 +44,8 @@
4344
*/
4445
class LogstashStructuredLogFormatter extends JsonWriterStructuredLogFormatter<LogEvent> {
4546

46-
LogstashStructuredLogFormatter() {
47-
super(LogstashStructuredLogFormatter::jsonMembers);
47+
LogstashStructuredLogFormatter(StructureLoggingJsonMembersCustomizer<?> customizer) {
48+
super(LogstashStructuredLogFormatter::jsonMembers, customizer);
4849
}
4950

5051
private static void jsonMembers(JsonWriter.Members<LogEvent> members) {

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/log4j2/StructuredLogLayout.java

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,11 @@
3030
import org.apache.logging.log4j.core.layout.AbstractStringLayout;
3131

3232
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
33+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
3334
import org.springframework.boot.logging.structured.StructuredLogFormatter;
3435
import org.springframework.boot.logging.structured.StructuredLogFormatterFactory;
3536
import org.springframework.boot.logging.structured.StructuredLogFormatterFactory.CommonFormatters;
37+
import org.springframework.boot.util.Instantiator;
3638
import org.springframework.core.env.Environment;
3739
import org.springframework.util.Assert;
3840

@@ -102,14 +104,29 @@ public StructuredLogLayout build() {
102104
}
103105

104106
private void addCommonFormatters(CommonFormatters<LogEvent> commonFormatters) {
105-
commonFormatters.add(CommonStructuredLogFormat.ELASTIC_COMMON_SCHEMA,
106-
(instantiator) -> new ElasticCommonSchemaStructuredLogFormatter(
107-
instantiator.getArg(Environment.class)));
108-
commonFormatters.add(CommonStructuredLogFormat.GRAYLOG_EXTENDED_LOG_FORMAT,
109-
(instantiator) -> new GraylogExtendedLogFormatStructuredLogFormatter(
110-
instantiator.getArg(Environment.class)));
111-
commonFormatters.add(CommonStructuredLogFormat.LOGSTASH,
112-
(instantiator) -> new LogstashStructuredLogFormatter());
107+
commonFormatters.add(CommonStructuredLogFormat.ELASTIC_COMMON_SCHEMA, this::createEcsFormatter);
108+
commonFormatters.add(CommonStructuredLogFormat.GRAYLOG_EXTENDED_LOG_FORMAT, this::createGraylogFormatter);
109+
commonFormatters.add(CommonStructuredLogFormat.LOGSTASH, this::createLogstashFormatter);
110+
}
111+
112+
private ElasticCommonSchemaStructuredLogFormatter createEcsFormatter(Instantiator<?> instantiator) {
113+
Environment environment = instantiator.getArg(Environment.class);
114+
StructureLoggingJsonMembersCustomizer<?> jsonMembersCustomizer = instantiator
115+
.getArg(StructureLoggingJsonMembersCustomizer.class);
116+
return new ElasticCommonSchemaStructuredLogFormatter(environment, jsonMembersCustomizer);
117+
}
118+
119+
private GraylogExtendedLogFormatStructuredLogFormatter createGraylogFormatter(Instantiator<?> instantiator) {
120+
Environment environment = instantiator.getArg(Environment.class);
121+
StructureLoggingJsonMembersCustomizer<?> jsonMembersCustomizer = instantiator
122+
.getArg(StructureLoggingJsonMembersCustomizer.class);
123+
return new GraylogExtendedLogFormatStructuredLogFormatter(environment, jsonMembersCustomizer);
124+
}
125+
126+
private LogstashStructuredLogFormatter createLogstashFormatter(Instantiator<?> instantiator) {
127+
StructureLoggingJsonMembersCustomizer<?> jsonMembersCustomizer = instantiator
128+
.getArg(StructureLoggingJsonMembersCustomizer.class);
129+
return new LogstashStructuredLogFormatter(jsonMembersCustomizer);
113130
}
114131

115132
}

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/logback/ElasticCommonSchemaStructuredLogFormatter.java

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
2929
import org.springframework.boot.logging.structured.ElasticCommonSchemaProperties;
3030
import org.springframework.boot.logging.structured.JsonWriterStructuredLogFormatter;
31+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
3132
import org.springframework.boot.logging.structured.StructuredLogFormatter;
3233
import org.springframework.core.env.Environment;
3334

@@ -43,9 +44,9 @@ class ElasticCommonSchemaStructuredLogFormatter extends JsonWriterStructuredLogF
4344
private static final PairExtractor<KeyValuePair> keyValuePairExtractor = PairExtractor.of((pair) -> pair.key,
4445
(pair) -> pair.value);
4546

46-
ElasticCommonSchemaStructuredLogFormatter(Environment environment,
47-
ThrowableProxyConverter throwableProxyConverter) {
48-
super((members) -> jsonMembers(environment, throwableProxyConverter, members));
47+
ElasticCommonSchemaStructuredLogFormatter(Environment environment, ThrowableProxyConverter throwableProxyConverter,
48+
StructureLoggingJsonMembersCustomizer<?> customizer) {
49+
super((members) -> jsonMembers(environment, throwableProxyConverter, members), customizer);
4950
}
5051

5152
private static void jsonMembers(Environment environment, ThrowableProxyConverter throwableProxyConverter,

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/logback/GraylogExtendedLogFormatStructuredLogFormatter.java

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
3838
import org.springframework.boot.logging.structured.GraylogExtendedLogFormatProperties;
3939
import org.springframework.boot.logging.structured.JsonWriterStructuredLogFormatter;
40+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
4041
import org.springframework.boot.logging.structured.StructuredLogFormatter;
4142
import org.springframework.core.env.Environment;
4243
import org.springframework.core.log.LogMessage;
@@ -69,8 +70,8 @@ class GraylogExtendedLogFormatStructuredLogFormatter extends JsonWriterStructure
6970
private static final Set<String> ADDITIONAL_FIELD_ILLEGAL_KEYS = Set.of("id", "_id");
7071

7172
GraylogExtendedLogFormatStructuredLogFormatter(Environment environment,
72-
ThrowableProxyConverter throwableProxyConverter) {
73-
super((members) -> jsonMembers(environment, throwableProxyConverter, members));
73+
ThrowableProxyConverter throwableProxyConverter, StructureLoggingJsonMembersCustomizer<?> customizer) {
74+
super((members) -> jsonMembers(environment, throwableProxyConverter, members), customizer);
7475
}
7576

7677
private static void jsonMembers(Environment environment, ThrowableProxyConverter throwableProxyConverter,

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/logback/LogstashStructuredLogFormatter.java

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
import org.springframework.boot.json.JsonWriter.PairExtractor;
3636
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
3737
import org.springframework.boot.logging.structured.JsonWriterStructuredLogFormatter;
38+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
3839
import org.springframework.boot.logging.structured.StructuredLogFormatter;
3940

4041
/**
@@ -48,8 +49,9 @@ class LogstashStructuredLogFormatter extends JsonWriterStructuredLogFormatter<IL
4849
private static final PairExtractor<KeyValuePair> keyValuePairExtractor = PairExtractor.of((pair) -> pair.key,
4950
(pair) -> pair.value);
5051

51-
LogstashStructuredLogFormatter(ThrowableProxyConverter throwableProxyConverter) {
52-
super((members) -> jsonMembers(throwableProxyConverter, members));
52+
LogstashStructuredLogFormatter(ThrowableProxyConverter throwableProxyConverter,
53+
StructureLoggingJsonMembersCustomizer<?> customizer) {
54+
super((members) -> jsonMembers(throwableProxyConverter, members), customizer);
5355
}
5456

5557
private static void jsonMembers(ThrowableProxyConverter throwableProxyConverter,

‎spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/logback/StructuredLogEncoder.java

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
import ch.qos.logback.core.encoder.EncoderBase;
2626

2727
import org.springframework.boot.logging.structured.CommonStructuredLogFormat;
28+
import org.springframework.boot.logging.structured.StructureLoggingJsonMembersCustomizer;
2829
import org.springframework.boot.logging.structured.StructuredLogFormatter;
2930
import org.springframework.boot.logging.structured.StructuredLogFormatterFactory;
3031
import org.springframework.boot.logging.structured.StructuredLogFormatterFactory.CommonFormatters;
@@ -85,24 +86,29 @@ private void addCommonFormatters(CommonFormatters<ILoggingEvent> commonFormatter
8586
commonFormatters.add(CommonStructuredLogFormat.LOGSTASH, this::createLogstashFormatter);
8687
}
8788

88-
private StructuredLogFormatter<ILoggingEvent> createEcsFormatter(
89-
Instantiator<StructuredLogFormatter<ILoggingEvent>> instantiator) {
89+
private StructuredLogFormatter<ILoggingEvent> createEcsFormatter(Instantiator<?> instantiator) {
9090
Environment environment = instantiator.getArg(Environment.class);
9191
ThrowableProxyConverter throwableProxyConverter = instantiator.getArg(ThrowableProxyConverter.class);
92-
return new ElasticCommonSchemaStructuredLogFormatter(environment, throwableProxyConverter);
92+
StructureLoggingJsonMembersCustomizer<?> jsonMembersCustomizer = instantiator
93+
.getArg(StructureLoggingJsonMembersCustomizer.class);
94+
return new ElasticCommonSchemaStructuredLogFormatter(environment, throwableProxyConverter,
95+
jsonMembersCustomizer);
9396
}
9497

95-
private StructuredLogFormatter<ILoggingEvent> createGraylogFormatter(
96-
Instantiator<StructuredLogFormatter<ILoggingEvent>> instantiator) {
98+
private StructuredLogFormatter<ILoggingEvent> createGraylogFormatter(Instantiator<?> instantiator) {
9799
Environment environment = instantiator.getArg(Environment.class);
98100
ThrowableProxyConverter throwableProxyConverter = instantiator.getArg(ThrowableProxyConverter.class);
99-
return new GraylogExtendedLogFormatStructuredLogFormatter(environment, throwableProxyConverter);
101+
StructureLoggingJsonMembersCustomizer<?> jsonMembersCustomizer = instantiator
102+
.getArg(StructureLoggingJsonMembersCustomizer.class);
103+
return new GraylogExtendedLogFormatStructuredLogFormatter(environment, throwableProxyConverter,
104+
jsonMembersCustomizer);
100105
}
101106

102-
private StructuredLogFormatter<ILoggingEvent> createLogstashFormatter(
103-
Instantiator<StructuredLogFormatter<ILoggingEvent>> instantiator) {
107+
private StructuredLogFormatter<ILoggingEvent> createLogstashFormatter(Instantiator<?> instantiator) {
104108
ThrowableProxyConverter throwableProxyConverter = instantiator.getArg(ThrowableProxyConverter.class);
105-
return new LogstashStructuredLogFormatter(throwableProxyConverter);
109+
StructureLoggingJsonMembersCustomizer<?> jsonMembersCustomizer = instantiator
110+
.getArg(StructureLoggingJsonMembersCustomizer.class);
111+
return new LogstashStructuredLogFormatter(throwableProxyConverter, jsonMembersCustomizer);
106112
}
107113

108114
@Override

0 commit comments

Comments
 (0)