Skip to content

Commit 8f18865

Browse files
pankajagrawal16pankajagrawal16
authored
feat: Json layout modern implementation (#670)
* feat: Modern way of configuring json layout via JsonTemplateLayout * feat: Sample ECSLayout with powertools preconfigured * docs: Modern way of configuring json layout via JsonTemplateLayout * chore: Move modern plugins to internal package
1 parent b048f82 commit 8f18865

File tree

12 files changed

+344
-6
lines changed

12 files changed

+344
-6
lines changed

‎docs/core/logging.md‎

Lines changed: 99 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,16 +13,23 @@ Logging provides an opinionated logger with output structured as JSON.
1313

1414
## Initialization
1515

16-
Powertools extends the functionality of Log4J. Below is an example `#!xml log4j2.xml` file, with the `#!java LambdaJsonLayout` configured.
16+
Powertools extends the functionality of Log4J. Below is an example `#!xml log4j2.xml` file, with the `JsonTemplateLayout` using `#!json LambdaJsonLayout.json` configured.
17+
18+
!!! info "LambdaJsonLayout is now deprecated"
19+
20+
Configuring utiltiy using `<LambdaJsonLayout/>` plugin is deprecated now. While utility still supports the old configuration, we strongly recommend upgrading the
21+
`log4j2.xml` configuration to `JsonTemplateLayout` instead. [JsonTemplateLayout](https://logging.apache.org/log4j/2.x/manual/json-template-layout.html) is recommended way of doing structured logging.
22+
23+
Please follow [this guide](#upgrade-to-jsontemplatelayout-from-deprecated-lambdajsonlayout-configuration-in-log4j2xml) for upgrade steps.
1724

1825
=== "log4j2.xml"
1926

2027
```xml hl_lines="5"
2128
<?xml version="1.0" encoding="UTF-8"?>
22-
<Configuration packages="com.amazonaws.services.lambda.runtime.log4j2">
29+
<Configuration>
2330
<Appenders>
2431
<Console name="JsonAppender" target="SYSTEM_OUT">
25-
<LambdaJsonLayout compact="true" eventEol="true"/>
32+
<JsonTemplateLayout eventTemplateUri="classpath:LambdaJsonLayout.json" />
2633
</Console>
2734
</Appenders>
2835
<Loggers>
@@ -123,6 +130,44 @@ to customise what is logged.
123130
}
124131
```
125132

133+
### Customising fields in logs
134+
135+
- Utility by default emits `timestamp` field in the logs in format `yyyy-MM-dd'T'HH:mm:ss.SSSZz` and in system default timezone.
136+
If you need to customize format and timezone, you can do so by configuring `log4j2.component.properties` and configuring properties as shown in example below:
137+
138+
=== "log4j2.component.properties"
139+
140+
```properties hl_lines="1 2"
141+
log4j.layout.jsonTemplate.timestampFormatPattern=yyyy-MM-dd'T'HH:mm:ss.SSSZz
142+
log4j.layout.jsonTemplate.timeZone=Europe/Oslo
143+
```
144+
145+
- Utility also provides sample template for [Elastic Common Schema(ECS)](https://www.elastic.co/guide/en/ecs/current/ecs-reference.html) layout.
146+
The field emitted in logs will follow specs from [ECS](https://www.elastic.co/guide/en/ecs/current/ecs-reference.html) together with field captured by utility as mentioned [above](#standard-structured-keys).
147+
148+
Use `LambdaEcsLayout.json` as `eventTemplateUri` when configuring `JsonTemplateLayout`.
149+
150+
=== "log4j2.xml"
151+
152+
```xml hl_lines="5"
153+
<?xml version="1.0" encoding="UTF-8"?>
154+
<Configuration>
155+
<Appenders>
156+
<Console name="JsonAppender" target="SYSTEM_OUT">
157+
<JsonTemplateLayout eventTemplateUri="classpath:LambdaEcsLayout.json" />
158+
</Console>
159+
</Appenders>
160+
<Loggers>
161+
<Logger name="JsonLogger" level="INFO" additivity="false">
162+
<AppenderRef ref="JsonAppender"/>
163+
</Logger>
164+
<Root level="info">
165+
<AppenderRef ref="JsonAppender"/>
166+
</Root>
167+
</Loggers>
168+
</Configuration>
169+
```
170+
126171
## Setting a Correlation ID
127172

128173
You can set a Correlation ID using `correlationIdPath` attribute by passing a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}.
@@ -417,3 +462,54 @@ via `samplingRate` attribute on annotation.
417462
Variables:
418463
POWERTOOLS_LOGGER_SAMPLE_RATE: 0.5
419464
```
465+
466+
467+
## Upgrade to JsonTemplateLayout from deprecated LambdaJsonLayout configuration in log4j2.xml
468+
469+
Prior to version [1.10.0](https://github.com/awslabs/aws-lambda-powertools-java/releases/tag/v1.10.0), only supported way of configuring `log4j2.xml` was via `<LambdaJsonLayout/>`. This plugin is
470+
deprecated now and will be removed in future version. Switching to `JsonTemplateLayout` is straight forward.
471+
472+
Below examples shows deprecated and new configuration of `log4j2.xml`.
473+
474+
=== "Deprecated configuration of log4j2.xml"
475+
476+
```xml hl_lines="5"
477+
<?xml version="1.0" encoding="UTF-8"?>
478+
<Configuration>
479+
<Appenders>
480+
<Console name="JsonAppender" target="SYSTEM_OUT">
481+
<LambdaJsonLayout compact="true" eventEol="true"/>
482+
</Console>
483+
</Appenders>
484+
<Loggers>
485+
<Logger name="JsonLogger" level="INFO" additivity="false">
486+
<AppenderRef ref="JsonAppender"/>
487+
</Logger>
488+
<Root level="info">
489+
<AppenderRef ref="JsonAppender"/>
490+
</Root>
491+
</Loggers>
492+
</Configuration>
493+
```
494+
495+
=== "New configuration of log4j2.xml"
496+
497+
```xml hl_lines="5"
498+
<?xml version="1.0" encoding="UTF-8"?>
499+
<Configuration>
500+
<Appenders>
501+
<Console name="JsonAppender" target="SYSTEM_OUT">
502+
<JsonTemplateLayout eventTemplateUri="classpath:LambdaJsonLayout.json" />
503+
</Console>
504+
</Appenders>
505+
<Loggers>
506+
<Logger name="JsonLogger" level="INFO" additivity="false">
507+
<AppenderRef ref="JsonAppender"/>
508+
</Logger>
509+
<Root level="info">
510+
<AppenderRef ref="JsonAppender"/>
511+
</Root>
512+
</Loggers>
513+
</Configuration>
514+
```
515+

‎pom.xml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,11 @@
167167
<artifactId>log4j-api</artifactId>
168168
<version>${log4j.version}</version>
169169
</dependency>
170+
<dependency>
171+
<groupId>org.apache.logging.log4j</groupId>
172+
<artifactId>log4j-layout-template-json</artifactId>
173+
<version>${log4j.version}</version>
174+
</dependency>
170175
<dependency>
171176
<groupId>org.apache.logging.log4j</groupId>
172177
<artifactId>log4j-jcl</artifactId>

‎powertools-logging/pom.xml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,10 @@
5353
<groupId>com.fasterxml.jackson.core</groupId>
5454
<artifactId>jackson-databind</artifactId>
5555
</dependency>
56-
56+
<dependency>
57+
<groupId>org.apache.logging.log4j</groupId>
58+
<artifactId>log4j-layout-template-json</artifactId>
59+
</dependency>
5760
<dependency>
5861
<groupId>org.apache.logging.log4j</groupId>
5962
<artifactId>log4j-core</artifactId>

‎powertools-logging/src/main/java/software/amazon/lambda/powertools/logging/internal/AbstractJacksonLayoutCopy.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
import com.fasterxml.jackson.databind.JsonMappingException;
3434
import com.fasterxml.jackson.databind.ObjectWriter;
3535

36+
@Deprecated
3637
abstract class AbstractJacksonLayoutCopy extends AbstractStringLayout {
3738

3839
protected static final String DEFAULT_EOL = "\r\n";

‎powertools-logging/src/main/java/software/amazon/lambda/powertools/logging/internal/JacksonFactoryCopy.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
import java.util.HashSet;
1515
import java.util.Set;
1616

17+
@Deprecated
1718
abstract class JacksonFactoryCopy {
1819

1920
static class JSON extends JacksonFactoryCopy {

‎powertools-logging/src/main/java/software/amazon/lambda/powertools/logging/internal/LambdaJsonLayout.java‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,10 @@
4343
import static java.time.Instant.ofEpochMilli;
4444
import static java.time.format.DateTimeFormatter.ISO_ZONED_DATE_TIME;
4545

46+
/***
47+
* Note: The LambdaJsonLayout should be considered to be deprecated. Please use JsonTemplateLayout instead.
48+
*/
49+
@Deprecated
4650
@Plugin(name = "LambdaJsonLayout", category = Node.CATEGORY, elementType = Layout.ELEMENT_TYPE, printObject = true)
4751
public final class LambdaJsonLayout extends AbstractJacksonLayoutCopy {
4852
private static final String DEFAULT_FOOTER = "]";

‎powertools-logging/src/main/java/software/amazon/lambda/powertools/logging/internal/PowertoolsResolver.java‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
package software.amazon.lambda.powertools.logging.internal;
2+
3+
import org.apache.logging.log4j.core.LogEvent;
4+
import org.apache.logging.log4j.layout.template.json.resolver.EventResolver;
5+
import org.apache.logging.log4j.layout.template.json.util.JsonWriter;
6+
import org.apache.logging.log4j.util.ReadOnlyStringMap;
7+
8+
final class PowertoolsResolver implements EventResolver {
9+
10+
private final EventResolver internalResolver;
11+
12+
PowertoolsResolver() {
13+
internalResolver = new EventResolver() {
14+
@Override
15+
public boolean isResolvable(LogEvent value) {
16+
ReadOnlyStringMap contextData = value.getContextData();
17+
return null != contextData && !contextData.isEmpty();
18+
}
19+
20+
@Override
21+
public void resolve(LogEvent logEvent, JsonWriter jsonWriter) {
22+
StringBuilder stringBuilder = jsonWriter.getStringBuilder();
23+
// remove dummy field to kick inn powertools resolver
24+
stringBuilder.setLength(stringBuilder.length() - 4);
25+
26+
// Inject all the context information.
27+
ReadOnlyStringMap contextData = logEvent.getContextData();
28+
contextData.forEach((key, value) -> {
29+
jsonWriter.writeSeparator();
30+
jsonWriter.writeString(key);
31+
stringBuilder.append(':');
32+
jsonWriter.writeValue(value);
33+
});
34+
}
35+
};
36+
}
37+
38+
static String getName() {
39+
return "powertools";
40+
}
41+
42+
@Override
43+
public void resolve(LogEvent value, JsonWriter jsonWriter) {
44+
internalResolver.resolve(value, jsonWriter);
45+
}
46+
47+
@Override
48+
public boolean isResolvable(LogEvent value) {
49+
return internalResolver.isResolvable(value);
50+
}
51+
}

‎powertools-logging/src/main/java/software/amazon/lambda/powertools/logging/internal/PowertoolsResolverFactory.java‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
package software.amazon.lambda.powertools.logging.internal;
2+
3+
import org.apache.logging.log4j.core.LogEvent;
4+
import org.apache.logging.log4j.core.config.plugins.Plugin;
5+
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
6+
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
7+
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverFactory;
8+
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolver;
9+
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverConfig;
10+
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverFactory;
11+
12+
@Plugin(name = "PowertoolsResolverFactory", category = TemplateResolverFactory.CATEGORY)
13+
public final class PowertoolsResolverFactory implements EventResolverFactory {
14+
15+
private static final PowertoolsResolverFactory INSTANCE = new PowertoolsResolverFactory();
16+
17+
private PowertoolsResolverFactory() {}
18+
19+
@PluginFactory
20+
public static PowertoolsResolverFactory getInstance() {
21+
return INSTANCE;
22+
}
23+
24+
@Override
25+
public String getName() {
26+
return PowertoolsResolver.getName();
27+
}
28+
29+
@Override
30+
public TemplateResolver<LogEvent> create(EventResolverContext context,
31+
TemplateResolverConfig config) {
32+
return new PowertoolsResolver();
33+
}
34+
}

‎powertools-logging/src/main/resources/LambdaEcsLayout.json‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
{
2+
"@timestamp": {
3+
"$resolver": "timestamp",
4+
"pattern": {
5+
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
6+
"timeZone": "UTC"
7+
}
8+
},
9+
"ecs.version": "1.2.0",
10+
"log.level": {
11+
"$resolver": "level",
12+
"field": "name"
13+
},
14+
"message": {
15+
"$resolver": "message",
16+
"stringified": true
17+
},
18+
"process.thread.name": {
19+
"$resolver": "thread",
20+
"field": "name"
21+
},
22+
"log.logger": {
23+
"$resolver": "logger",
24+
"field": "name"
25+
},
26+
"labels": {
27+
"$resolver": "mdc",
28+
"flatten": true,
29+
"stringified": true
30+
},
31+
"tags": {
32+
"$resolver": "ndc"
33+
},
34+
"error.type": {
35+
"$resolver": "exception",
36+
"field": "className"
37+
},
38+
"error.message": {
39+
"$resolver": "exception",
40+
"field": "message"
41+
},
42+
"error.stack_trace": {
43+
"$resolver": "exception",
44+
"field": "stackTrace",
45+
"stackTrace": {
46+
"stringified": true
47+
}
48+
},
49+
"": {
50+
"$resolver": "powertools"
51+
}
52+
}

0 commit comments

Comments
 (0)