Skip to content

web.md

Latest commit

 

History

History
432 lines (320 loc) · 10 KB

web.md

File metadata and controls

432 lines (320 loc) · 10 KB

Getting started with servlet-based web

First remove any preexisting log configuration (i.e. logback.xml) and so on.

Then import the cloud-logging BOM:

Maven BOM coordinates

Add

<properties>
    <cloud-logging.version>x.y.z</cloud-logging.version>
</properties>

and

<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>bom</artifactId>
    <version>${cloud-logging.version}</version>
    <type>pom</type>
    <scope>import</scope>    
</dependency>

or

Gradle BOM coordinates

For

ext {
   cloudLoggingVersion = 'x.y.z'
}

add

implementation platform("no.entur.logging.cloud:bom:${cloudLoggingVersion}")
testImplementation platform("no.entur.logging.cloud:bom:${cloudLoggingVersion}")

Please note that features below must be individually added via artifact import.

Spring Boot starter

Add the spring-boot-starter artifact coordinates to your project.

Maven Spring Boot Starter coordinates 
<!-- both must be added -->
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>spring-boot-starter-gcp-web</artifactId>
</dependency>
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>spring-boot-starter-gcp-web-test</artifactId>
    <scope>test</scope>
</dependency>

or

Gradle Spring Boot Starter coordinates 
// both must be added
implementation ("no.entur.logging.cloud:spring-boot-starter-gcp-web")
testImplementation ("no.entur.logging.cloud:spring-boot-starter-gcp-web-test")

Configure log levels via Spring, i.e. application.properties like

logging.level.root=INFO
logging.level.my.package=WARN

Optional: Request-response logging

Import the request-response Spring Boot starters:

Maven Spring Boot Starter coordinates 
<!-- both must be added -->
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>request-response-spring-boot-starter-gcp-web</artifactId>
</dependency>
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>request-response-spring-boot-starter-gcp-web-test</artifactId>
    <scope>test</scope>
</dependency>

or

Gradle Spring Boot Starter coordinates 
// both must be added
implementation ("no.entur.logging.cloud:request-response-spring-boot-starter-gcp-web")
testImplementation ("no.entur.logging.cloud:request-response-spring-boot-starter-gcp-web-test")

Please note:

  • Logging invalid JSON or making too long log statements might have unexpected consequences for fluentbit handling in GCP kubernetes.
  • For request-response-logging, this library assumes that locally produced JSON has valid syntax and is without pretty-printing.
    • This assumption is made so to avoid the processing overhead of unnecessarily parsing JSON documents known to be valid.
    • If this assumption for some reason does not hold, use a BodyFilter to compensate.

Some Logbook excludes (actuator, openapi) are included by default. Add more using

logbook:
   exclude:
      - /too/much/data/here

See Logbook for additional configuration options.

Adjust request-response message. By default the protocol + host is included in the request-response message: 
GET https://may.app.host/v1/ticket-distribution-groups?orderId=eq%3AG3TUR9&page=1
...
200 OK https://may.app.host/v1/ticket-distribution-groups?orderId=eq%3AG3HUR9&page=1 (in 50 ms)

this can be simplified to

GET /v1/ticket-distribution-groups
...
200 GET /v1/ticket-distribution-groups (in 50 ms)

Incoming calls:

entur:
  logging:
    request-response:
      format:
        server:
          message:
            scheme: false
            host: false
            port: false
            path: false
            query: false

and/or for outgoing calls:

entur:
  logging:
    request-response:
      format:
        client:
          message:
            scheme: false
            host: false
            port: false
            path: false
            query: false
Adjust request-response logger name. ``` entur.logging.request-response.logger.level=INFO entur.logging.request-response.logger.name=no.entur.logging.cloud ```

Optional: On-demand logging

This feature adjusts the log level for individual web server requests, taking into account actual behaviour.

  • increase log level for happy-cases (i.e. WARN or ERROR), otherwise
  • reduce log level (i.e. INFO) for
    • unexpected HTTP response codes
    • unexpected log statement levels (i.e. ERROR)
    • unexpectedly long call duration
    • troubleshooting

Import the on-demand Spring Boot starters:

Maven Spring Boot Starter coordinates 
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>on-demand-spring-boot-starter-gcp-web</artifactId>
</dependency>

or

Gradle Spring Boot Starter coordinates 
implementation ("no.entur.logging.cloud:on-demand-spring-boot-starter-gcp-web")

While disabled by default, on-demand logging can be enabled using

entur.logging.http.ondemand.enabled=true

Set the servlet order and pattern (defaults):

entur.logging.http.ondemand.filter-order=-2147483648
entur.logging.http.ondemand.filter-url-patterns=/*

Then configure log levels

entur.logging.http.ondemand.success.level=warn
entur.logging.http.ondemand.failure.level=info

where

  • success: log level for the happy case
    • cached log statements are discarded
  • failure: log level for the unhappy case
    • cached log statements are printed

A failure can be triggered by

  • HTTP status codes
  • High severity log statements (i.e. warn or error)

configured as by

entur.logging.http.ondemand.failure.http.status-code.equalOrHigherThan=400
# or
entur.logging.http.ondemand.failure.http.status-code.equal-to=404,405
# or
entur.logging.http.ondemand.failure.http.status-code.not-equal-to=200,201

and/or by log level severity

entur.logging.http.ondemand.failure.logger.level=error

optionally limited to specific loggers, i.e.

entur.logging.http.ondemand.failure.logger.name[0]=my.app
entur.logging.http.ondemand.failure.logger.name[1]=my.lib

There is also a troubleshoot variant

entur.logging.http.ondemand.troubleshoot.level=debug
entur.logging.http.ondemand.troubleshoot.http.headers[0].name=x-debug-this-request

which allows for additional logging in the presence of certain HTTP headers.

Toggle log output-format during testing

Toggling between output modes in a unit test:

try (Closeable c = CompositeConsoleOutputControl.useHumanReadableJsonEncoder()) {
    // your logging here
}

Additional log levels

For additional error levels, try the DevOpsLogger:

DevOpsLogger LOGGER = DevOpsLoggerFactory.getLogger(MyClass.class);

// ... your code here

LOGGER.errorTellMeTomorrow("Error statement");
LOGGER.errorInterruptMyDinner("Critical statement");
LOGGER.errorWakeMeUpRightNow("Alert statement");
Maven Logger API coordinates 
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>api</artifactId>
</dependency>

or

Gradle Logger API coordinates 
implementation ("no.entur.logging.cloud:api")

gRPC client MDC support

To forward correlation id + add MDC-style context to log statements within gRPC interceptors (via Scope), add artifacts

Maven gRPC MDC coordinates 
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>correlation-id-trace-grpc-netty</artifactId>
</dependency>
<dependency>
    <groupId>no.entur.logging.cloud</groupId>
    <artifactId>spring-boot-autoconfigure-gcp-grpc-mdc</artifactId>
</dependency>

or

Gradle gRPC MDC coordinates 
implementation ("no.entur.logging.cloud:correlation-id-trace-grpc-netty")
implementation ("no.entur.logging.cloud:spring-boot-autoconfigure-gcp-grpc-mdc")

and

  • use the CopyCorrelationIdFromGrpcMdcContextToRequestClientInterceptor client interceptor,
  • wrap the call(s) in a GrpcMdcContext (or CorrelationIdGrpcMdcContext).

Running applications locally

For 'classic' one-line log output when running a server locally, additionally add the logging test artifacts to the main scope during local execution only.

  • Maven: Use profiles
  • Gradle:
    • Use configurations, and/or
    • add dependencies directly to task
Gradle bootRun example 
tasks.register("logPlainly") {
   dependencies {
      implementation("no.entur.logging.cloud:request-response-spring-boot-starter-gcp-web-test")
      implementation("no.entur.logging.cloud:spring-boot-starter-gcp-web-test")
   }
}

tasks.withType(JavaExec).configureEach {
   dependsOn("logPlainly")
}

Then configure desired output by specifying entur.logging.style

entur.logging.style=humanReadablePlain|humanReadableJson|machineReadableJson

Recommended additions

Add Prometheus via io.micrometer:micrometer-registry-prometheus.

Troubleshooting

request-response logging not working

Did you import the relevant artifacts? Both the main and test artifacts must be added.

on-demand logging not working

Did you import the relevant artifact?

The feature is disabled by default; set the following property to enable it:

entur.logging.http.ondemand.enabled=true