Add support for Sentry monitoring

Includes addition to documentation.

Author: pvannierop

README.md

@@ -1,5 +1,22 @@
 # Restructure Kafka connector output files
 
+<!-- TOC -->
+* [Restructure Kafka connector output files](#restructure-kafka-connector-output-files)
+  * [Upgrade instructions](#upgrade-instructions)
+  * [Docker usage](#docker-usage)
+  * [Command line usage](#command-line-usage)
+    * [File Format](#file-format)
+    * [Compression](#compression)
+    * [Redis](#redis)
+    * [Source and target](#source-and-target)
+    * [Path format](#path-format)
+    * [Cleaner](#cleaner)
+    * [Service](#service)
+  * [Sentry monitoring](#sentry-monitoring)
+  * [Local build](#local-build)
+    * [Extending the connector](#extending-the-connector)
+<!-- TOC -->
+
 Data streamed by a Kafka Connector will be converted to a RADAR-base oriented output directory, by organizing it by project, user and collection date.
 It supports data written by [RADAR S3 sink connector](https://github.com/RADAR-base/RADAR-S3-Connector) is streamed to files based on topic name only. This package transforms that output to a local directory structure as follows: `projectId/userId/topic/date_hour.csv`. The date and hour are extracted from the `time` field of each record, and is formatted in UTC time.
 
@@ -225,6 +242,14 @@ The cleaner can also be enabled with the `--cleaner` command-line flag. To run t
 
 To run the output generator as a service that will regularly poll the source directory, add the `--service` flag and optionally the `--interval` flag to adjust the polling interval or use the corresponding configuration file parameters.
 
+## Sentry monitoring
+
+To enable Sentry monitoring:
+
+1. Set a `SENTRY_DSN` environment variable that points to the desired Sentry DSN.
+2. (Optional) Set the `SENTRY_LOG_LEVEL` environment variable to control the minimum log level of events sent to Sentry.
+   The default log level for Sentry is `WARN`. Possible values are `TRACE`, `DEBUG`, `INFO`, `WARN`, and `ERROR`.
+
 ## Local build
 
 This package requires at least Java JDK 8. Build the distribution with

build.gradle.kts

@@ -9,6 +9,9 @@ plugins {
     id("org.radarbase.radar-kotlin") version Versions.radarCommons
     id("org.radarbase.radar-publishing") version Versions.radarCommons
     id("com.avast.gradle.docker-compose") version Versions.dockerCompose
+    // TODO Remove this when new release of radar-commons is available and used in this project.
+    // This version has Sentry suppor1t built in for radar-kotlin plugin.
+    id("io.sentry.jvm.gradle") version "4.11.0"
 }
 
 description = "RADAR-base output restructuring"

src/main/resources/log4j2.xml

@@ -3,11 +3,16 @@
         <Console name="STDOUT" target="SYSTEM_OUT">
             <PatternLayout pattern="[%d] %-5level - %msg (%F:%L)%n"/>
         </Console>
+        <!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
+             When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
+        <Sentry name="Sentry" debug="false"/>
     </Appenders>
 
     <Loggers>
         <Root level="${env:LOG4J_LOG_LEVEL:-INFO}">
             <AppenderRef ref="STDOUT"/>
+            <!-- Note that the Sentry logging threshold is at WARN level by default -->
+            <appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
         </Root>
     </Loggers>
 </Configuration>