Skip to content

Commit 685feaa

Browse files
altenhofaltenhof
authored
Merge pull request #30 from DeyanZhelyazkov/master
Add custom metrics client library.
2 parents 6165e74 + ad63378 commit 685feaa

38 files changed

+1976
-13
lines changed

‎README.md

Lines changed: 113 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,31 +3,34 @@
33
[![Build Status](https://travis-ci.org/SAP/cf-java-logging-support.svg?branch=master)](https://travis-ci.org/SAP/cf-java-logging-support)
44
## Summary
55

6-
This is a collection of support libraries for Java applications running on Cloud Foundry that serve two main purposes: It provides (a) means to emit *structured application log messages* and (b) instrument parts of your application stack to *collect request metrics*.
6+
This is a collection of support libraries for Java applications running on Cloud Foundry that serves three main purposes: It provides (a) means to emit *structured application log messages*, (b) instrument parts of your application stack to *collect request metrics* and (c) java clients for producing *custom metrics*.
77

8-
When we say structured, we actually mean in JSON format. In that sense, it shares ideas with [logstash-logback-encoder](https://github.com/logstash/logstash-logback-encoder) (and a first internal version was actually based on it), but takes a simpler approach as we want to ensure that these structured messages adhere to standardized formats. With such standardized formats in place, it becomes much easier to ingest, process and search such messages in log analysis stacks like, e.g., [ELK](https://www.elastic.co/webinars/introduction-elk-stack).
8+
When we say structured, we actually mean in JSON format. In that sense, it shares ideas with [logstash-logback-encoder](https://github.com/logstash/logstash-logback-encoder) (and a first internal version was actually based on it), but takes a simpler approach as we want to ensure that these structured messages adhere to standardized formats. With such standardized formats in place, it becomes much easier to ingest, process and search such messages in log analysis stacks such as [ELK](https://www.elastic.co/webinars/introduction-elk-stack).
99

10-
If you're interested in the specifications of these standardized formats, you may want to have closer look at the `fields.yml` files in the [beats folder](./cf-java-logging-support-core/beats).
10+
If you're interested in the specifications of these standardized formats, you may want to have a closer look at the `fields.yml` files in the [beats folder](./cf-java-logging-support-core/beats).
1111

12-
While [logstash-logback-encoder](https://github.com/logstash/logstash-logback-encoder) is tied to [logback](http://logback.qos.ch/), we've tried to stay implementation neutral and have implemented the core functionality on top of [slf4j](http://www.slf4j.org/), but provide implementations for both [logback](http://logback.qos.ch/) and [log4j2](http://logging.apache.org/log4j/2.x/) (and we're open to contributions that would support other implementations).
12+
While [logstash-logback-encoder](https://github.com/logstash/logstash-logback-encoder) is tied to [logback](http://logback.qos.ch/), we've tried to keep implementation neutral and have implemented the core functionality on top of [slf4j](http://www.slf4j.org/), but provided implementations for both [logback](http://logback.qos.ch/) and [log4j2](http://logging.apache.org/log4j/2.x/) (and we're open to contributions that would support other implementations).
1313

1414
The instrumentation part is currently focusing on providing [request filters for Java Servlets](http://www.oracle.com/technetwork/java/filters-137243.html) and [client and server filters for Jersey](https://jersey.java.net/documentation/latest/filters-and-interceptors.html), but again, we're open to contributions for other APIs and frameworks.
1515

16-
Last, there are also two sibling projects on [node.js logging support](https://github.com/SAP/cf-nodejs-logging-support) and [python logging support](https://github.com/SAP/cf-python-logging-support).
16+
The custom metrics clients part allows users to easily define and push custom metrics. The clients configure all necessary components and make it possible to define custom metrics with minimal code change.
17+
18+
Lastly, there are also two sibling projects on [node.js logging support](https://github.com/SAP/cf-nodejs-logging-support) and [python logging support](https://github.com/SAP/cf-python-logging-support).
1719

1820
## Features and dependencies
1921

2022
As you can see from the structure of this repository, we're not providing one *uber* JAR that contains everything, but provide each feature separately. We also try to stay away from wiring up too many dependencies by tagging almost all of them as *provided.* As a consequence, it's your task to get all runtime dependencies resolved in your application POM file.
2123

2224
All in all, you should do the following:
2325

24-
* make up your mind which features you actually need,
25-
* adjust your Maven dependencies accordingly,
26-
* pick your favorite logging implementation, and
27-
* adjust your logging configuration accordingly.
26+
1. Make up your mind which features you actually need.
27+
2. Adjust your Maven dependencies accordingly.
28+
3. Pick your favorite logging implementation.
29+
And
30+
4. Adjust your logging configuration accordingly.
2831

2932

30-
Say, you want to make use of the *servlet filter* feature, then you need to add the following dependency to your POM with property `cf-logging-version` referring to the latest nexus version (currently `2.2.3`):
33+
Let's say you want to make use of the *servlet filter* feature, then you need to add the following dependency to your POM with property `cf-logging-version` referring to the latest nexus version (currently `2.2.3`):
3134

3235
```xml
3336
<properties>
@@ -48,9 +51,20 @@ Say, you want to make use of the *servlet filter* feature, then you need to add
4851

4952
This feature only depends on the servlet API which you have included in your POM anyhow. You can find more information about the *servlet filter* feature (like e.g. how to adjust the web.xml) in the [Wiki](https://github.com/SAP/cf-java-logging-support/wiki/Instrumenting-Servlets).
5053

54+
If you want to use the `custom metrics` `spring-boot client`, just define the following dependency:
55+
56+
``` xml
57+
58+
<dependency>
59+
<groupId>com.sap.cloud.cf.monitoring.clients</groupId>
60+
<artifactId>clients-spring-boot</artifactId>
61+
<version>${cf-logging-version}</version>
62+
</dependency>
63+
```
64+
5165
## Implementation variants and logging configurations
5266

53-
The *core* feature (on which all other features rely) is just using the `org.slf4j` API, but to actually get logs written, you need to pick an implementation feature. As stated above, we have two implementations
67+
The *core* feature (on which all other features rely) is just using the `org.slf4j` API, but to actually get logs written, you need to pick an implementation feature. As stated above, we have two implementations:
5468

5569
* `cf-java-logging-support-logback` based on [logback](http://logback.qos.ch/), and
5670
* `cf-java-logging-support-log4j2` based on [log4j2](http://logging.apache.org/log4j/2.x/).
@@ -93,9 +107,9 @@ Again, we don't include dependencies to those implementation backends ourselves,
93107
</dependency>
94108
```
95109

96-
As they have slightly different ways to do configuration, you again need to do that yourself. But we hope that we've found an easy way to accomplish that. The one thing you have to do is pick our *encoder* in your `logback.xml` if you're using `logback` or our `layout` in your `log4j2.xml`if you're using `log4j2`.
110+
As they have slightly differ in configuration, you again will need to do that yourself. But we hope that we've found an easy way to accomplish that. The one thing you have to do is pick our *encoder* in your `logback.xml` if you're using `logback` or our `layout` in your `log4j2.xml`if you're using `log4j2`.
97111

98-
Here are sort of the minimal configurations you'd need:
112+
Here are the minimal configurations you'd need:
99113

100114
*logback.xml*:
101115

@@ -144,6 +158,92 @@ Here are sort of the minimal configurations you'd need:
144158
</Configuration>
145159
```
146160

161+
## Custom metrics client usage
162+
163+
With the custom metrics clients you can send metrics defined inside your code. If you choose not to use one of these clients, you can still directly push the metrics using the REST API. Once send the metrics can be consumed in Kibana.
164+
To use the clients you'd need:
165+
166+
*Using spring-boot client in Spring Boot 2 application:*
167+
168+
``` xml
169+
<dependency>
170+
<groupId>com.sap.cloud.cf.monitoring.clients</groupId>
171+
<artifactId>clients-spring-boot</artifactId>
172+
<version>${cf-logging-version}</version>
173+
</dependency>
174+
```
175+
176+
The spring-boot client uses `Spring Boot Actuator` which allows to read predefined metrics and write custom metrics. The Actuator supports [Micrometer](https://github.com/micrometer-metrics/micrometer) and is part of Actuator's dependencies.
177+
In your code you work directly with `Micrometer`. Define your custom metrics and iterate with them:
178+
179+
``` java
180+
import java.util.ArrayList;
181+
import java.util.Arrays;
182+
import java.util.List;
183+
import java.util.concurrent.atomic.AtomicInteger;
184+
185+
import org.springframework.web.bind.annotation.RequestMapping;
186+
import org.springframework.web.bind.annotation.RestController;
187+
import io.micrometer.core.instrument.Counter;
188+
import io.micrometer.core.instrument.LongTaskTimer;
189+
import io.micrometer.core.instrument.Metrics;
190+
import io.micrometer.core.instrument.Tag;
191+
192+
@RestController
193+
public class DemoController {
194+
195+
private Counter counter;
196+
private AtomicInteger concurrentHttpRequests;
197+
private LongTaskTimer longTimer;
198+
199+
DemoController() {
200+
this.counter = Metrics.counter("demo.contoller.number.of.requests", "unit", "requests");
201+
List<Tag> tags = new ArrayList<Tag>(Arrays.asList(new Tag[] { Tag.of("parallel", "clients") }));
202+
this.concurrentHttpRequests = Metrics.gauge("demo.controller.number.of.clients.being.served", tags,
203+
new AtomicInteger(0));
204+
this.longTimer = Metrics.more().longTaskTimer("demo.controller.time.spends.in.serving.clients");
205+
}
206+
207+
@RequestMapping("/")
208+
public String index() {
209+
longTimer.record(() -> {
210+
this.counter.increment();
211+
concurrentHttpRequests.addAndGet(1);
212+
try {
213+
Thread.sleep(1000);
214+
} catch (InterruptedException e) {
215+
LOGGER.error(e);
216+
} finally {
217+
concurrentHttpRequests.addAndGet(-1);
218+
}
219+
});
220+
221+
return "Greetings from Custom Metrics!";
222+
}
223+
}
224+
```
225+
In the example above, three custom metrics are defined and used. The metrics are `Counter`, `LongTaskTimer` and `Gauge`.
226+
227+
## Custom metrics client configurations
228+
229+
This client library supports the following configurations regarding sending custom metrics:
230+
* `interval`: the interval for sending metrics, in millis. **Default value: `60000`**
231+
* `enabled`: enables or disables the sending of metrics. **Default value: `true`**
232+
* `metrics`: array of whitelisted metric names. Only mentioned metrics would be processed and sent. If it is an empty array all metrics are being sent. **Default value: `[]`**
233+
234+
Configurations are read from environment variable named `CUSTOM_METRICS`. To change the default values, you should override the environment variable with your custom values. Example:
235+
236+
```
237+
{
238+
"interval": 30000,
239+
"enabled": true,
240+
"metrics": [
241+
"my.whitelist.metric.1",
242+
"my.whitelist.metric.2"
243+
]
244+
}
245+
```
246+
147247
## Dynamic Log Levels
148248

149249
This library provides the possibility to change the log-level threshold for a

‎cf-java-monitoring-custom-metrics-clients/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
**/target/

‎cf-java-monitoring-custom-metrics-clients/clients-core/pom.xml

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
<project xmlns="http://maven.apache.org/POM/4.0.0"
2+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
4+
<modelVersion>4.0.0</modelVersion>
5+
<parent>
6+
<groupId>com.sap.cloud.cf.monitoring.clients</groupId>
7+
<artifactId>cf-java-monitoring-custom-metrics-clients</artifactId>
8+
<version>2.2.3</version>
9+
</parent>
10+
11+
<artifactId>clients-core</artifactId>
12+
<packaging>jar</packaging>
13+
<name>cf-java-monitoring-custom-metrics-clients-core</name>
14+
15+
<build>
16+
<plugins>
17+
<plugin>
18+
<groupId>org.apache.maven.plugins</groupId>
19+
<artifactId>maven-compiler-plugin</artifactId>
20+
<configuration>
21+
<source>1.7</source>
22+
<target>1.7</target>
23+
</configuration>
24+
</plugin>
25+
</plugins>
26+
</build>
27+
28+
<dependencies>
29+
<dependency>
30+
<groupId>org.apache.httpcomponents</groupId>
31+
<artifactId>httpclient</artifactId>
32+
</dependency>
33+
<dependency>
34+
<groupId>com.google.code.gson</groupId>
35+
<artifactId>gson</artifactId>
36+
</dependency>
37+
<dependency>
38+
<groupId>org.slf4j</groupId>
39+
<artifactId>slf4j-api</artifactId>
40+
<scope>provided</scope>
41+
</dependency>
42+
<!-- Test dependencies-->
43+
<dependency>
44+
<groupId>junit</groupId>
45+
<artifactId>junit</artifactId>
46+
<scope>test</scope>
47+
</dependency>
48+
<dependency>
49+
<groupId>org.mockito</groupId>
50+
<artifactId>mockito-all</artifactId>
51+
<scope>test</scope>
52+
</dependency>
53+
</dependencies>
54+
</project>

‎cf-java-monitoring-custom-metrics-clients/clients-core/src/main/java/com/sap/cloud/cf/monitoring/client/MonitoringClient.java

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
package com.sap.cloud.cf.monitoring.client;
2+
3+
import java.util.List;
4+
5+
import com.sap.cloud.cf.monitoring.client.exceptions.MonitoringClientException;
6+
import com.sap.cloud.cf.monitoring.client.model.Metric;
7+
8+
public interface MonitoringClient {
9+
10+
/**
11+
* Send single metric to the Monitoring service
12+
*
13+
* @param metric
14+
* @throws MonitoringClientException
15+
*/
16+
void send(Metric metric) throws MonitoringClientException;
17+
18+
/**
19+
* Send list of metrics to the Monitoring service
20+
*
21+
* @param metrics
22+
* @throws MonitoringClientException
23+
*/
24+
void send(List<Metric> metrics) throws MonitoringClientException;
25+
}

0 commit comments

Comments
 (0)