Skip to content

Commit 67bec33

Browse files
pavolloffaypavolloffay
authored
Add netty body, headers capture and filter evaluation (#193)
* Add vertx-web 3.0 payload headers capture Signed-off-by: Pavol Loffay <[email protected]> * working basic setup Signed-off-by: Pavol Loffay <[email protected]> * Add netty Signed-off-by: Pavol Loffay <[email protected]> * Working vertx test Signed-off-by: Pavol Loffay <[email protected]> * Passing unit tests Signed-off-by: Pavol Loffay <[email protected]> * Replace more Signed-off-by: Pavol Loffay <[email protected]> * Rename Signed-off-by: Pavol Loffay <[email protected]> * Passing but request body is not chunked Signed-off-by: Pavol Loffay <[email protected]> * All passing Signed-off-by: Pavol Loffay <[email protected]> * cleanup Signed-off-by: Pavol Loffay <[email protected]> * Move netty to a separate module Signed-off-by: Pavol Loffay <[email protected]> * Add readme Signed-off-by: Pavol Loffay <[email protected]> * Add readme2 Signed-off-by: Pavol Loffay <[email protected]> * Changes Signed-off-by: Pavol Loffay <[email protected]> * Data capture config Signed-off-by: Pavol Loffay <[email protected]> * Add todo Signed-off-by: Pavol Loffay <[email protected]> * Data capture config for netty-4.1 Signed-off-by: Pavol Loffay <[email protected]> * content type charset settings Signed-off-by: Pavol Loffay <[email protected]> * make timeout nicely configurable Signed-off-by: Pavol Loffay <[email protected]> * remove unused vertx instrumentation Signed-off-by: Pavol Loffay <[email protected]> * Use hypertrace call depth Signed-off-by: Pavol Loffay <[email protected]> * Working blocking for 4.0 Signed-off-by: Pavol Loffay <[email protected]> * Add blocking test for vertx Signed-off-by: Pavol Loffay <[email protected]> * blocking netty-4.1 Signed-off-by: Pavol Loffay <[email protected]> * remove prints Signed-off-by: Pavol Loffay <[email protected]> * remove todo Signed-off-by: Pavol Loffay <[email protected]> * Use implementation scope Signed-off-by: Pavol Loffay <[email protected]>
1 parent 4503f92 commit 67bec33

File tree

45 files changed

+2414
-59
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+2414
-59
lines changed

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,11 @@ List of supported frameworks with additional capabilities:
1818
| [Apache HttpClient](https://hc.apache.org/index.html) | 4.0+ |
1919
| [gRPC](https://github.com/grpc/grpc-java) | 1.5+ |
2020
| [JAX-RS Client](https://javaee.github.io/javaee-spec/javadocs/javax/ws/rs/client/package-summary.html) | 2.0+ |
21+
| [Netty](https://github.com/netty/netty) (only server) | 4.0+ |
2122
| [OkHttp](https://github.com/square/okhttp/) | 3.0+ |
2223
| [Servlet](https://javaee.github.io/javaee-spec/javadocs/javax/servlet/package-summary.html) | 2.3+ |
2324
| [Spark Web Framework](https://github.com/perwendel/spark) | 2.3+ |
25+
| [Vert.x](https://vertx.io) (only server) | 3.0+ |
2426

2527
### Adding custom filter implementation
2628

‎filter-api/src/main/java/org/hypertrace/agent/filter/mock/MockFilter.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,9 @@ class MockFilter implements Filter {
2727

2828
@Override
2929
public boolean evaluateRequestHeaders(Span span, Map<String, String> headers) {
30-
if (headers.containsKey("mockblock")) {
30+
if (headers.containsKey("http.request.header.mockblock")
31+
|| headers.containsKey("rpc.request.header.mockblock")
32+
|| headers.containsKey("mockblock")) {
3133
span.setAttribute("hypertrace.mock.filter.result", "true");
3234
return true;
3335
}

‎instrumentation/apache-httpclient-4.0/src/main/java/io/opentelemetry/javaagent/instrumentation/hypertrace/apachehttpclient/v4_0/ApacheHttpClientUtils.java

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@
1919
import io.opentelemetry.api.common.AttributeKey;
2020
import io.opentelemetry.api.trace.Span;
2121
import io.opentelemetry.javaagent.instrumentation.hypertrace.apachehttpclient.v4_0.ApacheHttpClientObjectRegistry.SpanAndAttributeKey;
22-
import java.io.ByteArrayOutputStream;
2322
import java.io.IOException;
2423
import java.io.UnsupportedEncodingException;
2524
import java.nio.charset.Charset;
@@ -31,8 +30,9 @@
3130
import org.apache.http.HttpMessage;
3231
import org.apache.http.HttpResponse;
3332
import org.hypertrace.agent.config.Config.AgentConfig;
33+
import org.hypertrace.agent.core.BoundedByteArrayOutputStream;
3434
import org.hypertrace.agent.core.BoundedByteArrayOutputStreamFactory;
35-
import org.hypertrace.agent.core.ContentEncodingUtils;
35+
import org.hypertrace.agent.core.ContentTypeCharsetUtils;
3636
import org.hypertrace.agent.core.ContentTypeUtils;
3737
import org.hypertrace.agent.core.HypertraceConfig;
3838
import org.hypertrace.agent.core.HypertraceSemanticAttributes;
@@ -101,19 +101,20 @@ public static void traceEntity(
101101
return;
102102
}
103103

104+
String charsetStr = ContentTypeUtils.parseCharset(contentType.getValue());
105+
Charset charset = ContentTypeCharsetUtils.toCharset(charsetStr);
106+
104107
if (entity.isRepeatable()) {
105108
try {
106-
ByteArrayOutputStream byteArrayOutputStream = BoundedByteArrayOutputStreamFactory.create();
109+
BoundedByteArrayOutputStream byteArrayOutputStream =
110+
BoundedByteArrayOutputStreamFactory.create(charset);
107111
entity.writeTo(byteArrayOutputStream);
108-
String encoding =
109-
entity.getContentEncoding() != null ? entity.getContentEncoding().getValue() : "";
110112

111-
Charset charset = ContentEncodingUtils.toCharset(encoding);
112113
try {
113-
String body = byteArrayOutputStream.toString(charset.name());
114+
String body = byteArrayOutputStream.toStringWithSuppliedCharset();
114115
span.setAttribute(bodyAttributeKey, body);
115116
} catch (UnsupportedEncodingException e) {
116-
log.error("Could not parse charset from encoding {}", encoding, e);
117+
log.error("Could not parse charset from encoding {}", charsetStr, e);
117118
}
118119
} catch (IOException e) {
119120
log.error("Could not read request input stream from repeatable request entity/body", e);

‎instrumentation/apache-httpclient-4.0/src/main/java/io/opentelemetry/javaagent/instrumentation/hypertrace/apachehttpclient/v4_0/HttpEntityInstrumentation.java

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@
2525

2626
import io.opentelemetry.javaagent.instrumentation.hypertrace.apachehttpclient.v4_0.ApacheHttpClientObjectRegistry.SpanAndAttributeKey;
2727
import io.opentelemetry.javaagent.tooling.TypeInstrumentation;
28-
import java.io.ByteArrayOutputStream;
2928
import java.io.InputStream;
3029
import java.io.OutputStream;
3130
import java.io.UnsupportedEncodingException;
@@ -36,10 +35,13 @@
3635
import net.bytebuddy.description.method.MethodDescription;
3736
import net.bytebuddy.description.type.TypeDescription;
3837
import net.bytebuddy.matcher.ElementMatcher;
38+
import org.apache.http.Header;
3939
import org.apache.http.HttpEntity;
40+
import org.hypertrace.agent.core.BoundedByteArrayOutputStream;
4041
import org.hypertrace.agent.core.BoundedByteArrayOutputStreamFactory;
41-
import org.hypertrace.agent.core.ContentEncodingUtils;
4242
import org.hypertrace.agent.core.ContentLengthUtils;
43+
import org.hypertrace.agent.core.ContentTypeCharsetUtils;
44+
import org.hypertrace.agent.core.ContentTypeUtils;
4345
import org.hypertrace.agent.core.GlobalObjectRegistry;
4446
import org.hypertrace.agent.core.GlobalObjectRegistry.SpanAndBuffer;
4547

@@ -88,13 +90,17 @@ public static void exit(@Advice.This HttpEntity thizz, @Advice.Return InputStrea
8890
contentSize = ContentLengthUtils.DEFAULT;
8991
}
9092

91-
String encoding =
92-
thizz.getContentEncoding() != null ? thizz.getContentEncoding().getValue() : "";
93-
Charset charset = ContentEncodingUtils.toCharset(encoding);
93+
Header contentTypeHeader = thizz.getContentType();
94+
String charsetStr = null;
95+
if (contentTypeHeader != null) {
96+
charsetStr = ContentTypeUtils.parseCharset(contentTypeHeader.getValue());
97+
}
98+
Charset charset = ContentTypeCharsetUtils.toCharset(charsetStr);
99+
94100
SpanAndBuffer spanAndBuffer =
95101
new SpanAndBuffer(
96102
clientSpan.span,
97-
BoundedByteArrayOutputStreamFactory.create((int) contentSize),
103+
BoundedByteArrayOutputStreamFactory.create((int) contentSize, charset),
98104
clientSpan.attributeKey,
99105
charset);
100106
GlobalObjectRegistry.inputStreamToSpanAndBufferMap.put(inputStream, spanAndBuffer);
@@ -114,8 +120,16 @@ public static void enter(
114120
if (contentSize <= 0 || contentSize == Long.MAX_VALUE) {
115121
contentSize = ContentLengthUtils.DEFAULT;
116122
}
117-
ByteArrayOutputStream byteArrayOutputStream =
118-
BoundedByteArrayOutputStreamFactory.create((int) contentSize);
123+
124+
Header contentTypeHeader = thizz.getContentType();
125+
String charsetStr = null;
126+
if (contentTypeHeader != null) {
127+
charsetStr = ContentTypeUtils.parseCharset(contentTypeHeader.getValue());
128+
}
129+
Charset charset = ContentTypeCharsetUtils.toCharset(charsetStr);
130+
131+
BoundedByteArrayOutputStream byteArrayOutputStream =
132+
BoundedByteArrayOutputStreamFactory.create((int) contentSize, charset);
119133

120134
GlobalObjectRegistry.outputStreamToBufferMap.put(outputStream, byteArrayOutputStream);
121135
}
@@ -129,14 +143,10 @@ public static void exit(
129143
return;
130144
}
131145

132-
String encoding =
133-
thizz.getContentEncoding() != null ? thizz.getContentEncoding().getValue() : "";
134-
Charset charset = ContentEncodingUtils.toCharset(encoding);
135-
136-
ByteArrayOutputStream bufferedOutStream =
146+
BoundedByteArrayOutputStream bufferedOutStream =
137147
GlobalObjectRegistry.outputStreamToBufferMap.remove(outputStream);
138148
try {
139-
String requestBody = bufferedOutStream.toString(charset.name());
149+
String requestBody = bufferedOutStream.toStringWithSuppliedCharset();
140150
spanAndAttributeKey.span.setAttribute(spanAndAttributeKey.attributeKey, requestBody);
141151
} catch (UnsupportedEncodingException e) {
142152
// should not happen, the charset has been parsed before

‎instrumentation/apache-httpclient-4.0/src/main/java/io/opentelemetry/javaagent/instrumentation/hypertrace/apachehttpclient/v4_0/readall/ApacheClientReadAllInstrumentationModule.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
import java.io.ByteArrayOutputStream;
3636
import java.io.IOException;
3737
import java.io.InputStream;
38+
import java.nio.charset.Charset;
3839
import java.util.Arrays;
3940
import java.util.HashMap;
4041
import java.util.List;
@@ -197,7 +198,8 @@ public static void exit(@Return Object response) {
197198

198199
BufferedInputStream bufferedInputStream = new BufferedInputStream(inputStream);
199200
ByteArrayOutputStream buffer =
200-
BoundedByteArrayOutputStreamFactory.create((int) contentSize);
201+
BoundedByteArrayOutputStreamFactory.create(
202+
(int) contentSize, Charset.defaultCharset());
201203
byte ch;
202204
while ((ch = (byte) bufferedInputStream.read()) != -1) {
203205
buffer.write(ch);

‎instrumentation/build.gradle.kts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ dependencies{
4040
implementation(project(":instrumentation:jaxrs-client-2.0"))
4141
implementation(project(":instrumentation:java-streams"))
4242
implementation(project(":instrumentation:apache-httpasyncclient-4.1"))
43+
implementation(project(":instrumentation:netty:netty-4.0"))
44+
implementation(project(":instrumentation:netty:netty-4.1"))
4345
implementation(project(":otel-extensions"))
4446
}
4547

‎instrumentation/java-streams/src/test/java/io/opentelemetry/javaagent/instrumentation/hypertrace/java/outputstream/OutputStreamInstrumentationModuleTest.java

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@
1919
import java.io.ByteArrayOutputStream;
2020
import java.io.IOException;
2121
import java.io.OutputStream;
22+
import java.nio.charset.Charset;
23+
import java.nio.charset.StandardCharsets;
24+
import org.hypertrace.agent.core.BoundedByteArrayOutputStream;
25+
import org.hypertrace.agent.core.BoundedByteArrayOutputStreamFactory;
2226
import org.hypertrace.agent.core.GlobalObjectRegistry;
2327
import org.hypertrace.agent.testing.AbstractInstrumenterTest;
2428
import org.junit.jupiter.api.Assertions;
@@ -28,6 +32,9 @@ class OutputStreamInstrumentationModuleTest extends AbstractInstrumenterTest {
2832
private static final String STR1 = "hellofoobarrandom";
2933
private static final String STR2 = "someother";
3034

35+
private static final int MAX_SIZE = Integer.MAX_VALUE;
36+
private static final Charset DEFAULT_CHARSET = StandardCharsets.UTF_8;
37+
3138
@Test
3239
public void write() {
3340
OutputStream outputStream = new TestByteArrayOutputStream();
@@ -79,7 +86,8 @@ public void writeBytesOffset() {
7986
}
8087

8188
private void write(OutputStream outputStream, Runnable read, String expected) {
82-
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
89+
BoundedByteArrayOutputStream buffer =
90+
BoundedByteArrayOutputStreamFactory.create(MAX_SIZE, DEFAULT_CHARSET);
8391
GlobalObjectRegistry.outputStreamToBufferMap.put(outputStream, buffer);
8492
read.run();
8593
Assertions.assertEquals(expected, buffer.toString());

‎instrumentation/jaxrs-client-2.0/src/main/java/io/opentelemetry/javaagent/instrumentation/hypertrace/jaxrs/v2_0/JaxrsClientEntityInterceptor.java

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@
1919
import io.opentelemetry.api.trace.Span;
2020
import io.opentelemetry.context.Context;
2121
import io.opentelemetry.javaagent.instrumentation.jaxrsclient.v2_0.ClientTracingFilter;
22-
import java.io.ByteArrayOutputStream;
2322
import java.io.IOException;
2423
import java.io.InputStream;
2524
import java.io.OutputStream;
25+
import java.nio.charset.Charset;
2626
import javax.ws.rs.WebApplicationException;
2727
import javax.ws.rs.core.HttpHeaders;
2828
import javax.ws.rs.core.MediaType;
@@ -31,9 +31,10 @@
3131
import javax.ws.rs.ext.WriterInterceptor;
3232
import javax.ws.rs.ext.WriterInterceptorContext;
3333
import org.hypertrace.agent.config.Config.AgentConfig;
34+
import org.hypertrace.agent.core.BoundedByteArrayOutputStream;
3435
import org.hypertrace.agent.core.BoundedByteArrayOutputStreamFactory;
35-
import org.hypertrace.agent.core.ContentEncodingUtils;
3636
import org.hypertrace.agent.core.ContentLengthUtils;
37+
import org.hypertrace.agent.core.ContentTypeCharsetUtils;
3738
import org.hypertrace.agent.core.ContentTypeUtils;
3839
import org.hypertrace.agent.core.GlobalObjectRegistry;
3940
import org.hypertrace.agent.core.GlobalObjectRegistry.SpanAndBuffer;
@@ -75,18 +76,21 @@ public Object aroundReadFrom(ReaderInterceptorContext responseContext)
7576
InputStream entityStream = responseContext.getInputStream();
7677
Object entity = null;
7778
try {
78-
String encodingStr = responseContext.getHeaders().getFirst(HttpHeaders.CONTENT_ENCODING);
7979
String contentLengthStr = responseContext.getHeaders().getFirst(HttpHeaders.CONTENT_LENGTH);
8080
int contentLength = ContentLengthUtils.parseLength(contentLengthStr);
8181

82-
ByteArrayOutputStream buffer = BoundedByteArrayOutputStreamFactory.create(contentLength);
82+
String charsetStr = null;
83+
if (mediaType != null) {
84+
charsetStr = mediaType.getParameters().get(MediaType.CHARSET_PARAMETER);
85+
}
86+
Charset charset = ContentTypeCharsetUtils.toCharset(charsetStr);
87+
88+
BoundedByteArrayOutputStream buffer =
89+
BoundedByteArrayOutputStreamFactory.create(contentLength, charset);
8390
GlobalObjectRegistry.inputStreamToSpanAndBufferMap.put(
8491
entityStream,
8592
new SpanAndBuffer(
86-
currentSpan,
87-
buffer,
88-
HypertraceSemanticAttributes.HTTP_RESPONSE_BODY,
89-
ContentEncodingUtils.toCharset(encodingStr)));
93+
currentSpan, buffer, HypertraceSemanticAttributes.HTTP_RESPONSE_BODY, charset));
9094
entity = responseContext.proceed();
9195
} catch (Exception ex) {
9296
log.error("Exception while capturing response body", ex);
@@ -118,8 +122,15 @@ public void aroundWriteTo(WriterInterceptorContext requestContext)
118122
}
119123
}
120124

125+
MediaType mediaType = requestContext.getMediaType();
126+
String charsetStr = null;
127+
if (mediaType != null) {
128+
charsetStr = mediaType.getParameters().get(MediaType.CHARSET_PARAMETER);
129+
}
130+
Charset charset = ContentTypeCharsetUtils.toCharset(charsetStr);
131+
121132
// TODO length is not known
122-
ByteArrayOutputStream buffer = BoundedByteArrayOutputStreamFactory.create();
133+
BoundedByteArrayOutputStream buffer = BoundedByteArrayOutputStreamFactory.create(charset);
123134
OutputStream entityStream = requestContext.getOutputStream();
124135
try {
125136
GlobalObjectRegistry.outputStreamToBufferMap.put(entityStream, buffer);
@@ -128,8 +139,8 @@ public void aroundWriteTo(WriterInterceptorContext requestContext)
128139
log.error("Failed to capture request body", ex);
129140
} finally {
130141
GlobalObjectRegistry.outputStreamToBufferMap.remove(entityStream);
131-
// TODO encoding is not known
132-
currentSpan.setAttribute(HypertraceSemanticAttributes.HTTP_REQUEST_BODY, buffer.toString());
142+
currentSpan.setAttribute(
143+
HypertraceSemanticAttributes.HTTP_REQUEST_BODY, buffer.toStringWithSuppliedCharset());
133144
}
134145
}
135146
}

‎instrumentation/netty/netty-4.0/build.gradle.kts

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
plugins {
2+
`java-library`
3+
id("net.bytebuddy.byte-buddy")
4+
id("io.opentelemetry.instrumentation.auto-instrumentation")
5+
muzzle
6+
}
7+
8+
muzzle {
9+
pass {
10+
group = "io.netty"
11+
module = "netty-codec-http"
12+
versions = "[4.0.0.Final,4.1.0.Final)"
13+
assertInverse = true
14+
}
15+
pass {
16+
group = "io.netty"
17+
module = "netty-all"
18+
versions = "[4.0.0.Final,4.1.0.Final)"
19+
assertInverse = true
20+
}
21+
fail {
22+
group = "io.netty"
23+
module = "netty"
24+
versions = "[,]"
25+
}
26+
pass {
27+
group = "io.vertx"
28+
module = "vertx-core"
29+
versions = "[2.0.0,3.3.0)"
30+
assertInverse = true
31+
}
32+
}
33+
34+
afterEvaluate{
35+
io.opentelemetry.instrumentation.gradle.bytebuddy.ByteBuddyPluginConfigurator(project,
36+
sourceSets.main.get(),
37+
"io.opentelemetry.javaagent.tooling.muzzle.collector.MuzzleCodeGenerationPlugin",
38+
project(":javaagent-tooling").configurations["instrumentationMuzzle"] + configurations.runtimeClasspath
39+
).configure()
40+
}
41+
42+
val versions: Map<String, String> by extra
43+
44+
dependencies {
45+
implementation("io.opentelemetry.javaagent.instrumentation:opentelemetry-javaagent-netty-4.0:${versions["opentelemetry_java_agent"]}")
46+
47+
implementation("io.netty:netty-codec-http:4.0.0.Final")
48+
49+
testImplementation(project(":testing-common"))
50+
}
51+

0 commit comments

Comments
 (0)