Skip to content
This repository was archived by the owner on Jan 9, 2020. It is now read-only.

Commit 94f7e04

Browse files
alexmnycjerryshao
authored andcommitted
[SPARK-22030][CORE] GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB
## What changes were proposed in this pull request? Upgrade codahale metrics library so that Graphite constructor can re-resolve hosts behind a CNAME with re-tried DNS lookups. When Graphite is deployed behind an ELB, ELB may change IP addresses based on auto-scaling needs. Using current approach yields Graphite usage impossible, fixing for that use case - Upgrade to codahale 3.1.5 - Use new Graphite(host, port) constructor instead of new Graphite(new InetSocketAddress(host, port)) constructor ## How was this patch tested? The same logic is used for another project that is using the same configuration and code path, and graphite re-connect's behind ELB's are no longer an issue This are proposed changes for codahale lib - dropwizard/metrics@v3.1.2...v3.1.5#diff-6916c85d2dd08d89fe771c952e3b8512R120. Specifically, https://github.com/dropwizard/metrics/blob/b4d246d34e8a059b047567848b3522567cbe6108/metrics-graphite/src/main/java/com/codahale/metrics/graphite/Graphite.java#L120 Please review http://spark.apache.org/contributing.html before opening a pull request. Author: alexmnyc <[email protected]> Closes apache#19210 from alexmnyc/patch-1.
1 parent c66d64b commit 94f7e04

File tree

4 files changed

+11
-11
lines changed

4 files changed

+11
-11
lines changed

core/src/main/scala/org/apache/spark/metrics/sink/GraphiteSink.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,8 @@ private[spark] class GraphiteSink(val property: Properties, val registry: Metric
6868
MetricsSystem.checkMinimalPollingPeriod(pollUnit, pollPeriod)
6969

7070
val graphite = propertyToOption(GRAPHITE_KEY_PROTOCOL).map(_.toLowerCase(Locale.ROOT)) match {
71-
case Some("udp") => new GraphiteUDP(new InetSocketAddress(host, port))
72-
case Some("tcp") | None => new Graphite(new InetSocketAddress(host, port))
71+
case Some("udp") => new GraphiteUDP(host, port)
72+
case Some("tcp") | None => new Graphite(host, port)
7373
case Some(p) => throw new Exception(s"Invalid Graphite protocol: $p")
7474
}
7575

dev/deps/spark-deps-hadoop-2.6

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -139,10 +139,10 @@ machinist_2.11-0.6.1.jar
139139
macro-compat_2.11-1.1.1.jar
140140
mail-1.4.7.jar
141141
mesos-1.3.0-shaded-protobuf.jar
142-
metrics-core-3.1.2.jar
143-
metrics-graphite-3.1.2.jar
144-
metrics-json-3.1.2.jar
145-
metrics-jvm-3.1.2.jar
142+
metrics-core-3.1.5.jar
143+
metrics-graphite-3.1.5.jar
144+
metrics-json-3.1.5.jar
145+
metrics-jvm-3.1.5.jar
146146
minlog-1.3.0.jar
147147
mx4j-3.0.2.jar
148148
netty-3.9.9.Final.jar

dev/deps/spark-deps-hadoop-2.7

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -140,10 +140,10 @@ machinist_2.11-0.6.1.jar
140140
macro-compat_2.11-1.1.1.jar
141141
mail-1.4.7.jar
142142
mesos-1.3.0-shaded-protobuf.jar
143-
metrics-core-3.1.2.jar
144-
metrics-graphite-3.1.2.jar
145-
metrics-json-3.1.2.jar
146-
metrics-jvm-3.1.2.jar
143+
metrics-core-3.1.5.jar
144+
metrics-graphite-3.1.5.jar
145+
metrics-json-3.1.5.jar
146+
metrics-jvm-3.1.5.jar
147147
minlog-1.3.0.jar
148148
mx4j-3.0.2.jar
149149
netty-3.9.9.Final.jar

pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@
138138
<chill.version>0.8.4</chill.version>
139139
<ivy.version>2.4.0</ivy.version>
140140
<oro.version>2.0.8</oro.version>
141-
<codahale.metrics.version>3.1.2</codahale.metrics.version>
141+
<codahale.metrics.version>3.1.5</codahale.metrics.version>
142142
<avro.version>1.7.7</avro.version>
143143
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
144144
<jets3t.version>0.9.3</jets3t.version>

0 commit comments

Comments
 (0)