FR: add custom instance name tags to instrumented services #329
Description
What happened?
We frequently create multiple instances of the same service in a single server (each instance may serve a different purpose or have an entirely different implementation). Frequently picking the instance to use is part of the server's API, although sometimes the instance is chosen with custom logic on the server for each request. We want to be able to instrument each these instances and distinguish which instance was used when creating metrics/log/trace events etc. Currently in order to distinguish between instances of the same service we need to give them distinct names (and can't use com.palantir.tritium.Tritium#instrument because the metric naming there is entirely determined by the service interface). This is our current workaround for instrumenting these:
public static <T, U extends T> T instrument(
Class<T> serviceInterface,
U delegate,
TaggedMetricRegistry metricRegistry,
String instanceName) {
String serviceName = MetricRegistry.name(serviceInterface, instanceName);
return Instrumentation.builder(serviceInterface, delegate)
.withTaggedMetrics(metricRegistry, serviceName)
// ... similarly add logging and tracing, also with the custom name
.build();
}
This gets called once for each instance of the service (with a different instanceName for each instance).
What did you want to happen?
Having multiple instances of the same service interface in a single server should ideally be handled in a first-class way. The service instance name can be added as a tag (which would allow the metric names to be consistently determined by the interface class).
One possible implementation of this would be to add a custom map of tags to TaggedMetricsServiceInvocationEventHandler and also add helper methods in com.palantir.tritium.Tritium where the instance name is an argument (in addition to all the existing args). The implementation doesn't matter as much as having a simple method to call where the args should be:
Class<T> serviceInterfanceU delegateTaggedMetricRegistry metricRegistryString instanceName
And the result is an instrumented service where the instanceName gets added to metrics (and logging/tracing etc) as a tag.