Open-telemetry Packages Causing 504/Memory Leak

[nirvana620]

Hi, we're implementing tracing in our node v12 application running on EKS K8s v1.20 utilizing nginx. The traces are being distributed to tempo and We're initializing ./tracing.js before initializing application and we've ran into several issues.

1. If we export tracing.js with the application we encounter memory leaks, we've resolved that by running tracing.js before the application.

2. Current issue, we're experiencing a 504 gateway errors on a majority of our requests, if we disable the tracing.js process we're still experiencing issues unless we comment out all the required packages.

Any advice moving forwards? Thanks!

Our tracing.js file:

const process = require('process');
const opentelemetry = require('@opentelemetry/sdk-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { WinstonInstrumentation } = require('@opentelemetry/instrumentation-winston');
const { Resource } = require('@opentelemetry/resources');
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');

const traceExporter = new OTLPTraceExporter({
  url: `http://${process.env.HOST_IP}:<IP>/v1/traces`,
});

const sdk = new opentelemetry.NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: '<service_name>',
  }),
  traceExporter,
  instrumentations: [
    getNodeAutoInstrumentations(),
    new WinstonInstrumentation()],
});

sdk.start()
  .then(() => console.log('Tracing initialized'))
  .catch(error => console.log('Error initializing tracing', error));

process.on('SIGTERM', () => {
  sdk.shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch(error => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

[vmarchaud]

Do you have more information on those 504 from the gateway logs ? Are you able to reproduce locally the memory leak ?
The best way forward would be to find a way to reproduce locally, then extract a code sample that you can share so we can reproduce ourselves and fix the issue (or even better if you have time try to find the root yourself and make a PR).