Apache and Ngnix OtelExporterOtlpHeaders changes (#240)

gnm444 · DebajitDas · web-flow · commit 715bab89e061 · 2023-01-04T11:39:19.000+05:30
* Apache OtelExporterHeaders changes commit -1

Apache OtelExporterHeaders changes commit -1

* Update ApacheConfig.cpp

Added  OTLP headers in getProcessConfig

* OtelExportHeader config count changes

config count added , env_config added

* appd_set_otelExporterOtlpHeaders correction

appd_set_otelExporterOtlpHeaders ApacheModuleOtelExporterHeaders are added

* Changes related OTLP headers

Added Tenant config changes

* Merge conflicts resolved regarding appd namespace

Merge conflicts resolved regarding appd namespace

* updating comments

updating comments

* Removed space

Removed extra space

* headers variable is correctly added

Added metadata asper the structure

* Adding otlp headers

Added Otlp headers instead of string

* Correcting typo

Correcting mistake in declaration

* Export headers tokenization

Export headers tokens parsing

* Changes for tokenization

Changes for tokenization

* appd to otel name change

appd to otel name change

* appd to otel

name change

* appd name change

appd name change

* corrections in name space

appd to otel

* Ngnix changes for OTEL Http Headers

Ngnix changes for OTEL Http Headers

* Declaration of ngx_str_t

ngx_str_t

* Updating variable

variable name

* Disabling OTEL headers

This is only for production not for dev

* Review comments addressed

Review comments added, name changes done

* Review comments -2

Changes in naming

* Api Utils change

Changes in ApiUtils to read from env

* Reverted unwanted space 

Reverted unwanted space  in Integration test

* Enabling the otel headers config 

Enabling the otel headers config

* Checking for empty strings

Empty headers

* Disable OTLP headers

Disable OTLP headers

* Adding a check to fix IT

Integration test

* Reverting the check

Reverting some changes

* Initializaing the otelExporterEndpoint

otelExporterEndpoint initialization

* Updating README

updated the new about the new directive ApacheModuleOtelExporterHeaders

* Read me updated for OTEL exporter headers directive

Added Read me info on OTEL exporter headers

Co-authored-by: DEBAJIT DAS &lt;85024550+DebajitDas@users.noreply.github.com&gt;
diff --git a/instrumentation/otel-webserver-module/README.md b/instrumentation/otel-webserver-module/README.md
@@ -58,6 +58,7 @@ Monitoring individual modules is crucial to the instrumentation of Apache web se
 |*ApacheModuleWebserverContext*                  |                 | OPTIONAL: Takes 3 values(space-seperated) ServiceName, ServiceNamespace and ServiceInstanceId|
 |*ApacheModuleSegmentType*                       |                 | OPTIONAL: Specify the string (FIRST/LAST/CUSTOM) to be filtered for Span Name Creation|
 |*ApacheModuleSegmentParameter*                  |                 | OPTIONAL: Specify the segment count or segment numbers that you want to display for Span Creation|
+|*ApacheModuleOtelExporterHeaders*               |                 | OPTIONAL: OTEL Exporter header info or Metadata like API key for OTLP endpoint. a list of key value pairs, and these are expected to be represented in a format matching to the W3C Correlation-Context, except that additional semi-colon delimited metadata is not supported, i.e.: key1=value1,key2=value2. |
 
 A sample configuration is mentioned in [opentelemetry_module.conf](https://github.com/cisco-open/otel-webserver-module/blob/main/opentelemetry_module.conf)
 
@@ -174,6 +175,7 @@ Currently, Nginx Webserver module monitores some fixed set of modules, which get
 |*NginxModuleSegmentParameter*                  |                 | OPTIONAL: Specify the segment count or segment numbers that you want to display for Span Creation|
 |*NginxModuleRequestHeaders*                    |                 | OPTIONAL: Specify the request headers to be captured in the span attributes. The headers are Case-Sensitive and should be comma-separated. e.g.```NginxModuleRequestHeaders               Accept-Charset,Accept-Encoding,User-Agent;```|
 |*NginxModuleResponseHeaders*                   |                  | OPTIONAL: Specify the response headers to be captured in the span attributes. The headers are Case-Sensitive and should be comma-separated. e.g.```NginxModuleResponseHeaders                  Content-Length,Content-Type;```|
+|*NginxModuleOtelExporterOtlpHeaders*           |                  | OPTIONAL: OTEL exporter headers like Meta data related exposrted end point. a list of key value pairs, and these are expected to be represented in a format matching to the W3C Correlation-Context, except that additional semi-colon delimited metadata is not supported, i.e.: key1=value1,key2=value2.|
 
 ### Build and Installation
 #### Prerequisites
diff --git a/instrumentation/otel-webserver-module/conf/nginx/opentelemetry_module.conf b/instrumentation/otel-webserver-module/conf/nginx/opentelemetry_module.conf
@@ -2,6 +2,7 @@ NginxModuleEnabled ON;
 #NginxModule Otel Exporter details
 NginxModuleOtelSpanExporter otlp;
 NginxModuleOtelExporterEndpoint docker.for.mac.localhost:4317;
+#NginxModuleOtelExporterOtlpHeaders Authorization=AuthorizationToken;
 # SSL Certificates
 #NginxModuleOtelSslEnabled ON
 #NginxModuleOtelSslCertificatePath 
diff --git a/instrumentation/otel-webserver-module/include/apache/ApacheConfig.h b/instrumentation/otel-webserver-module/include/apache/ApacheConfig.h
@@ -39,6 +39,9 @@ class otel_cfg
     const char* getOtelExporterEndpoint() { return otelExporterEndpoint; }
     int getOtelExporterEndpointInitialized() { return otelExporterEndpoint_initialized; }
 
+    const char* getOtelExporterOtlpHeaders() { return otelExporterOtlpHeaders; }
+    int otelExporterOtlpHeadersInitialized() { return otelExporterOtlpHeaders_initialized; }
+
     int getOtelSslEnabled() { return otelSslEnabled; }
     int getOtelSslEnabledInitialized() { return otelSslEnabled_initialized; }
 
@@ -123,6 +126,9 @@ class otel_cfg
     const char *otelExporterEndpoint;   // REQUIRED: Collector endpoint where the OpenTelemetry Exporter inside OTel SDK sends traces
     int otelExporterEndpoint_initialized;
 
+    const char *otelExporterOtlpHeaders;   // OPTIONAL: AppDynamics  Custom metadata for OTEL Exporter EX: OTEL_EXPORTER_OTLP_HEADERS="api-key=key,other-config-value=value"
+    int otelExporterOtlpHeaders_initialized;
+
     int otelSslEnabled;      // OPTIONAL: Decision whether connection to the Exporter endpoint is secured
     int otelSslEnabled_initialized;
 
@@ -224,6 +230,7 @@ class ApacheConfigHandlers
     static const char* otel_set_enabled(cmd_parms *cmd, void *conf, const char *arg);
     static const char* otel_set_otelExporterType(cmd_parms *cmd, void *conf, const char *arg);
     static const char* otel_set_otelExporterEndpoint(cmd_parms *cmd, void *conf, const char *arg);
+    static const char* otel_set_otelExporterOtlpHeaders(cmd_parms *cmd, void *conf, const char *arg);
     static const char* otel_set_otelSslEnabled(cmd_parms *cmd, void *conf, const char *arg);
     static const char* otel_set_otelSslCertificatePath(cmd_parms *cmd, void *conf, const char *arg);
     static const char* otel_set_otelProcessorType(cmd_parms *cmd, void *conf, const char *arg);
diff --git a/instrumentation/otel-webserver-module/include/apache/ApacheHooks.h b/instrumentation/otel-webserver-module/include/apache/ApacheHooks.h
@@ -33,6 +33,7 @@ class ApacheHooks
     static const char* OTEL_INTERACTION_HANDLE_KEY;
     static const char* OTEL_REQ_HANDLE_KEY;
     static const int LOWEST_HTTP_ERROR_CODE = 400;
+    static const int CONFIG_COUNT = 17; // Number of key value pairs in config
     static const std::initializer_list<const char*> httpHeaders;
     static void registerHooks(apr_pool_t *p);
 
diff --git a/instrumentation/otel-webserver-module/include/core/api/OpentelemetrySdk.h b/instrumentation/otel-webserver-module/include/core/api/OpentelemetrySdk.h
@@ -27,6 +27,7 @@
 /* {{{ For API user: environment variables to specify Controller connectivity */
 #define OTEL_SDK_ENV_OTEL_EXPORTER_TYPE "OTEL_SDK_ENV_OTEL_EXPORTER_TYPE"
 #define OTEL_SDK_ENV_OTEL_EXPORTER_ENDPOINT "OTEL_SDK_ENV_OTEL_EXPORTER_ENDPOINT"             /*required*/
+#define OTEL_SDK_ENV_OTEL_EXPORTER_OTLPHEADERS "OTEL_SDK_ENV_OTEL_EXPORTER_OTLPHEADERS"             /*optional*/
 #define OTEL_SDK_ENV_OTEL_SSL_ENABLED "OTEL_SDK_ENV_OTEL_SSL_ENABLED"             /*optional*/
 #define OTEL_SDK_ENV_OTEL_SSL_CERTIFICATE_PATH "OTEL_SDK_ENV_OTEL_SSL_CERTIFICATE_PATH"   /*optional*/
 #define OTEL_SDK_ENV_OTEL_PROCESSOR_TYPE "OTEL_SDK_ENV_OTEL_PROCESSOR_TYPE"
@@ -53,6 +54,7 @@
 #define OTEL_SDK_ENV_SEGMENT_TYPE "OTEL_SDK_ENV_SEGMENT_TYPE"
 #define OTEL_SDK_ENV_SEGMENT_PARAMETER "OTEL_SDK_ENV_SEGMENT_PARAMETER"
 
+
 /* {{{ For API user: API User logger */
 #define OTEL_LOG_API_USER_LOGGER  "api_user"    /*logging at the level of sdk function call*/
 #define OTEL_LOG_API_LOGGER  "api"              /*logging at the level of sdk core functionality*/
diff --git a/instrumentation/otel-webserver-module/include/core/api/TenantConfig.h b/instrumentation/otel-webserver-module/include/core/api/TenantConfig.h
@@ -45,6 +45,7 @@ class TenantConfig
     const std::string& getOtelLibraryVersion() const {return otelLibraryVersion;}
     const std::string& getOtelExporterType() const {return otelExporterType;}
     const std::string& getOtelExporterEndpoint() const {return otelExporterEndpoint;}
+    const std::string& getOtelExporterOtlpHeaders() const {return otelExporterOtlpHeaders;}
     const std::string& getOtelProcessorType() const {return otelProcessorType;}
     const unsigned getOtelMaxQueueSize() const {return otelMaxQueueSize;}
     const unsigned getOtelScheduledDelayMillis() const {return otelScheduledDelayMillis;}
@@ -61,6 +62,7 @@ class TenantConfig
     void setServiceInstanceId(const std::string& serviceInstanceId) { this->serviceInstanceId = serviceInstanceId; }
     void setOtelExporterType(const std::string& otelExporterType) { this->otelExporterType = otelExporterType; }
     void setOtelExporterEndpoint(const std::string& otelExporterEndpoint) { this->otelExporterEndpoint = otelExporterEndpoint; }
+    void setOtelExporterOtlpHeaders(const std::string& otelExporterOtlpHeaders) { this->otelExporterOtlpHeaders = otelExporterOtlpHeaders; }
     void setOtelProcessorType(const std::string& otelProcessorType) { this->otelProcessorType = otelProcessorType; }
     void setOtelMaxQueueSize(const unsigned int otelMaxQueueSize) { this->otelMaxQueueSize = otelMaxQueueSize; }
     void setOtelScheduledDelayMillis(const unsigned int otelScheduledDelayMillis) { this->otelScheduledDelayMillis = otelScheduledDelayMillis; }
@@ -80,6 +82,7 @@ class TenantConfig
 
     std::string otelExporterType;
     std::string otelExporterEndpoint;
+    std::string otelExporterOtlpHeaders;
     bool otelSslEnabled;
     std::string otelSslCertPath;
 
@@ -108,6 +111,7 @@ inline std::ostream& operator<< (std::ostream &os, const otel::core::TenantConfi
         << "\n OtelSamplerType                  " << config.getOtelSamplerType()
         << "\n OtelSslEnabled                   " << config.getOtelSslEnabled()
         << "\n OtelSslCertPath                  " << config.getOtelSslCertPath()
+        << "\n OtelExportOtlpHeaders            " << config.getOtelExporterOtlpHeaders()
         << "";
     return os;
 }
diff --git a/instrumentation/otel-webserver-module/opentelemetry_module.conf b/instrumentation/otel-webserver-module/opentelemetry_module.conf
@@ -14,6 +14,7 @@ ApacheModuleEnabled ON
 #ApacheModule Otel Exporter details
 ApacheModuleOtelSpanExporter otlp
 ApacheModuleOtelExporterEndpoint collector:4317
+#ApacheModuleOtelExporterHeaders api-key=abc123
 
 # SSL Certificates
 #ApacheModuleOtelSslEnabled ON
diff --git a/instrumentation/otel-webserver-module/src/apache/ApacheConfig.cpp b/instrumentation/otel-webserver-module/src/apache/ApacheConfig.cpp
@@ -121,6 +121,14 @@ const char* ApacheConfigHandlers::otel_set_otelExporterEndpoint(cmd_parms *cmd,
     return helperChar(cmd, cfg, arg, cfg->otelExporterEndpoint, cfg->otelExporterEndpoint_initialized, "otel_set_otelExporterEndpoint");
 }
 
+//  char *otelExporterOtlpHeaders;
+//  int otelExporterOtlpHeaders_initialized;
+const char* ApacheConfigHandlers::otel_set_otelExporterOtlpHeaders(cmd_parms *cmd, void *conf, const char *arg)
+{
+    otel_cfg* cfg = (otel_cfg*) conf;
+    return helperChar(cmd, cfg, arg, cfg->otelExporterOtlpHeaders, cfg->otelExporterOtlpHeaders_initialized, "otel_set_otelExporterOtlpHeaders");
+}
+
 //  char *otelSslEnabled;
 //  int otelSslEnabled_initialized;
 const char* ApacheConfigHandlers::otel_set_otelSslEnabled(cmd_parms *cmd, void *conf, const char *arg)
@@ -432,6 +440,10 @@ void otel_cfg::init()
     otelExporterEndpoint = "";
     otelExporterEndpoint_initialized = 0;
 
+    // otelExporterOtlpHeaders         Optional: OTLP headers as key value pairs
+    otelExporterOtlpHeaders = "";
+    otelExporterOtlpHeaders_initialized = 0;
+
     // otelSslEnabled       OPTIONAL: Decides whether the connection to the endpoint is secured
     otelSslEnabled = 0;
     otelSslEnabled_initialized = 0;
@@ -771,6 +783,9 @@ otel_cfg* ApacheConfigHandlers::getProcessConfig(const request_rec* r)
     process_cfg->otelExporterEndpoint = apr_pstrdup(r->server->process->pool, our_config->otelExporterEndpoint);
     process_cfg->otelExporterEndpoint_initialized = our_config->otelExporterEndpoint_initialized;
 
+    process_cfg->otelExporterOtlpHeaders = apr_pstrdup(r->server->process->pool, our_config->otelExporterOtlpHeaders);
+    process_cfg->otelExporterOtlpHeaders_initialized = our_config->otelExporterOtlpHeaders_initialized;
+
     process_cfg->otelSslEnabled = our_config->otelSslEnabled;
     process_cfg->otelSslEnabled_initialized = our_config->otelSslEnabled_initialized;
 
diff --git a/instrumentation/otel-webserver-module/src/apache/ApacheHooks.cpp b/instrumentation/otel-webserver-module/src/apache/ApacheHooks.cpp
@@ -339,10 +339,11 @@ bool ApacheHooks::initialize_opentelemetry(const request_rec *r)
         wsAgent.initDependency();
 
         // ENV RECORDS SIZE TO INCLUDE THE LOG PATH AND THE AGGREGATOR DIRECTORY
-        //
-        // Update the apr_pcalloc if we add another parameter to the input array!
+        // Update the CONFIG_COUNT in apr_pcalloc if we add another parameter to the input array!
+        
         OTEL_SDK_ENV_RECORD* env_config =
-                (OTEL_SDK_ENV_RECORD*) apr_pcalloc(r->pool, 16 * sizeof(OTEL_SDK_ENV_RECORD));
+                (OTEL_SDK_ENV_RECORD*) apr_pcalloc(r->pool, CONFIG_COUNT * sizeof(OTEL_SDK_ENV_RECORD));
+
         int ix = 0;
 
         // Otel Exporter Type
@@ -425,6 +426,11 @@ bool ApacheHooks::initialize_opentelemetry(const request_rec *r)
         env_config[ix].value = our_config->getSegmentParameter();
         ++ix;
 
+        // Segment Parameter
+        env_config[ix].name = OTEL_SDK_ENV_OTEL_EXPORTER_OTLPHEADERS;
+        env_config[ix].value = our_config->getOtelExporterOtlpHeaders();
+        ++ix;
+
         // !!!
         // Remember to update the apr_pcalloc call size if we add another parameter to the input array!
         // !!!
diff --git a/instrumentation/otel-webserver-module/src/apache/mod_apache_otel.cpp b/instrumentation/otel-webserver-module/src/apache/mod_apache_otel.cpp
@@ -64,6 +64,12 @@ static const command_rec otel_cmds[] =
             NULL,
             OR_ALL,
             "Collector Endpoint where the OpenTelemetry Exporter inside OTel SDK sends traces"),
+    AP_INIT_TAKE1(
+            "apacheModuleOtelExporterHeaders",
+            (CMD_HAND_TYPE)ApacheConfigHandlers::otel_set_otelExporterOtlpHeaders,
+            NULL,
+            OR_ALL,
+            "AppDynamics Otel export Headers key value pairs"),
     AP_INIT_TAKE1(
             "apacheModuleOtelSslEnabled",
             (CMD_HAND_TYPE)ApacheConfigHandlers::otel_set_otelSslEnabled,
diff --git a/instrumentation/otel-webserver-module/src/core/api/ApiUtils.cpp b/instrumentation/otel-webserver-module/src/core/api/ApiUtils.cpp
@@ -167,6 +167,7 @@ OTEL_SDK_STATUS_CODE ApiUtils::ReadSettingsFromReader(
 
     std::string otelExporterType;
     std::string otelExporterEndpoint;
+    std::string otelExporterOtlpHeaders;
     bool otelSslEnabled;
     std::string otelSslCertPath;
     std::string otelLibraryName;
@@ -255,12 +256,16 @@ OTEL_SDK_STATUS_CODE ApiUtils::ReadSettingsFromReader(
     reader.ReadOptional(
         std::string(OTEL_SDK_ENV_SEGMENT_PARAMETER), segmentParameter);
 
+    reader.ReadOptional(
+            std::string(OTEL_SDK_ENV_OTEL_EXPORTER_OTLPHEADERS), otelExporterOtlpHeaders);
+
 
     tenantConfig.setServiceNamespace(serviceNamespace);
     tenantConfig.setServiceName(serviceName);
     tenantConfig.setServiceInstanceId(serviceInstanceId);
     tenantConfig.setOtelExporterType(otelExporterType);
     tenantConfig.setOtelExporterEndpoint(otelExporterEndpoint);
+    tenantConfig.setOtelExporterOtlpHeaders(otelExporterOtlpHeaders);
     tenantConfig.setOtelLibraryName(otelLibraryName);
     tenantConfig.setOtelProcessorType(otelProcessorType);
     tenantConfig.setOtelSamplerType(otelSamplerType);
diff --git a/instrumentation/otel-webserver-module/src/core/sdkwrapper/SdkHelperFactory.cpp b/instrumentation/otel-webserver-module/src/core/sdkwrapper/SdkHelperFactory.cpp
@@ -26,6 +26,7 @@
 #include "opentelemetry/sdk/trace/samplers/trace_id_ratio.h"
 #include "opentelemetry/sdk/resource/resource.h"
 #include "opentelemetry/exporters/otlp/otlp_grpc_exporter.h"
+#include "opentelemetry/exporters/otlp/otlp_environment.h"
 #include "opentelemetry/baggage/propagation/baggage_propagator.h"
 #include <module_version.h>
 #include <fstream>
@@ -132,6 +133,32 @@ OtelSpanExporter SdkHelperFactory::GetExporter(
             LOG4CXX_TRACE(mLogger, "Ssl Credentials are enabled for exporter, path: "
                 << opts.ssl_credentials_cacert_path);
         }
+
+        opentelemetry::common::KeyValueStringTokenizer tokenizer{config->getOtelExporterOtlpHeaders()};
+        opentelemetry::nostd::string_view header_key;
+        opentelemetry::nostd::string_view header_value;
+        bool header_valid = true;
+        std::unordered_set<std::string> remove_cache;
+
+        while (tokenizer.next(header_valid, header_key, header_value))
+        {
+            if (header_valid)
+            {
+                std::string key = static_cast<std::string>(header_key);
+                if (remove_cache.end() == remove_cache.find(key))
+                {
+                    remove_cache.insert(key);
+                    auto range = opts.metadata.equal_range(key);
+                    if (range.first != range.second)
+                    {
+                        opts.metadata.erase(range.first, range.second);
+                    }
+                }
+
+                opts.metadata.emplace(std::make_pair(std::move(key), static_cast<std::string>(header_value)));
+            }
+        }
+
         exporter.reset(new opentelemetry::exporter::otlp::OtlpGrpcExporter(opts));
     }
 
diff --git a/instrumentation/otel-webserver-module/src/nginx/ngx_http_opentelemetry_module.c b/instrumentation/otel-webserver-module/src/nginx/ngx_http_opentelemetry_module.c
@@ -202,6 +202,13 @@ static ngx_command_t ngx_http_opentelemetry_commands[] = {
       offsetof(ngx_http_opentelemetry_loc_conf_t, nginxModuleOtelExporterEndpoint),
       NULL},
 
+    { ngx_string("NginxModuleOtelExporterOtlpHeaders"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_str_slot,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      offsetof(ngx_http_opentelemetry_loc_conf_t, nginxModuleOtelExporterOtlpHeaders),
+      NULL},
+
     { ngx_string("NginxModuleOtelSpanProcessor"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
       ngx_conf_set_str_slot,
@@ -449,6 +456,7 @@ static char* ngx_http_opentelemetry_merge_loc_conf(ngx_conf_t *cf, void *parent,
 
     ngx_conf_merge_str_value(conf->nginxModuleOtelSpanExporter, prev->nginxModuleOtelSpanExporter, "");
     ngx_conf_merge_str_value(conf->nginxModuleOtelExporterEndpoint, prev->nginxModuleOtelExporterEndpoint, "");
+    ngx_conf_merge_str_value(conf->nginxModuleOtelExporterOtlpHeaders, prev->nginxModuleOtelExporterOtlpHeaders, "");
     ngx_conf_merge_value(conf->nginxModuleOtelSslEnabled, prev->nginxModuleOtelSslEnabled, 0);
     ngx_conf_merge_str_value(conf->nginxModuleOtelSslCertificatePath, prev->nginxModuleOtelSslCertificatePath, "");
     ngx_conf_merge_str_value(conf->nginxModuleOtelSpanProcessor, prev->nginxModuleOtelSpanProcessor, "");
@@ -913,7 +921,7 @@ static ngx_flag_t ngx_initialize_opentelemetry(ngx_http_request_t *r)
 
 
         // Update the apr_pcalloc if we add another parameter to the input array!
-        OTEL_SDK_ENV_RECORD* env_config = ngx_pcalloc(r->pool, 16 * sizeof(OTEL_SDK_ENV_RECORD));
+        OTEL_SDK_ENV_RECORD* env_config = ngx_pcalloc(r->pool, CONFIG_COUNT * sizeof(OTEL_SDK_ENV_RECORD));
         if(env_config == NULL)
         {
             ngx_writeError(r->connection->log, __func__, "Not Able to allocate memory for the Env Config");
@@ -936,6 +944,11 @@ static ngx_flag_t ngx_initialize_opentelemetry(ngx_http_request_t *r)
         env_config[ix].value = (const char*)(conf->nginxModuleOtelExporterEndpoint).data;
         ++ix;
 
+        // Otel Exporter OTEL headers
+        env_config[ix].name = OTEL_SDK_ENV_OTEL_EXPORTER_OTLPHEADERS;
+        env_config[ix].value = (const char*)(conf->nginxModuleOtelExporterOtlpHeaders).data;
+        ++ix;
+
         // Otel SSL Enabled
         env_config[ix].name = OTEL_SDK_ENV_OTEL_SSL_ENABLED;
         env_config[ix].value = conf->nginxModuleOtelSslEnabled == 1 ? "1" : "0";
@@ -1446,6 +1459,7 @@ static void traceConfig(ngx_http_request_t *r, ngx_http_opentelemetry_loc_conf_t
     ngx_writeTrace(r->connection->log, __func__, " Config { :"
                                                       "(Enabled=\"%ld\")"
                                                       "(OtelExporterEndpoint=\"%s\")"
+                                                      "(OtelExporterOtlpHeader=\"%s\")"
                                                       "(OtelSslEnabled=\"%ld\")"
                                                       "(OtelSslCertificatePath=\"%s\")"
                                                       "(OtelSpanExporter=\"%s\")"
@@ -1468,6 +1482,7 @@ static void traceConfig(ngx_http_request_t *r, ngx_http_opentelemetry_loc_conf_t
                                                       " }",
                                                       conf->nginxModuleEnabled,
                                                       (conf->nginxModuleOtelExporterEndpoint).data,
+                                                      (conf->nginxModuleOtelExporterOtlpHeaders).data,
                                                       conf->nginxModuleOtelSslEnabled,
                                                       (conf->nginxModuleOtelSslCertificatePath).data,
                                                       (conf->nginxModuleOtelSpanExporter).data,
diff --git a/instrumentation/otel-webserver-module/src/nginx/ngx_http_opentelemetry_module.h b/instrumentation/otel-webserver-module/src/nginx/ngx_http_opentelemetry_module.h
@@ -23,6 +23,7 @@
 
 #define LOWEST_HTTP_ERROR_CODE 400
 #define STATUS_CODE_BYTE_COUNT 6
+static const int CONFIG_COUNT = 17; // Number of key value pairs in config
 
 /*  The following enum has one-to-one mapping with
     otel_monitored_modules[] defined in .c file.
@@ -101,6 +102,7 @@ typedef struct {
     ngx_str_t   nginxModuleSegmentParameter;
     ngx_str_t   nginxModuleRequestHeaders;
     ngx_str_t   nginxModuleResponseHeaders;
+    ngx_str_t   nginxModuleOtelExporterOtlpHeaders;
 } ngx_http_opentelemetry_loc_conf_t;
 
 /*
