Send traces from opentelemetry source to kafka sink

[otani88]

Hi everyone. We want to send traces from opentelemetry source to kafka sink with encoding protobuf. Right now it looks like:

sources:
    otel_traces:
      type: opentelemetry
      grpc:
        address: 0.0.0.0:4317
      http:
        address: 0.0.0.0:4318

sinks:
    kafka_traces:
      type: kafka
      inputs:
        - otel_traces.traces
      bootstrap_servers: <some_server>
      topic: traces-topic
      encoding:
        codec: protobuf
        protobuf:
          message_type: "opentelemetry.proto.trace.v1.Span"
          desc_file: "/etc/vector/opentelemetry.desc"
      compression: lz4

But I get error like:
2024-08-06T17:20:07.620634Z ERROR vector::cli: Configuration error. error=Data type mismatch between otel_traces.traces (Trace) and kafka_traces (Log)

Are there some ways to send traces to Kafka? Consumer of kafka topic is quickwit which expected protobuf format for work with OTEL traces.

[jszwedko]

It looks like the kafka sink can't currently accept traces, although it should be able to.

vector/src/sinks/kafka/config.rs

Lines 278 to 279 in aab836b

	Input::new(self.encoding.config().input_type() & (DataType::Log | DataType::Metric))
	.with_schema_requirement(requirements)

That just needs to have the & (DataType::Log | DataType::Metric) bit removed and then testing to make sure it works with traces.