Skip to content

[kafka.segmentio] not creating topics #185

@EnochTheWise

Description

@EnochTheWise

vflow.conf

pid-file: vflow.pid
mq-name: kafka.segmentio
ipfix-enabled: true
ipfix-port: 4739
ipfix-workers: 10
sflow-enabled: true
sflow-port: 6343
sflow-workers: 10
netflow5-enabled: true
netflow5-port: 9996
netflow5-workers: 10
netflow9-enabled: true
netflow9-port: 9995
netflow9-workers: 10

mq.conf

brokers:
    - kafka-etl-01.wz.local:9092
    - kafka-etl-02.wz.local:9092
    - kafka-etl-03.wz.local:9092
$ ./vflow -config vflow.conf -mqueue-conf mq.conf
[vflow] 2023/02/16 10:15:30 Welcome to vFlow v.0.9.0 Apache License 2.0
[vflow] 2023/02/16 10:15:30 Copyright (C) 2018 Verizon. github.com/VerizonDigital/vflow
[vflow] 2023/02/16 10:15:30 ipfix is running (UDP: listening on [::]:4739 workers#: 10)
[vflow] 2023/02/16 10:15:30 starting prometheus http server ...
[vflow] 2023/02/16 10:15:30 netflow v5 is running (UDP: listening on [::]:9996 workers#: 10)
[vflow] 2023/02/16 10:15:30 sFlow is running (UDP: listening on [::]:6343 workers#: 10)
[vflow] 2023/02/16 10:15:30 netflow v9 is running (UDP: listening on [::]:9995 workers#: 10)
[vflow] 2023/02/16 10:15:30 start producer: Kafka, brokers: [kafka-etl-01.wz.local:9092 kafka-etl-02.wz.local:9092 kafka-etl-03.wz.local:9092], topic: vflow.netflow5
[vflow] 2023/02/16 10:15:30 start producer: Kafka, brokers: [kafka-etl-01.wz.local:9092 kafka-etl-02.wz.local:9092 kafka-etl-03.wz.local:9092], topic: vflow.sflow
[vflow] 2023/02/16 10:15:30 start producer: Kafka, brokers: [kafka-etl-01.wz.local:9092 kafka-etl-02.wz.local:9092 kafka-etl-03.wz.local:9092], topic: vflow.ipfix
[vflow] 2023/02/16 10:15:30 start producer: Kafka, brokers: [kafka-etl-01.wz.local:9092 kafka-etl-02.wz.local:9092 kafka-etl-03.wz.local:9092], topic: vflow.netflow9
[vflow] 2023/02/16 10:15:30 ipfix RPC enabled
[vflow] 2023/02/16 10:15:46 error writing to kafka: [3] Unknown Topic Or Partition: the request is for a topic or partition that does not exist on this broker
[vflow] 2023/02/16 10:16:00 error writing to kafka: [3] Unknown Topic Or Partition: the request is for a topic or partition that does not exist on this broker
[vflow] 2023/02/16 10:16:13 error writing to kafka: [3] Unknown Topic Or Partition: the request is for a topic or partition that does not exist on this broker
[vflow] 2023/02/16 10:16:27 error writing to kafka: [3] Unknown Topic Or Partition: the request is for a topic or partition that does not exist on this broker
[vflow] 2023/02/16 10:16:41 error writing to kafka: [3] Unknown Topic Or Partition: the request is for a topic or partition that does not exist on this broker

PS Autocreate topics is ON

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions