How to capture weave-net logs from fluent-bit #5549

guna7750 · 2022-06-09T10:44:33Z

guna7750
Jun 9, 2022

I'm using fluent-bit for viewing weave-net logs, but the path for input file is not able to mount

https://fluentbit.io
Invalid configuration value at FILTER.Name
[2022/06/09 10:07:18] [ info] [engine] started (pid=1)
[2022/06/09 10:07:18] [ info] [storage] version=1.0.6, initializing...
[2022/06/09 10:07:18] [ info] [storage] in-memory
[2022/06/09 10:07:18] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/06/09 10:07:18] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2022/06/09 10:07:18] [ info] [sp] stream processor started
[2022/06/09 10:07:18] [ info] [input:tail:tail.0] inotify_fs_add(): inode=7345928 watch_fd=1 name=/mnt/resource/logs/liftlinx.log
[2022/06/09 10:07:18] [ info] [input:tail:tail.0] inotify_fs_add(): inode=7344616 watch_fd=2 name=/mnt/resource/logs/location.log
[2022/06/09 10:07:18] [ info] [input:tail:tail.0] inotify_fs_add(): inode=7344757 watch_fd=3 name=/mnt/resource/logs/order.log`

fluent-bit configuration

kind: DaemonSet

replicaCount -- Only applicable if kind=Deployment

replicaCount: 1

image:
repository: igs-wms-docker-stable-local.artifactory-na.honeywell.com/fluent-bit
pullPolicy: Always
tag: 1.6.10

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
create: true
annotations: {}
name:

initContainer:
enabled: true
repository: igs-wms-docker-stable-local.artifactory-na.honeywell.com/momentum-init
tag: 2.0.3
URLs: "elasticsearch-client.essential:9200" #make sure all the depencencies mentioned are seperated by ";"
RETRY_TIME: "30"
ELASTIC_AUTH_ENABLED: true
ELASTIC_SSL_ENABLED: true

envVar:
NAMESPACE_NAME: essential

Specify where fluent-bit can find logs

hostLogDir:
varLog: /mnt/resource/logs

elasticsearch:
auth:
enabled: true
ssl: true
external_logging:
enabled: true
customer: master
environment: ci
fluent_external_elasticsearch_host: 0.0.0.0
fluent_external_elasticsearch_port: 9200

rbac:
create: true

podSecurityContext:
{}

fsGroup: 2000

dnsConfig: {}

nameservers:

- 1.2.3.4

searches:

- ns1.svc.cluster-domain.example

- my.dns.search.suffix

options:

- name: ndots

value: "2"

- name: edns0

securityContext:
{}

capabilities:

drop:

- ALL

readOnlyRootFilesystem: true

runAsNonRoot: true

runAsUser: 1000

service:
type: ClusterIP
port: 2020
labels:
{}
annotations:
{}
# prometheus.io/path: "/api/v1/metrics/prometheus"
# prometheus.io/port: "2020"
# prometheus.io/scrape: "true"

serviceMonitor:
enabled: false

namespace: monitoring

interval: 10s

scrapeTimeout: 10s

selector:

prometheus: my-prometheus

livenessProbe:
httpGet:
path: /
port: http

readinessProbe:
httpGet:
path: /
port: http

resources:
{}

limits:

cpu: 100m

memory: 128Mi

requests:

cpu: 100m

memory: 128Mi

nodeSelector: {}

tolerations: []

affinity: {}

podAnnotations: {}

podLabels: {}

priorityClassName: ""

env:

name: FLUENT_ELASTICSEARCH_HOST
value: elasticsearch-client.essential
name: FLUENT_ELASTICSEARCH_PORT
value: "9200"
name: ELASTIC_USERNAME
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
name: ELASTIC_PASSWORD
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password

envFrom: []

extraPorts: []

- port: 5170

containerPort: 5170

protocol: TCP

name: tcp

extraVolumes:

hostPath:
path: /var/log/audit/kube-apiserver-audit.log
type: FileOrCreate
name: k8seventlog
hostPath:
path: /var/log/pods/*.log
type: FileOrCreate
name: weavenetlog

extraVolumeMounts:

mountPath: /var/log/audit/kube-apiserver-audit.log
name: k8seventlog
readOnly: true
mountPath: /var/log/pods/*.log
name: weavenetlog
readOnly: true

updateStrategy: {}

type: RollingUpdate

rollingUpdate:

maxUnavailable: 1

existingConfigMap: ""

config:
service: |
[SERVICE]
Flush 1
Daemon Off
Log_Level info
Parsers_File custom_parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020

inputs: |
[INPUT]
Name tail
Tag weslogs.*
Path ${HOST_LOG_LOCATION}/*.log
DB ${HOST_LOG_LOCATION}/flb_services.db
Mem_Buf_Limit 16MB
Key message
Multiline On
Parser_Firstline logmsg_multiline
Skip_Long_Lines On
Refresh_Interval 10

[INPUT]
    Name              tail
    Tag               weavenetlogs.*
    Path              /var/log/pods/**/weave/0.log
    DB                ${HOST_LOG_LOCATION}/flb_weavenet_services.db
    Mem_Buf_Limit     16MB
    Key               message
    Multiline         On
    Parser_Firstline  logmsg_multiline
    Skip_Long_Lines   On
    Refresh_Interval  10

[INPUT]
    Name              tail
    Tag               k8s_event.*
    Path              /var/log/audit/kube-apiserver-audit.log
    DB                ${HOST_LOG_LOCATION}/flb_k8s_event.db
    Mem_Buf_Limit     8MB
    Key               message
    Multiline         On
    Parser_Firstline  k8s_event
    Skip_Long_Lines   On
    Refresh_Interval  10

filters: |
[FILTER]
Name weslogsparser
Match weslogs.*
Parser logmsg_multiline
Merge_Log On
Merge_Log_Key log_processed

[FILTER]
    Name                weavenetlogsparser
    Match               weavenetlogs.*
    Parser              logmsg_multiline
    Merge_Log           On
    Merge_Log_Key       log_processed

[FILTER]
    Name                k8seventfilter
    Match               k8s_event.*
    Parser              k8s_event
    Merge_Log           On
    Merge_Log_Key       log_processed

outputs: |
[OUTPUT]
Name es
Match weslogs.*
Host ${FLUENT_ELASTICSEARCH_HOST}
Port ${FLUENT_ELASTICSEARCH_PORT}
HTTP_User ${ELASTIC_USERNAME}
HTTP_Passwd ${ELASTIC_PASSWORD}
Logstash_Format On
Logstash_Prefix fluent-bit
Replace_Dots On
Retry_Limit False
tls Off
tls.verify Off

[OUTPUT]
    Name            es
    Match           weavenetlogs.*
    Host            ${FLUENT_ELASTICSEARCH_HOST}
    Port            ${FLUENT_ELASTICSEARCH_PORT}
    HTTP_User       ${ELASTIC_USERNAME}
    HTTP_Passwd     ${ELASTIC_PASSWORD}
    Logstash_Format On
    Logstash_Prefix fluent-bit
    Replace_Dots    On
    Retry_Limit     False
    tls             Off
    tls.verify      Off

[OUTPUT]
    Name            es
    Match           k8s_event.*
    Host            ${FLUENT_ELASTICSEARCH_HOST}
    Port            ${FLUENT_ELASTICSEARCH_PORT}
    HTTP_User       ${ELASTIC_USERNAME}
    HTTP_Passwd     ${ELASTIC_PASSWORD}
    Logstash_Format On
    Logstash_Prefix k8-event
    Replace_Dots    On
    Retry_Limit     False
    tls             On
    tls.verify      Off

outputs_external: |
[OUTPUT]
Name es
Match weslogs.*
Host ${FLUENT_EXTERNAL_ELASTICSEARCH_HOST}
Port ${FLUENT_EXTERNAL_ELASTICSEARCH_PORT}
HTTP_User ${EXTERNAL_ELASTIC_USERNAME}
HTTP_Passwd ${EXTERNAL_ELASTIC_PASSWORD}
Logstash_Format On
Logstash_Prefix ${FLUENT_INDEX_PREFIX}
Replace_Dots On
Retry_Limit False
tls On
tls.verify Off

[OUTPUT]
    Name            es
    Match           weavenetlogs.*
    Host            ${FLUENT_EXTERNAL_ELASTICSEARCH_HOST}
    Port            ${FLUENT_EXTERNAL_ELASTICSEARCH_PORT}
    HTTP_User       ${EXTERNAL_ELASTIC_USERNAME}
    HTTP_Passwd     ${EXTERNAL_ELASTIC_PASSWORD}
    Logstash_Format On
    Logstash_Prefix ${FLUENT_INDEX_PREFIX}
    Replace_Dots    On
    Retry_Limit     False
    tls             On
    tls.verify      Off

outputs_ssl: |
[OUTPUT]
Name es
Match weslogs.*
Host ${FLUENT_ELASTICSEARCH_HOST}
Port ${FLUENT_ELASTICSEARCH_PORT}
HTTP_User ${ELASTIC_USERNAME}
HTTP_Passwd ${ELASTIC_PASSWORD}
Logstash_Format On
Logstash_Prefix fluent-bit
Replace_Dots On
Retry_Limit False
tls On
tls.verify Off

[OUTPUT]
    Name            es
    Match           weavenetlogs.*
    Host            ${FLUENT_ELASTICSEARCH_HOST}
    Port            ${FLUENT_ELASTICSEARCH_PORT}
    HTTP_User       ${ELASTIC_USERNAME}
    HTTP_Passwd     ${ELASTIC_PASSWORD}
    Logstash_Format On
    Logstash_Prefix fluent-bit
    Replace_Dots    On
    Retry_Limit     False
    tls             On
    tls.verify      Off

[OUTPUT]
    Name            es
    Match           k8s_event.*
    Host            ${FLUENT_ELASTICSEARCH_HOST}
    Port            ${FLUENT_ELASTICSEARCH_PORT}
    HTTP_User       ${ELASTIC_USERNAME}
    HTTP_Passwd     ${ELASTIC_PASSWORD}
    Logstash_Format On
    Logstash_Prefix k8-event
    Replace_Dots    On
    Retry_Limit     False
    tls             On
    tls.verify      Off

[PARSER]
    Name   k8s_event
    Format json

`

agup006 · 2022-06-23T21:46:31Z

agup006
Jun 23, 2022
Maintainer

Hi @guna7750 the error you are running into is based on the Filter not existing. The following Filter weslogsparser does not exist

[FILTER]
Name weslogsparser
Match weslogs.*
Parser logmsg_multiline
Merge_Log On
Merge_Log_Key log_processed

1 reply

hari819 Jul 16, 2022

please can you help me on how to mount the certificates for fluent-bit and enable TLS , i tried extraVolumes and extraVolumeMounts , with TLS on , but in the end i am getting a weird error "[tls] crt_file '/etc/fluent-certs/tls.crt' 336445449: error:0909006C:PEM routines:get_name:no start line" , but the cert file is having "begin certificate" and "end certifiate" as expected

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to capture weave-net logs from fluent-bit #5549

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

How to capture weave-net logs from fluent-bit #5549

Uh oh!

Uh oh!

guna7750 Jun 9, 2022

fluent-bit configuration

replicaCount -- Only applicable if kind=Deployment

Specify where fluent-bit can find logs

fsGroup: 2000

nameservers:

- 1.2.3.4

searches:

- ns1.svc.cluster-domain.example

- my.dns.search.suffix

options:

- name: ndots

value: "2"

- name: edns0

capabilities:

drop:

- ALL

readOnlyRootFilesystem: true

runAsNonRoot: true

runAsUser: 1000

namespace: monitoring

interval: 10s

scrapeTimeout: 10s

selector:

prometheus: my-prometheus

limits:

cpu: 100m

memory: 128Mi

requests:

cpu: 100m

memory: 128Mi

- port: 5170

containerPort: 5170

protocol: TCP

name: tcp

type: RollingUpdate

rollingUpdate:

maxUnavailable: 1

Replies: 1 comment · 1 reply

Uh oh!

agup006 Jun 23, 2022 Maintainer

Uh oh!

hari819 Jul 16, 2022

guna7750
Jun 9, 2022

Replies: 1 comment 1 reply

agup006
Jun 23, 2022
Maintainer