[ASK] Why sources file and Sinks console generated random log?

[kholisrag]

Hi all, need your insight and help, not sure below behavior is bug or not.

using this vector config :

    data_dir: /vector-data-dir
    sources:
      file_log:
        type: file
        include:
          - "/var/log/sidecar/*.log"
    sinks:
      stdout:
        type: console
        inputs:
          - file_log
        encoding:
          codec: json

somehow its generate random log like below,

looking at the trend, seem its demo_log like in ref: https://vector.dev/guides/level-up/transformation/

any method to prevent it? since I just want to tailing the file logs, not generate random logs,
Thanks!

[jszwedko]

Hi @kholisrag !

Those logs look a lot like the ones that come from the demo_logs source that is used in that guide. Are you sure they aren't from before or that they didn't accidentally end up in the files being tailed there?

[kholisrag]

I'm pretty sure, I use the above mentioned config

[jszwedko]

How are you running Vector? Maybe it is picking up another config file that you aren't aware of?

[kholisrag]

Hi @jszwedko, its a sidecar in k8s pods, the partial sidecar yaml manifest looks like below

...
    - command:
        - /bin/sh
        - -c
        - |-
          #!/bin/sh
          set -e
          mkdir -pv /etc/vector/
          cat > /etc/vector/vector.yaml <<EOF
          data_dir: /vector-data-dir
          sources:
            file_log:
              type: file
              include:
                - "/var/log/sidecar/*.log"
              remove_after_secs: 86400
          sinks:
            ## print to stdout with json codec / format
            stdout:
              type: console
              target: stderr
              inputs:
                - file_log
              encoding:
                codec: json
          EOF
          vector --config-dir /etc/vector/
      env:
        - name: VECTOR_SELF_NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        - name: VECTOR_SELF_POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: VECTOR_SELF_POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: VECTOR_LOG
          value: info
      image: timberio/vector:0.31.X-alpine
      imagePullPolicy: IfNotPresent
      name: tailing-logs
      resources:
        limits:
          cpu: 100m
          memory: 100Mi
        requests:
          cpu: 50m
          memory: 50Mi
      securityContext: {}
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
        - mountPath: /var/log/sidecar
          name: logs-volume
        - mountPath: /vector-data-dir
          name: tailing-log-db
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: kube-api-access-8jmg9
          readOnly: true
...