Skip to content

Prometheus exporter usage #5404

@yeplaa

Description

@yeplaa

I use currently the fluent component with fluent-plugin-prometheus plugin to deploy the prometheus exporter.
I generate my metrics by integrating the labels Kubernetes (retrieving from fluent-plugin-kubernetes plugin) Also i use the internal grep plugin to filter my messages. Bellow an extract of my configuration :

 <source>
      @type forward
      @log_level debug

      port 24224
      bind 0.0.0.0
    </source>

    
    
    # Cf https://docs.fluentd.org/filter/grep
    <filter {{ .Release.Namespace }}.kubernetes.var.log.containers.**>
      @type grep
      <regexp>
        key $.kubernetes.container_name
        pattern /^elasticsearch$/
      </regexp>
      
      <regexp>
        key $.kubernetes.labels.productname
        pattern /^elasticsearch$/
      </regexp>

      <regexp>
        key $.kubernetes.labels.domain
        pattern /^dbms$/
      </regexp>

      <regexp>
        # log beginning : [2018-07-04T13:34:26,108][ERROR]
        key $.message
        pattern /^\[[\d]{4}-[\d]{2}-[\d]{2}T[\d]{2}:[\d]{2}:[\d]{2},[\d].*\[ERROR\].*$/
      </regexp>

    </filter>


    <filter {{ .Release.Namespace }}.kubernetes.var.log.containers.**>
      @type prometheus
      <metric>
        name All_elasticsearch_counter
        type counter
        desc The total number of foo in message.
        <labels>
          container $.kubernetes.container_name
          namespace_name $.kubernetes.namespace_name
          pod_name $.kubernetes.pod_name
          domain $.kubernetes.labels.domain
          productname $.kubernetes.labels.productname
          setname $.kubernetes.labels.setname
        </labels>
      </metric> 
    </filter>


    # Exposition metrics promtheus / exporter
    <source>
      @type prometheus
      bind 0.0.0.0
      port 24231
      metrics_path /metrics
    </source>

Do you know if it's possible to perform the same feature with fluent-bit (and its Metrics plugins)?
Reading the doc I'm not sure

Thanks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions