Need help with failover template

[jmg011]

Is your feature request related to a problem? Please describe.
Need some help reviewing the log for the custom template that is not returning the expected output

Describe the solution you'd like
Template review and correction

Describe alternatives you've considered
Custom monitor

Additional context

{"level":"error","Poller":"Poller_Name","collector":"Zapi:Failover","caller":"goharvest2/cmd/collectors/zapi/collector/zapi.go:410","time":"2021-11-09T13:11:41-08:00","message":"skipped instance [datacenter-testcluster-01]: not found in cache"}

Created below template

name:                       Failover
query:                      cf-get-iter
object:                     failover

counters:
  storage-failover-info:
    - sfo-giveback-info
    - sfo-interconnect-info
    - ^^sfo-node-info
    - sfo-options-info
    - sfo-storage-info
    - sfo-takeover-info:
      - takeover-by-partner-not-possible-reason
      - takeover-by-partner-possible
      - takeover-enabled
      - takeover-failure-reason
      - takeover-module
      - takeover-of-partner-not-possible-reason
      - takeover-of-partner-possible
      - takeover-reason
      - takeover-state
      - time-since-takeover
      - time-until-takeover

plugins:
  LabelAgent:
    value_mapping: status status up 0
export_options:
  include_all_labels: true

[rahulguptajss]

@Hardikl Could you review this template

[cgrinds]

hi @jmg011 there are few mistakes in your posted yaml and I'm not 100% sure what you're trying to extract but here's a minimal example you can build from.

A few comments:

• use value_to_num instead of value_mapping since value_mapping will be removed in the next release of Harvest (see release notes).

• when writing new collector templates, it's often easier to run the one collector you are developing in the foreground to quickly iterate on your changes. e.g. bin/harvest start u2 --foreground --collectors Zapi --objects Failover It also make it easier to read the errors, redirect the logs, spot indentation errors, etc.

• the skipped instance message means that Harvest is unable to determine what an instance of the object in question is. Meaning, Harvest sends the ZAPI, parses the response, and uses the template to extract instance, metric, and labels from the response. In this case, no instances are found and Harvest logs a error letting you know that there's probably something wrong.

• Instance keys are defined in the template with the ^^ prefix, which you used for sfo-node-info. It's correct to use ^^, but it needs to be used with a leaf node of the XML response, not a parent. Here's an example of what I mean. Here's a snippet of the ZAPI response

      <storage-failover-info>
        <sfo-node-info>
          <node-related-info>
            <node>umeng-aff300-05</node>

With that response, the way we reference node would be like this. I'm also telling Harvest that node is part of the instanceKey that will be used to determine unique instances. In the minimal example, instance uniqueness is determined by node and nvram-id.

storage-failover-info:
    - sfo-giveback-info
    - sfo-node-info:
      - node-related-info:
        - ^^node      => node

Minimal Example

name:                       Failover
query:                      cf-get-iter
object:                     failover

counters:
  storage-failover-info:
    - sfo-giveback-info
    - sfo-node-info:
      - node-related-info:
        - ^^node      => node
        - ^^nvram-id  => nvram-id
        - ^node-state => node-state
    - sfo-takeover-info:
      - takeover-related-info:
        - ^takeover-by-partner-possible

plugins:
  LabelAgent:
    value_to_num:
      - nodes_status node-state connected connected `0`

export_options:
  include_all_labels: true

And curling to test this new collector template

curl -s 'http://localhost:12990/metrics' | grep ^failover
failover_nodes_status{datacenter="dc-1",cluster="umeng-aff300-05-06",sfo_takeover_related_by_partner_possible="true",node="umeng-aff300-05",node-state="connected",nvram-id="538006256"} 1
failover_nodes_status{datacenter="dc-1",cluster="umeng-aff300-05-06",node="umeng-aff300-06",node-state="connected",nvram-id="538005910",sfo_takeover_related_by_partner_possible="true"} 1

[jmg011]

@cgrinds Thank you so much! I had added node instance key but at a wrong attribute layer, and it was causing error so I had removed it. I think with your explanation and examples, I understand this better now. Thank you!

[cgrinds]

@jmg011 are you all set on this one? If so let's close the issue

[jmg011]

@cgrinds - I could make this work. We can close on this. I did not realize this was still open.