How to collect all QoS metrics?

[jmg011]

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
All QoS metrics

Describe alternatives you've considered
Custom Monitor

Additional context
N/A

[ruanruijuan]

Do the workload*.yaml files under https://github.com/NetApp/harvest/tree/b1a40efd764c6467c53a0231e191c958aa27f660/conf/zapiperf/cdot/9.8.0 help? Workloads counters are Qos counters.

[jmg011]

Thank you. I think that should help. I do not see the Network layer QoS breakdown counter. Is it part of some other counter?

[jmg011]

@ruanruijuan I have the QoS metrics getting collected. Is there a way to pull the QoS policy_group limits (Thresholds) as well?

[ruanruijuan]

@jmg011 I think you mean "qos-policy-group-get-iter.max-throughput"?

[jmg011]

@ruanruijuan

yes, that is correct. I built this yaml file and updated the custom.yaml for zapiperf. But the collection is failing with below error:

{"func":"goexit","line":"1581","source":"asm_amd64.s"}],"error":"api request rejected => Object "qos-policy-group-get-iter" was not found.","task":"counter","caller":"goharvest2/cmd/poller/collector/collector.go:340","time":"2021-10-25T17:09:16-07:00"}

========= yaml file ===========

name: Qoslimit
query: qos-policy-group-get-iter
object: qos

collect_only_labels: true

counters:
qos-policy-group-info:
- ^policy-group => policy-group
- ^^vserver => svm
- ^policy-group-class => policy-group-class
- ^max-throughput => max-throughput
- ^min-throughput => min-throughput
- ^num-workloads => num-workloads

plugins:
LabelAgent:
value_mapping: status status up 0

export_options:
include_all_labels: true

[cgrinds]

hi @jmg011 looks like you might have added your template to zapiperf instead of zapi? These counters are config related instead of performance zapi related so the template needs to be associated with the zapi collector.

I took your yaml file from above and made a few changes shown below. The pgid instance key was needed to disambiguate instances since the svm key alone isn't unique.

name: Qoslimit
query: qos-policy-group-get-iter
object: qos

collect_only_labels: true

counters:
  qos-policy-group-info:
    - ^policy-group => policy_group
    - ^^vserver => svm
    - ^^pgid
    - ^policy-group-class => policy_group_class
    - ^max-throughput => max_throughput
    - ^min-throughput => min_throughput
    - ^num-workloads => num_workloads

plugins:
  LabelAgent:
    value_mapping: status status up 0

export_options:
  include_all_labels: true

bin/harvest start u2 -f --objects Qoslimit

curl -s http://localhost:12995/metrics | grep ^qos
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",max_throughput="INF",min_throughput="0",num_workloads="0",policy_group_pgid="6269",policy_group="_Performance_Monitor_volumes",policy_group_class="system_defined",svm="umeng-aff300-05-06"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",max_throughput="INF",min_throughput="0",num_workloads="0",policy_group_pgid="12368",policy_group="node-1001",policy_group_class="load_control",svm="umeng-aff300-06"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",num_workloads="0",policy_group_pgid="9024",policy_group="test",policy_group_class="user_defined",svm="astra_devops_vs1",max_throughput="100IOPS",min_throughput="50IOPS"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",max_throughput="100IOPS",min_throughput="0",num_workloads="0",policy_group_pgid="2803",policy_group="test1",policy_group_class="user_defined",svm="astra_devops_vs1"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",policy_group="User-Best-Effort",policy_group_class="preset",svm="umeng-aff300-05-06",max_throughput="INF",min_throughput="0",num_workloads="67",policy_group_pgid="2"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",min_throughput="0",num_workloads="0",policy_group_pgid="14392",policy_group="_Auto-Control",policy_group_class="system_defined",svm="umeng-aff300-05-06",max_throughput="10000IOPS"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",policy_group_pgid="1",policy_group="_System-Best-Effort",policy_group_class="system_defined",svm="umeng-aff300-05-06",max_throughput="INF",min_throughput="0",num_workloads="3"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",max_throughput="INF",min_throughput="0",num_workloads="28",policy_group_pgid="3395",policy_group="_System-Work",policy_group_class="system_defined",svm="umeng-aff300-05-06"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",policy_group_class="user_defined",svm="umeng-aff300-05-svm1",max_throughput="10000000IOPS",min_throughput="0",num_workloads="5",policy_group_pgid="2825",policy_group="harvest_pg0"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",policy_group_pgid="14904",policy_group="harvest_pg1",policy_group_class="user_defined",svm="umeng-aff300-05-svm1",max_throughput="100000IOPS",min_throughput="10IOPS",num_workloads="0"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",policy_group_pgid="6848",policy_group="node-1000",policy_group_class="load_control",svm="umeng-aff300-05",max_throughput="INF",min_throughput="0",num_workloads="0"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",max_throughput="INF",min_throughput="0",num_workloads="0",policy_group_pgid="424",policy_group="Cache-User-Best-Effort",policy_group_class="preset",svm="umeng-aff300-05-06"} 0
qos_status{datacenter="dc-1",cluster="umeng-aff300-05-06",org="nb",site="munich",max_throughput="INF",min_throughput="0",num_workloads="2",policy_group_pgid="3",policy_group="_System-Background",policy_group_class="system_defined",svm="umeng-aff300-05-06"} 0

[jmg011]

Thank you @cgrinds. That was the problem. I had associated template with zapiperf. Configuring with Zapi worked as expected. I have the data I need. We can proceed with closing this issue.

[ruanruijuan]

That's good to know, @jmg011!
I'm converting this issue to "discussion", a new feature by Github. This type of issues are for inquiries, not necessarily for feature enhancement. However, we may want to consider to add QoS by default for the other customers later...