Rename tasks for XML reporting (#181)

elfiesmelfie · ayefimov-1 · commit 12990730f3f0 · 2025-03-10T15:17:40.000-04:00
* [ansible-lint] Update the config to ignore templates in task names * [telemetry_logging] Update task names for XML reporting * [common] Update task names for XML reporting * [common] Update the name for the pod tests to include the status_string and namespace that the pod is running in Add ceilometer_power_state to Metrics_verification test (#221) * Add test Ceilometer_power_state * Add the master_image to the zuul file * Add the master_image to the zuul file Add pod tests for Completed pods in the OpenStack namespace (#214) * [ci] Add logging tests: completed pods in openstack namespace * [ci] logging tests: update names for completed pods in openstack namespace * [ci] logging: update typo on pod name * [ci] Update pod list for completed status in openstack namespace This repo-setup-edpm-deployment-openstack-edpm-ipam doesn't need to be checked. --------- Co-authored-by: Alex Yefimov <126113326+ayefimov-1@users.noreply.github.com> [ci][logging] Add tests for services in the openstack namespace (#225) Co-authored-by: Alex Yefimov <126113326+ayefimov-1@users.noreply.github.com> [ci] Add endpoint tests to the logging job (#220) * [ci] Add endpoint tests to the logging job The test checks for the various openstack endpoints that we expect to exist Co-authored-by: "Alex Yefimov <ayefimov@redhat.com>" * [ci] Add vars/osp18_env to the logging playbook It contains the "openstack_cmd" var required for the endpoint tests * [common] Add documentation and default for openstack_cmd Fix kepler metrics tests (#224) Kepler metrics tests were added in [1] though the merged commit was not the correct one. This PR aplies changes from [2] that had more correct code of the tests. Fixes OSPRH-14029 [1] #193 [2] 0b5d856 Co-authored-by: Jaromír Wysoglad <jwysogla@redhat.com> Change required projects The projects should contain "github.com/" prefix. Fix rabbitmq test [zuul][graphing] Add the new dashboards to the test (#222) * Add the new dashboards to zuul job [ci] Add manifest tests to logging job (#228) Co-authored-by: Alex Yefimov <ayefimov@redhat.com> [telemetry_verify_metrics]: add openstack-network-exporter feature tests (#231) * feature_tests: add openstack-network-exporter feature tests Add tests for operator of openstack-network-exporter Signed-off-by: Aaron T. Smith <aasmith@rehdat.com> Co-authored-by: Aaron T. Smith <aasmith@redhat.com> Co-authored-by: Jaromír Wysoglad <smulumas@gmail.com> Add to irrelevant_files list for graphing, logging and autoscaling jobs Add to irrelevant_files list for graphing, logging and autoscaling jobs Add metrics verification tasks for power monitoring components (#193) Move container image patch to post_ctlplane hook Make autoscaling job voting Add tasks to show queried metrics [ci][logging] Move file tests to the compute nodes (#206) [ci] logging test: fix typo 0->O (#205) [ci][autoscaling] Add a note to the vars file for autoscaling (#215) The note explains why os-net-config is skipped Rename tasks for XML reporting (#181) * [ansible-lint] Update the config to ignore templates in task names * [telemetry_logging] Update task names for XML reporting * [common] Update task names for XML reporting * [common] Update the name for the pod tests to include the status_string and namespace that the pod is running in Add files to the zuul jobs Add files to the zuul jobs uncomment zuul.yaml Add ceilometer_power_state to Metrics_verification test (#221) * Add test Ceilometer_power_state * Add the master_image to the zuul file * Add the master_image to the zuul file Add pod tests for Completed pods in the OpenStack namespace (#214) * [ci] Add logging tests: completed pods in openstack namespace * [ci] logging tests: update names for completed pods in openstack namespace * [ci] logging: update typo on pod name * [ci] Update pod list for completed status in openstack namespace This repo-setup-edpm-deployment-openstack-edpm-ipam doesn't need to be checked. --------- Co-authored-by: Alex Yefimov <126113326+ayefimov-1@users.noreply.github.com> [ci][logging] Add tests for services in the openstack namespace (#225) Co-authored-by: Alex Yefimov <126113326+ayefimov-1@users.noreply.github.com> [ci] Add endpoint tests to the logging job (#220) * [ci] Add endpoint tests to the logging job The test checks for the various openstack endpoints that we expect to exist Co-authored-by: "Alex Yefimov <ayefimov@redhat.com>" * [ci] Add vars/osp18_env to the logging playbook It contains the "openstack_cmd" var required for the endpoint tests * [common] Add documentation and default for openstack_cmd Fix kepler metrics tests (#224) Kepler metrics tests were added in [1] though the merged commit was not the correct one. This PR aplies changes from [2] that had more correct code of the tests. Fixes OSPRH-14029 [1] #193 [2] 0b5d856 Co-authored-by: Jaromír Wysoglad <jwysogla@redhat.com> Change required projects The projects should contain "github.com/" prefix. Fix rabbitmq test [zuul][graphing] Add the new dashboards to the test (#222) * Add the new dashboards to zuul job Update .zuul.yaml [ci] Add manifest tests to logging job (#228) Co-authored-by: Alex Yefimov <ayefimov@redhat.com> [telemetry_verify_metrics]: add openstack-network-exporter feature tests (#231) * feature_tests: add openstack-network-exporter feature tests Add tests for operator of openstack-network-exporter Signed-off-by: Aaron T. Smith <aasmith@rehdat.com> Co-authored-by: Aaron T. Smith <aasmith@redhat.com> Co-authored-by: Jaromír Wysoglad <smulumas@gmail.com> Add to irrelevant_files list for graphing, logging and autoscaling jobs
diff --git a/.ansible-lint b/.ansible-lint
@@ -6,6 +6,7 @@ skip_list:
   - command-instead-of-module  # Using command rather than module
   - galaxy
   - meta-no-info  # meta/main.yml should contain relevant info
+  - name[template]  # Jinja templates should only be at the end of 'name'
   - risky-shell-pipe  # Shells that use pipes should set the pipefail option
   - unnamed-task  # All tasks should be named
   - var-naming[no-role-prefix]  # Variables names from within roles should use role_name_ as a prefix
diff --git a/.zuul.yaml b/.zuul.yaml
@@ -15,9 +15,9 @@
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: &required_projects
       - name: github.com/infrawatch/service-telemetry-operator
-      - name: openstack-k8s-operators/ci-framework
+      - name: github.com/openstack-k8s-operators/ci-framework
         override-checkout: main
-      - name: openstack-k8s-operators/install_yamls
+      - name: github.com/openstack-k8s-operators/install_yamls
         override-checkout: main
       - name: github.com/openstack-k8s-operators/dataplane-operator
         override-checkout: main
@@ -44,6 +44,15 @@
       - roles/test_sensubility/.*
       - roles/test_snmp_traps/.*
       - roles/test_verify_email/.*
+    files:
+      - roles/telemetry_autoscaling/.*
+      - .zuul.yaml
+      - ci/vars-functional-test.yml
+      - ci/run_autoscaling_osp18.yml
+      - ci/vars-use-master-containers.yml
+      - ci/use-master-containers.yml
+      - ci/patch-openstack-version.yaml
+      - scenarios/centos-9/multinode-ci.yml
 
 - job:
     name: functional-logging-tests-osp18
@@ -58,7 +67,15 @@
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: *required_projects
+<<<<<<< HEAD
     irrelevant-files: *irrelevant_files
+    files:
+      - roles/telemetry_logging/.*
+      - .zuul.yaml
+      - ci/vars-logging-test.yml
+      - ci/logging_tests_all.yml
+      - ci/logging_tests_computes.yml
+      - ci/logging_tests_controller.yml
 
 - job:
     name: functional-metric-verification-tests-osp18
@@ -72,10 +89,16 @@
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-autoscaling.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-metric-verification-test.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-power-monitoring.yml"
+        - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-use-master-containers.yml"
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: *required_projects
-    irrelevant-files: *irrelevant_files
+    irrelevant-files:  *irrelevant_files
+    files:
+      - roles/telemetry_verify_metrics/.*
+      - .zuul.yaml
+      - ci/vars-metric-verification-test.yml
+      - ci/run_verify_metrics_osp18.yml
 
 - job:
     name: feature-verification-tests-noop
@@ -97,12 +120,17 @@
       cifmw_extras:
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/ci-framework'].src_dir  }}/scenarios/centos-9/multinode-ci.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-autoscaling.yml"
+        - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-power-monitoring.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-graphing-test.yml"
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: *required_projects
     irrelevant-files: *irrelevant_files
-      
+    files:
+      - roles/telemetry_graphing/.*
+      - .zuul.yaml
+      - ci/vars-graphing-test.yml
+      - ci/run_graphing_test.yml
 
 - project:
     name: infrawatch/feature-verification-tests
diff --git a/ci/logging_tests_controller.yml b/ci/logging_tests_controller.yml
@@ -6,6 +6,8 @@
   environment:
     KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
     PATH: "{{ cifmw_path }}"
+  vars_files:
+    - vars/osp18_env.yml
   vars:
     common_pod_test_id: "RHOSO-12672"
     common_pod_status_str: "Running"
@@ -23,6 +25,30 @@
       - openshift
       - openstack-operators
       - openshift-logging
+    common_endpoint_test_id: "RHOSO-12682"
+    common_endpoint_list:
+      - [nova,compute,public]
+      - [nova,compute,internal]
+      - [placement,placement,public]
+      - [placement,placement,internal]
+      - [swift,object-store,public]
+      - [swift,object-store,internal]
+      - [cinderv3,volumev3,public]
+      - [cinderv3,volumev3,internal]
+      - [barbican,key-manager,public]
+      - [barbican,key-manager,internal]
+      - [keystone,identity,public]
+      - [keystone,identity,internal]
+      - [glance,image,public]
+      - [glance,image,internal]
+      - [neutron,network,public]
+      - [neutron,network,internal]
+
+    common_manifest_test_id: "RHOSO-12677"
+    common_manifest_list:
+      - "loki-operator 2"
+      - "loki-helm-operator 1"
+
     common_service_test_id: "RHOSO-12675"
     common_service_nspace: openshift-logging
     common_service_list:
@@ -143,7 +169,22 @@
     common_pod_nspace: openstack
     common_pod_status_str: "Completed"
     common_pod_list:
+      - bootstrap-edpm-deployment-openstack-edpm-ipam
+      - configure-network-edpm-deployment-openstack-edpm-ipam
+      - configure-os-edpm-deployment-openstack-edpm-ipam
+      - download-cache-edpm-deployment-openstack-edpm-ipam
+      - install-certs-edpm-deployment-openstack-edpm-ipam
+      - install-os-edpm-deployment-openstack-edpm-ipam
+      - libvirt-edpm-deployment-openstack-edpm-ipam
       - logging-edpm-deployment-openstack-edpm-ipam
+      - neutron-metadata-edpm-deployment-openstack-edpm-ipam
+      - nova-edpm-deployment-openstack-edpm-ipam
+      - ovn-edpm-deployment-openstack-edpm-ipam
+      - reboot-os-edpm-deployment-openstack-edpm-ipam
+      - run-os-edpm-deployment-openstack-edpm-ipam
+      - ssh-known-hosts-edpm-deployment
+      - telemetry-edpm-deployment-openstack-edpm-ipam
+      - validate-network-edpm-deployment-openstack-edpm-ipam
   tasks:
     - name: "Run pods completed tests"
       ansible.builtin.import_role:
@@ -157,13 +198,20 @@
     KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
     PATH: "{{ cifmw_path }}"
   vars:
+    common_service_test_id: "RHOSO-12749"
+    common_service_nspace: openstack
+    common_service_list:
+      - nova-internal
+      - nova-metadata-internal
+      - nova-novncproxy-cell1-public
+      - nova-public
     common_crd_test_id: "RHOSO-12670"
     common_crd_list:
       - alertingrules.loki.grafana.com
       - lokistacks.loki.grafana.com
       - recordingrules.loki.grafana.com
       - rulerconfigs.loki.grafana.com
   tasks:
-    - name: "Run the crds tests"
+    - name: "Run Services and CRD tests"
       ansible.builtin.import_role:
         name: common
diff --git a/roles/common/README.md b/roles/common/README.md
@@ -58,7 +58,8 @@ For endpoint_tests.yml tasks:
       - polarion ID number for each test.
     common_endpoint_list
       - list of endpoints to validate
-
+    openstack_cmd
+      - The command used to run openstack
 
 For project_tests.yml tasks:
 
@@ -156,6 +157,7 @@ can be set at the play level.
                 - ceilometer_agent_ipmi
                 - node_exporter
                 - kepler
+                - openstack_network_exporter
 
       - name: "Verify Endpoint"
         ansible.builtin.import_role:
@@ -175,7 +177,7 @@ can be set at the play level.
           common_project_list:
             - openshift-openstack-infra
             - openshift
-    
+
       - name: "Verify services"
         ansible.builtin.import_role:
           name: common
diff --git a/roles/common/defaults/main.yml b/roles/common/defaults/main.yml
@@ -0,0 +1,2 @@
+---
+openstack_cmd: "openstack"
diff --git a/roles/common/tasks/container_test.yml b/roles/common/tasks/container_test.yml
@@ -6,7 +6,9 @@
   changed_when: false
   register: container_status
 
-- name: Verify container status {{ common_container_test_id }}
+- name: |
+    TEST Verify {{ container_name }} container status
+    {{ common_container_test_id }}
   ansible.builtin.assert:
     that:
       - "'unhealthy' not in container_status.stdout"
diff --git a/roles/common/tasks/cr_tests.yml b/roles/common/tasks/cr_tests.yml
@@ -1,4 +1,7 @@
-- name: Verify that a CR exists {{ common_cr_test_id }}
+---
+- name: |
+    TEST Verify that the {{ item.kind }} {{ item.name }} CR exists
+    {{ common_cr_test_id }}
   ansible.builtin.command:
     cmd: |
       oc get {{ item.kind }} {{ item.name }}
diff --git a/roles/common/tasks/crd_tests.yml b/roles/common/tasks/crd_tests.yml
@@ -6,7 +6,9 @@
   changed_when: false
   register: output
 
-- name: Verify CRD is found "{{ common_crd_test_id }}"
+- name: |
+    TEST Verify CRD {{ item }} is found
+    {{ common_crd_test_id }}
   ansible.builtin.assert:
     that:
       - "'NotFound' not in output.stderr"
diff --git a/roles/common/tasks/endpoint_tests.yml b/roles/common/tasks/endpoint_tests.yml
@@ -6,9 +6,11 @@
   changed_when: false
   register: output
 
-- name: Check that endpoint exist {{ common_endpoint_test_id }}
+- name: |
+    TEST Check that {{ item[0] }} {{ item[2] }} endpoint exists
+    {{ common_endpoint_test_id }}
   ansible.builtin.assert:
     that:
       - output.stdout != ""
     success_msg: "The endpoint {{ item[0] }} is found."
-    fail_msg: "The endpoint for service {{ item[0] }} "
+    fail_msg: "The endpoint for service {{ item[0] }} on the {{ item[2] }} interface was not found"
diff --git a/roles/common/tasks/file_tests.yml b/roles/common/tasks/file_tests.yml
@@ -5,7 +5,7 @@
   register: fstats
 
 - name: |
-    Verify the file {{ item }} exists
+    TEST Verify the file {{ item }} exists
     {{ common_file_test_id }}
   ansible.builtin.assert:
     that:
diff --git a/roles/common/tasks/main.yml b/roles/common/tasks/main.yml
@@ -1,5 +1,5 @@
 ---
-- name: "Verify pod - {{ common_pod_test_id }}"
+- name: "Run pod tests"
   when:
     - common_pod_test_id is defined
     - common_pod_list is defined
diff --git a/roles/common/tasks/manifest_tests.yml b/roles/common/tasks/manifest_tests.yml
@@ -13,10 +13,12 @@
   register: pack_name
   changed_when: false
 
-- name: Get packagemanifest "{{ common_manifest_test_id }}"
+- name: |
+    TEST Get {{ item }} packagemanifest
+    {{ common_manifest_test_id }}
   ansible.builtin.shell:
     cmd: |
       oc get packagemanifests | grep "{{ pack_name.stdout }}" | wc -l
   register: num_found 
   changed_when: false
-  failed_when: num_expected.stdout != num_found.stdout
+  failed_when: num_expected.stdout != num_found.stdout
diff --git a/roles/common/tasks/pod_tests.yml b/roles/common/tasks/pod_tests.yml
@@ -11,7 +11,9 @@
         - podinstance.stdout_lines | length != 1
       changed_when: false
 
-    - name: Check pod {{ common_pod_test_id }}
+    - name: |
+        TEST Check {{ item }} pod is {{ common_pod_status_str }} in {{ common_pod_nspace }} namespace
+        {{ common_pod_test_id }}
       ansible.builtin.command:
         cmd: |
           oc get pod -n "{{ common_pod_nspace }}" "{{ podinstance.stdout }}"
diff --git a/roles/common/tasks/project_tests.yml b/roles/common/tasks/project_tests.yml
@@ -1,12 +1,14 @@
 ---
-- name: Verify project "{{ item }}"
+- name: Get the {{ item }} project
   ansible.builtin.shell:
     cmd: |
       oc get project "{{ item }}"
   changed_when: false
   register: output
 
-- name: Verify project is found "{{ common_project_test_id }}"
+- name: |
+   TEST Verify {{ item }} project exists
+   {{ common_project_test_id }}
   ansible.builtin.assert:
     that:
       - "'NotFound' not in output.stderr"
diff --git a/roles/common/tasks/service_tests.yml b/roles/common/tasks/service_tests.yml
@@ -7,7 +7,7 @@
   register: output
 
 - name: |
-    Verify {{ item }} service is running
+    TEST Verify {{ item }} service is running in {{ common_service_nspace }} namespace
     {{ common_service_test_id }}
   ansible.builtin.assert:
     that:
diff --git a/roles/common/tasks/subscription_tests.yml b/roles/common/tasks/subscription_tests.yml
@@ -6,7 +6,9 @@
   changed_when: false
   register: output
 
-- name: Check that subscription exist {{ common_subscription_test_id }}
+- name: |
+    TEST Check that {{ item }} subscription exists in {{ common_subscription_nspace }} namespace
+    {{ common_subscription_test_id }}
   ansible.builtin.assert:
     that:
       - "'NotFound' not in output.stderr"
diff --git a/roles/telemetry_graphing/files/dashboard-openstack-cloud.js b/roles/telemetry_graphing/files/dashboard-openstack-cloud.js
@@ -36,6 +36,9 @@ describe('OpenShift Console Dashboard Test', () => {
       { url: '/grafana-dashboard-openstack-rabbitmq', screenshot: 'openstack-rabbitmq' },
       { url: '/grafana-dashboard-openstack-node', screenshot: 'openstack-node' },
       { url: '/grafana-dashboard-openstack-vm', screenshot: 'openstack-vms' },
+      { url: '/grafana-dashboard-openstack-network-traffic', screenshot: 'openstack-network-traffic'},
+      { url: '/grafana-dashboard-openstack-kepler', screenshot: 'openstack-kepler'},
+      { url: '/grafana-dashboard-openstack-ceilometer-ipmi', screenshot: 'openstack-ceilometer-ipmi' }
     ];
     
 
diff --git a/roles/telemetry_logging/tasks/journal_tests.yml b/roles/telemetry_logging/tasks/journal_tests.yml
@@ -1,6 +1,6 @@
 ---
 - name: |
-    Get journals {{ item }}
+    TEST Get journals {{ item }}
     {{ journal_test_id }}
   become: true
   ansible.builtin.shell:
diff --git a/roles/telemetry_verify_metrics/README.md b/roles/telemetry_verify_metrics/README.md
@@ -27,13 +27,15 @@ Tests:
     - Use openstack observabilityclient to verify Ceilometer ipmi metrics are stored in Prometheus
 - Verify NodeExporter metrics are being exposed and stored
     - Use openstack observabilityclient to verify NodeExporter metrics are stored in Prometheus
+- Verfiy OpenstackNetworkExporter metrics are being exposed and stored
+    - Use openstack observabilityclient to verify OpenstackNetworkExporter metrics are stored in Prometheus
 - Verify Kepler metrics are being exposed and stored
     - Use openstack observabilityclient to verify Kepler metrics are stored in Prometheus
 
 Role Variables
 --------------
 openstack\_cmd - command to access openstack cli. For example: "oc rsh openstackclient openstack"
-telemetry\_verify\_metrics\_metric\_sources\_to\_test - List of sources to test. Current set of possible sources: ceilometer\_compute\_agent, ceilometer\_central\_agent, ceilometer\_ipmi_\_agent, node\_exporter, rabbitmq, kepler
+telemetry\_verify\_metrics\_metric\_sources\_to\_test - List of sources to test. Current set of possible sources: ceilometer\_compute\_agent, ceilometer\_central\_agent, ceilometer\_ipmi_\_agent, node\_exporter, rabbitmq, kepler, openstack_network_exporter
 
 Example Playbook
 ----------------
diff --git a/roles/telemetry_verify_metrics/defaults/main.yml b/roles/telemetry_verify_metrics/defaults/main.yml
@@ -5,5 +5,6 @@ telemetry_verify_metrics_metric_sources_to_test:
   - node_exporter
   - rabbitmq
   - kepler
+  - openstack_network_exporter
   # NOTE: IPMI metrics can only be verified when compute nodes are baremetal
   # - ceilometer_ipmi_agent
diff --git a/roles/telemetry_verify_metrics/tasks/main.yml b/roles/telemetry_verify_metrics/tasks/main.yml
@@ -15,10 +15,10 @@
       - kind: ceilometer
         name: ceilometer
         condition_type: Ready
-      - kind: rabbitmq
+      - kind: rabbitmqclusters
         name: rabbitmq
         condition_type: ReconcileSuccess
-      - kind: rabbitmq
+      - kind: rabbitmqclusters
         name: rabbitmq-cell1
         condition_type: ReconcileSuccess
   tags: precheck
@@ -56,7 +56,13 @@
 - name: Verify Kepler metrics are being exposed and stored
   ansible.builtin.include_tasks:
     file: verify_kepler_metrics.yml
-  tags: 
+  tags:
     - test
     - power_monitoring
   when: '"kepler" in telemetry_verify_metrics_metric_sources_to_test'
+
+- name: Verify OpenstackNetworkExporter metrics are being exposed and stored
+  ansible.builtin.include_tasks:
+    file: verify_openstack_network_exporter_metrics.yml
+  tags: test
+  when: '"openstack_network_exporter" in telemetry_verify_metrics_metric_sources_to_test'
diff --git a/roles/telemetry_verify_metrics/tasks/verify_ceilometer_compute_metrics.yml b/roles/telemetry_verify_metrics/tasks/verify_ceilometer_compute_metrics.yml
diff --git a/roles/telemetry_verify_metrics/tasks/verify_kepler_metrics.yml b/roles/telemetry_verify_metrics/tasks/verify_kepler_metrics.yml
diff --git a/roles/telemetry_verify_metrics/tasks/verify_openstack_network_exporter_metrics.yml b/roles/telemetry_verify_metrics/tasks/verify_openstack_network_exporter_metrics.yml
