Add to irrelevant_files list for graphing, logging

and autoscaling jobs

Add to irrelevant_files list for graphing, logging
and autoscaling jobs

Add metrics verification tasks for power monitoring components (#193)

Move container image patch to post_ctlplane hook

Make autoscaling job voting

Add tasks to show queried metrics

[ci][logging] Move file tests to the compute nodes (#206)

[ci] logging test: fix typo 0->O (#205)

[ci][autoscaling] Add a note to the vars file for autoscaling (#215)

The note explains why os-net-config is skipped

Rename tasks for XML reporting (#181)

* [ansible-lint] Update the config to ignore templates in task names
* [telemetry_logging] Update task names for XML reporting
* [common] Update task names for XML reporting
* [common] Update the name for the pod tests to include the
   status_string and namespace that the pod is running in

Add files to the zuul jobs

Add files to the zuul jobs

uncomment zuul.yaml

Add ceilometer_power_state to Metrics_verification test (#221)

* Add test Ceilometer_power_state

* Add the master_image to the zuul file

* Add the master_image to the zuul file

Add pod tests for Completed pods in the OpenStack namespace (#214)

* [ci] Add logging tests: completed pods in openstack namespace

* [ci] logging tests: update names for completed pods in openstack namespace

* [ci] logging: update typo on pod name

* [ci] Update pod list for completed status in openstack namespace

This repo-setup-edpm-deployment-openstack-edpm-ipam doesn't need to be
checked.

---------

Co-authored-by: Alex Yefimov <[email protected]>

[ci][logging] Add tests for services in the openstack namespace (#225)

Co-authored-by: Alex Yefimov <[email protected]>

[ci] Add endpoint tests to the logging job (#220)

* [ci] Add endpoint tests to the logging job

   The test checks for the various openstack endpoints that we expect to
exist

Co-authored-by: "Alex Yefimov <[email protected]>"

* [ci] Add vars/osp18_env to the logging playbook
   It contains the "openstack_cmd" var required for the endpoint tests

* [common] Add documentation and default for openstack_cmd

Fix kepler metrics tests (#224)

Kepler metrics tests were added in [1] though the merged commit
was not the correct one. This PR aplies changes from [2] that had
more correct code of the tests.

Fixes OSPRH-14029

[1] #193
[2] 0b5d856

Co-authored-by: Jaromír Wysoglad <[email protected]>

Change required projects

The projects should contain "github.com/" prefix.

Fix rabbitmq test

[zuul][graphing] Add the new dashboards to the test (#222)

* Add the new dashboards to zuul job

Update .zuul.yaml

[ci] Add manifest tests to logging job (#228)

Co-authored-by: Alex Yefimov <[email protected]>

[telemetry_verify_metrics]: add openstack-network-exporter feature tests (#231)

* feature_tests: add openstack-network-exporter feature tests

Add tests for operator of openstack-network-exporter

Signed-off-by: Aaron T. Smith <[email protected]>
Co-authored-by: Aaron T. Smith <[email protected]>
Co-authored-by: Jaromír Wysoglad <[email protected]>

Add to irrelevant_files list for graphing, logging
and autoscaling jobs

Author: ayefimov-1

.ansible-lint

@@ -6,6 +6,7 @@ skip_list:
   - command-instead-of-module  # Using command rather than module
   - galaxy
   - meta-no-info  # meta/main.yml should contain relevant info
+  - name[template]  # Jinja templates should only be at the end of 'name'
   - risky-shell-pipe  # Shells that use pipes should set the pipefail option
   - unnamed-task  # All tasks should be named
   - var-naming[no-role-prefix]  # Variables names from within roles should use role_name_ as a prefix

.zuul.yaml

@@ -10,13 +10,14 @@
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/ci-framework'].src_dir  }}/scenarios/centos-9/multinode-ci.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-autoscaling.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-functional-test.yml"
+        - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-use-master-containers.yml"
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: &required_projects
       - name: github.com/infrawatch/service-telemetry-operator
-      - name: openstack-k8s-operators/ci-framework
+      - name: github.com/openstack-k8s-operators/ci-framework
         override-checkout: main
-      - name: openstack-k8s-operators/install_yamls
+      - name: github.com/openstack-k8s-operators/install_yamls
         override-checkout: main
       - name: github.com/openstack-k8s-operators/dataplane-operator
         override-checkout: main
@@ -43,6 +44,15 @@
       - roles/test_sensubility/.*
       - roles/test_snmp_traps/.*
       - roles/test_verify_email/.*
+    files:
+      - roles/telemetry_autoscaling/.*
+      - .zuul.yaml
+      - ci/vars-functional-test.yml
+      - ci/run_autoscaling_osp18.yml
+      - ci/vars-use-master-containers.yml
+      - ci/use-master-containers.yml
+      - ci/patch-openstack-version.yaml
+      - scenarios/centos-9/multinode-ci.yml
 
 - job:
     name: functional-logging-tests-osp18
@@ -57,7 +67,15 @@
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: *required_projects
+<<<<<<< HEAD
     irrelevant-files: *irrelevant_files
+    files:
+      - roles/telemetry_logging/.*
+      - .zuul.yaml
+      - ci/vars-logging-test.yml
+      - ci/logging_tests_all.yml
+      - ci/logging_tests_computes.yml
+      - ci/logging_tests_controller.yml
 
 - job:
     name: functional-metric-verification-tests-osp18
@@ -70,10 +88,17 @@
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/ci-framework'].src_dir  }}/scenarios/centos-9/multinode-ci.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-autoscaling.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-metric-verification-test.yml"
+        - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-power-monitoring.yml"
+        - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-use-master-containers.yml"
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: *required_projects
-    irrelevant-files: *irrelevant_files
+    irrelevant-files:  *irrelevant_files
+    files:
+      - roles/telemetry_verify_metrics/.*
+      - .zuul.yaml
+      - ci/vars-metric-verification-test.yml
+      - ci/run_verify_metrics_osp18.yml
 
 - job:
     name: feature-verification-tests-noop
@@ -95,12 +120,17 @@
       cifmw_extras:
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/ci-framework'].src_dir  }}/scenarios/centos-9/multinode-ci.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-autoscaling.yml"
+        - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/openstack-k8s-operators/telemetry-operator'].src_dir }}/ci/vars-power-monitoring.yml"
         - "@{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/vars-graphing-test.yml"
     roles:
       - zuul: github.com/openstack-k8s-operators/ci-framework
     required-projects: *required_projects
     irrelevant-files: *irrelevant_files
-      
+    files:
+      - roles/telemetry_graphing/.*
+      - .zuul.yaml
+      - ci/vars-graphing-test.yml
+      - ci/run_graphing_test.yml
 
 - project:
     name: infrawatch/feature-verification-tests
@@ -111,7 +141,7 @@
             override-checkout: main
             irrelevant-files: *irrelevant_files
         - functional-tests-on-osp18:
-            voting: false
+            voting: true
         - functional-logging-tests-osp18
         - functional-graphing-tests-osp18
         - functional-metric-verification-tests-osp18

ci/logging_tests_computes.yml

@@ -11,7 +11,14 @@
     journal_list:
       - ceilometer_agent_compute
       - nova_compute
+    common_file_test_id: "RHOSO-12754"
+    common_file_list:
+      - /etc/rsyslog.d/10-telemetry.conf
   tasks:
     - name: "Run Telemetry Logging tests"
       ansible.builtin.import_role:
         name: telemetry_logging
+
+    - name: "Run the tests from the common role"
+      ansible.builtin.import_role:
+        name: common

ci/logging_tests_controller.yml

@@ -6,8 +6,10 @@
   environment:
     KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
     PATH: "{{ cifmw_path }}"
+  vars_files:
+    - vars/osp18_env.yml
   vars:
-    common_pod_test_id: "RHOS0-12672"
+    common_pod_test_id: "RHOSO-12672"
     common_pod_status_str: "Running"
     common_pod_nspace: openstack-operators
     common_pod_list:
@@ -23,6 +25,30 @@
       - openshift
       - openstack-operators
       - openshift-logging
+    common_endpoint_test_id: "RHOSO-12682"
+    common_endpoint_list:
+      - [nova,compute,public]
+      - [nova,compute,internal]
+      - [placement,placement,public]
+      - [placement,placement,internal]
+      - [swift,object-store,public]
+      - [swift,object-store,internal]
+      - [cinderv3,volumev3,public]
+      - [cinderv3,volumev3,internal]
+      - [barbican,key-manager,public]
+      - [barbican,key-manager,internal]
+      - [keystone,identity,public]
+      - [keystone,identity,internal]
+      - [glance,image,public]
+      - [glance,image,internal]
+      - [neutron,network,public]
+      - [neutron,network,internal]
+
+    common_manifest_test_id: "RHOSO-12677"
+    common_manifest_list:
+      - "loki-operator 2"
+      - "loki-helm-operator 1"
+
     common_service_test_id: "RHOSO-12675"
     common_service_nspace: openshift-logging
     common_service_list:
@@ -42,9 +68,6 @@
       - logging-loki-query-frontend-grpc
       - logging-loki-query-frontend-http
       - openstack-logging
-    common_file_test_id: "RHOSO-12754"
-    common_file_list:
-      - /etc/rsyslog.d/10-telemetry.conf
   tasks:
     - name: "Verify logging infrastructure components"
       ansible.builtin.import_role:
@@ -146,7 +169,22 @@
     common_pod_nspace: openstack
     common_pod_status_str: "Completed"
     common_pod_list:
+      - bootstrap-edpm-deployment-openstack-edpm-ipam
+      - configure-network-edpm-deployment-openstack-edpm-ipam
+      - configure-os-edpm-deployment-openstack-edpm-ipam
+      - download-cache-edpm-deployment-openstack-edpm-ipam
+      - install-certs-edpm-deployment-openstack-edpm-ipam
+      - install-os-edpm-deployment-openstack-edpm-ipam
+      - libvirt-edpm-deployment-openstack-edpm-ipam
       - logging-edpm-deployment-openstack-edpm-ipam
+      - neutron-metadata-edpm-deployment-openstack-edpm-ipam
+      - nova-edpm-deployment-openstack-edpm-ipam
+      - ovn-edpm-deployment-openstack-edpm-ipam
+      - reboot-os-edpm-deployment-openstack-edpm-ipam
+      - run-os-edpm-deployment-openstack-edpm-ipam
+      - ssh-known-hosts-edpm-deployment
+      - telemetry-edpm-deployment-openstack-edpm-ipam
+      - validate-network-edpm-deployment-openstack-edpm-ipam
   tasks:
     - name: "Run pods completed tests"
       ansible.builtin.import_role:
@@ -160,13 +198,20 @@
     KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
     PATH: "{{ cifmw_path }}"
   vars:
+    common_service_test_id: "RHOSO-12749"
+    common_service_nspace: openstack
+    common_service_list:
+      - nova-internal
+      - nova-metadata-internal
+      - nova-novncproxy-cell1-public
+      - nova-public
     common_crd_test_id: "RHOSO-12670"
     common_crd_list:
       - alertingrules.loki.grafana.com
       - lokistacks.loki.grafana.com
       - recordingrules.loki.grafana.com
       - rulerconfigs.loki.grafana.com
   tasks:
-    - name: "Run the crds tests"
+    - name: "Run Services and CRD tests"
       ansible.builtin.import_role:
         name: common

ci/run_autoscaling_osp18.yml

@@ -20,26 +20,6 @@
           tags:
             - setup
 
-        - name: Patch the openstackversions to use the master containers for aodh and heat
-          ansible.builtin.shell:
-            cmd: |
-              oc patch openstackversions controlplane --type merge --patch-file ci/patch-openstack-versions.yaml
-            chdir: "{{ fvt_dir }}"
-          when: "{{ patch_openstackversions | bool }}"
-          tags:
-            - setup
-
-        - name: Redeploy the dataplane
-          ansible.builtin.shell:
-            cmd: |
-              oc get osdpd edpm-deployment -oyaml > /tmp/osdpd.yaml
-              oc delete osdpd edpm-deployment
-              sleep 10
-              oc apply -f /tmp/osdpd.yaml
-          when: "{{ patch_openstackversions | bool }}"
-          tags:
-            - setup
-
         - name: Patch observabilityclient into openstackclient
           ansible.builtin.shell:
             cmd: |
@@ -50,50 +30,7 @@
           tags:
             - setup
 
-        - name: Wait until the oscp is resolved the changes to continue
-          ansible.builtin.shell:
-            cmd: |
-              oc get oscp | grep "Setup complete"
-          retries: 24
-          timeout: 5
-          until: output.stdout_lines | length == 1
-          register: output
-          when: "{{ patch_openstackversions | bool }}"
-          tags:
-            - setup
-
-        - name: Wait until the osdpd is redeployed to continue
-          ansible.builtin.shell:
-            cmd: |
-              oc get osdpd | grep "Setup complete"
-          retries: 60
-          delay: 45
-          until: output.stdout_lines | length == 1
-          register: output
-          when: "{{ patch_openstackversions | bool }}"
-          tags:
-            - setup
-
         - name: "Run Telemetry Autoscaling tests"
           ansible.builtin.import_role:
             name: telemetry_autoscaling
           ignore_errors: true
-
-      always:
-        - name: Revert the version update
-          ansible.builtin.shell:
-            cmd: |
-              oc patch  openstackversions controlplane  --type json -p='[{"op": "replace", "path": "/spec/customContainerImages", "value": {} }]'
-          when: "{{ patch_openstackversions | bool }}"
-
-        - name: Redeploy the dataplane
-          ansible.builtin.shell:
-            cmd: |
-              oc get osdpd edpm-deployment -oyaml > /tmp/osdpd.yaml
-              oc delete osdpd edpm-deployment
-              sleep 10
-              oc apply -f /tmp/osdpd.yaml
-          when: "{{ patch_openstackversions | bool }}"
-          tags:
-            - setup
-

ci/use-master-containers.yml

@@ -0,0 +1,18 @@
+- name: Run telemetry autoscaling tests on osp18
+  hosts:  "{{ cifmw_target_hook_host | default('localhost')  }}"
+  gather_facts: true
+  environment:
+    KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
+    PATH: "{{ cifmw_path }}"
+  vars_files:
+    - vars/common.yml
+    - vars/osp18_env.yml
+  tasks:
+    - name: Patch the openstackversions to use the master containers for aodh, heat and ceilometer
+      ansible.builtin.shell:
+        cmd: |
+          oc patch openstackversions controlplane --type merge --patch-file ci/patch-openstack-versions.yaml
+        chdir: "{{ fvt_dir }}"
+      tags:
+        - setup
+

ci/vars-functional-test.yml

@@ -1,5 +1,9 @@
 ---
-# temp: skip os-net-setup
+# This is added so we skip the os_net_setup role, which would normally be used
+# to create a network.
+# A network gets created by `make edpm_deploy_instance` during the autoscaling
+# tests.
+# skip os-net-setup
 cifmw_os_net_setup_config: []
 post_deploy_00_run_functional_test:
   source: "{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/run_autoscaling_osp18.yml"

ci/vars-use-master-containers.yml

@@ -0,0 +1,4 @@
+---
+post_ctlplane_deploy_99_modify_openstackversions:
+  source: "{{ ansible_user_dir }}/{{ zuul.projects['github.com/infrawatch/feature-verification-tests'].src_dir }}/ci/use-master-containers.yml"
+  type: playbook

roles/common/README.md

@@ -58,7 +58,8 @@ For endpoint_tests.yml tasks:
       - polarion ID number for each test.
     common_endpoint_list
       - list of endpoints to validate
-
+    openstack_cmd
+      - The command used to run openstack
 
 For project_tests.yml tasks:
 
@@ -155,6 +156,8 @@ can be set at the play level.
                 - ceilometer_agent_compute
                 - ceilometer_agent_ipmi
                 - node_exporter
+                - kepler
+                - openstack_network_exporter
 
       - name: "Verify Endpoint"
         ansible.builtin.import_role:
@@ -174,7 +177,7 @@ can be set at the play level.
           common_project_list:
             - openshift-openstack-infra
             - openshift
-    
+
       - name: "Verify services"
         ansible.builtin.import_role:
           name: common

roles/common/defaults/main.yml

@@ -0,0 +1,2 @@
+---
+openstack_cmd: "openstack"