|
| 1 | +# Test operator |
| 2 | + |
| 3 | +Execute tests via the [test-operator](https://openstack-k8s-operators.github.io/test-operator/). |
| 4 | + |
| 5 | +## Parameters |
| 6 | +* `cifmw_test_operator_artifacts_basedir`: (String) Directory where we will have all test-operator related files. Default value: `{{ cifmw_basedir }}/tests/test_operator` which defaults to `~/ci-framework-data/tests/test_operator` |
| 7 | +* `cifmw_test_operator_log_pod_definition`: (Object) The CR definition template for creating the test log pod. Default value: |
| 8 | +``` |
| 9 | + apiVersion: v1 |
| 10 | + kind: Pod |
| 11 | + metadata: |
| 12 | + name: "test-operator-logs-pod-{{ run_test_fw }}-{{ test_operator_instance_name }}" |
| 13 | + namespace: "{{ cifmw_test_operator_namespace }}" |
| 14 | + spec: |
| 15 | + containers: |
| 16 | + - name: test-operator-logs-container |
| 17 | + image: "{{ cifmw_test_operator_logs_image }}" |
| 18 | + command: ["sleep"] |
| 19 | + args: ["infinity"] |
| 20 | + volumeMounts: "{{ _test_operator_volume_mounts }}" |
| 21 | + volumes: "{{ _test_operator_volumes }}" |
| 22 | + tolerations: "{{ cifmw_test_operator_tolerations | default(omit) }}" |
| 23 | +``` |
| 24 | +* `cifmw_test_operator_stages`: (List) List of dictionaries defining the stages that should be used in the test operator role. List items options are: |
| 25 | + * `name`: (String) The name of the stage. The name must be unique. |
| 26 | + * `type`: (String) The framework name you would like to call, currently the options are: tempest, ansibletest, horizontest, tobiko. |
| 27 | + * `test_vars_file`: (String) Path to the file used for testing, this file should contain the testing params for this stage. Only parameters specific for the controller can be used (Tempest, Ansibletest, Horizontest and Tobiko). |
| 28 | + * `test_vars`: (String) Testing parameters for this specific stage if a `test_vars` is used the specified parameters would override the ones in the `test_vars_file`. Only parameters specific for the controller can be used (Tempest, Ansibletest, Horizontest and Tobiko). |
| 29 | + > Important note! Generally only the variables with the following structure can be used to override inside a stage: `cifmw_test_operator_[test-operator CR name]_[parameter name]`. For example, these variables cannot be overridden per stage: `cifmw_test_operator_default_registry`, `cifmw_test_operator_default_namespace`, `cifmw_test_operator_default_image_tag`. One exception is `cifmw_test_operator_namespace`, which allows running the testing frameworks in multiple namespaces. |
| 30 | + * `pre_test_stage_hooks`: (List) List of pre hooks to run as described [hooks README](https://github.com/openstack-k8s-operators/ci-framework/tree/main/roles/run_hook#hooks-expected-format). |
| 31 | + * `post_test_stage_hooks`: (List) List of post hooks to run as described [hooks README](https://github.com/openstack-k8s-operators/ci-framework/tree/main/roles/run_hook#hooks-expected-format). |
| 32 | + Default value: |
| 33 | +``` |
| 34 | +cifmw_test_operator_stages: |
| 35 | + - name: tempest |
| 36 | + type: tempest |
| 37 | +``` |
| 38 | + |
| 39 | + |
| 40 | +## Tempest specific parameters |
| 41 | +* `cifmw_test_operator_tempest_name`: (String) Value used in the `Tempest.Metadata.Name` field. The value specifies the name of some resources spawned by the test-operator role. Default value: `tempest-tests` |
| 42 | +* `cifmw_test_operator_tempest_tempestconf_config`: (Dict) This parameter can be used to customize the execution of the `discover-tempest-config` run. Please consult the test-operator documentation. For example, to pass a custom configuration for `tempest.conf`, use the `overrides` section: |
| 43 | +``` |
| 44 | +cifmw_test_operator_tempest_tempestconf_config: |
| 45 | + overrides: | |
| 46 | + identity.v3_endpoint_type public |
| 47 | +Default value: {} |
| 48 | +``` |
| 49 | +* `cifmw_test_operator_tempest_config`: (Object) Definition of Tempest CRD instance that is passed to the test-operator (see [the test-operator documentation](https://openstack-k8s-operators.github.io/test-operator/crds.html#tempest-custom-resource)). Default value: |
| 50 | +``` |
| 51 | + apiVersion: test.openstack.org/v1beta1 |
| 52 | + kind: Tempest |
| 53 | + metadata: |
| 54 | + name: "{{ cifmw_test_operator_tempest_name }}" |
| 55 | + namespace: "{{ cifmw_test_operator_namespace }}" |
| 56 | + spec: |
| 57 | + containerImage: "{{ cifmw_test_operator_tempest_image }}:{{ cifmw_test_operator_tempest_image_tag }}" |
| 58 | + storageClass: "{{ cifmw_test_operator_storage_class }}" |
| 59 | + tolerations: "{{ cifmw_test_operator_tolerations | default(omit) }}" |
| 60 | + nodeSelector: "{{ cifmw_test_operator_node_selector | default(omit) }}" |
| 61 | + networkAttachments: "{{ cifmw_test_operator_tempest_network_attachments }}" |
| 62 | + tempestRun: |
| 63 | + includeList: | |
| 64 | + {{ cifmw_test_operator_tempest_include_list | default('') }} |
| 65 | + excludeList: | |
| 66 | + {{ cifmw_test_operator_tempest_exclude_list | default('') }} |
| 67 | + concurrency: "{{ cifmw_test_operator_tempest_concurrency | default(8) }}" |
| 68 | + externalPlugin: "{{ cifmw_test_operator_tempest_external_plugin | default([]) }}" |
| 69 | + extraRPMs: "{{ cifmw_test_operator_tempest_extra_rpms | default([]) }}" |
| 70 | + extraImages: "{{ cifmw_test_operator_tempest_extra_images | default([]) }}" |
| 71 | + tempestconfRun: "{{ cifmw_test_operator_tempest_tempestconf_config | default(omit) }}" |
| 72 | + debug: "{{ cifmw_test_operator_tempest_debug }}" |
| 73 | +``` |
| 74 | + |
| 75 | +## Tobiko specific parameters |
| 76 | +* `cifmw_test_operator_tobiko_name`: (String) Value used in the `Tobiko.Metadata.Name` field. The value specifies the name of some resources spawned by the test-operator role. Default value: `tobiko-tests` |
| 77 | +* `cifmw_test_operator_tobiko_config`: (Dict) Definition of Tobiko CRD instance that is passed to the test-operator (see [the test-operator documentation](https://openstack-k8s-operators.github.io/test-operator/crds.html#tobiko-custom-resource)). Default value: |
| 78 | +``` |
| 79 | + apiVersion: test.openstack.org/v1beta1 |
| 80 | + kind: Tobiko |
| 81 | + metadata: |
| 82 | + name: "{{ cifmw_test_operator_tobiko_name }}" |
| 83 | + namespace: "{{ cifmw_test_operator_namespace }}" |
| 84 | + spec: |
| 85 | + kubeconfigSecretName: "{{ cifmw_test_operator_tobiko_kubeconfig_secret }}" |
| 86 | + storageClass: "{{ cifmw_test_operator_storage_class }}" |
| 87 | + containerImage: "{{ cifmw_test_operator_tobiko_image }}:{{ cifmw_test_operator_tobiko_image_tag }}" |
| 88 | + testenv: "{{ cifmw_test_operator_tobiko_testenv }}" |
| 89 | + version: "{{ cifmw_test_operator_tobiko_version }}" |
| 90 | + pytestAddopts: "{{ cifmw_test_operator_tobiko_pytest_addopts if cifmw_test_operator_tobiko_pytest_addopts is not none else omit }}" |
| 91 | + tolerations: "{{ cifmw_test_operator_tolerations | default(omit) }}" |
| 92 | + nodeSelector: "{{ cifmw_test_operator_node_selector | default(omit) }}" |
| 93 | + networkAttachments: "{{ cifmw_test_operator_tobiko_network_attachments }}" |
| 94 | + debug: "{{ cifmw_test_operator_tobiko_debug }}" |
| 95 | + # preventCreate: preventCreate is generated by the test_operator role based on the value of cifmw_test_operator_tobiko_prevent_create |
| 96 | + # numProcesses: numProcesses is generated by the test_operator role based on the value of cifmw_test_operator_tobiko_num_processes |
| 97 | + # privateKey: privateKey is automatically generated by the test_operator role |
| 98 | + # publicKey: publicKey is automatically generated by the test_operator role |
| 99 | + # config: config is generated combining cifmw_test_operator_tobiko_default_conf and cifmw_test_operator_tobiko_override_conf |
| 100 | + workflow: "{{ cifmw_test_operator_tobiko_workflow }}" |
| 101 | +``` |
| 102 | + |
| 103 | +## AnsibleTest specific parameters |
| 104 | +* `cifmw_test_operator_ansibletest_name`: (String) Value used in the `Ansibletest.Metadata.Name` field. The value specifies the name of some resources spawned by the test-operator role. Default value: `ansibletest` |
| 105 | +* `cifmw_test_operator_ansibletest_config`: Definition of AnsibleTest CRD instance that is passed to the test-operator (see [the test-operator documentation](https://openstack-k8s-operators.github.io/test-operator/crds.html)). Default value: |
| 106 | +``` |
| 107 | + apiVersion: test.openstack.org/v1beta1 |
| 108 | + kind: AnsibleTest |
| 109 | + metadata: |
| 110 | + name: "{{ cifmw_test_operator_ansibletest_name }}" |
| 111 | + namespace: "{{ cifmw_test_operator_namespace }}" |
| 112 | + spec: |
| 113 | + containerImage: "{{ cifmw_test_operator_ansibletest_image }}:{{ cifmw_test_operator_ansibletest_image_tag }}" |
| 114 | + extraConfigmapsMounts: "{{ cifmw_test_operator_ansibletest_extra_configmaps_mounts }}" |
| 115 | + storageClass: "{{ cifmw_test_operator_storage_class }}" |
| 116 | + computeSSHKeySecretName: "{{ cifmw_test_operator_ansibletest_compute_ssh_key_secret_name }}" |
| 117 | + workloadSSHKeySecretName: "{{ cifmw_test_operator_ansibletest_workload_ssh_key_secret_name }}" |
| 118 | + ansibleGitRepo: "{{ cifmw_test_operator_ansibletest_ansible_git_repo }}" |
| 119 | + ansiblePlaybookPath: "{{ cifmw_test_operator_ansibletest_ansible_playbook_path }}" |
| 120 | + ansibleCollections: "{{ cifmw_test_operator_ansibletest_ansible_collection }}" |
| 121 | + ansibleVarFiles: "{{ cifmw_test_operator_ansibletest_ansible_var_files }}" |
| 122 | + ansibleExtraVars: "{{ cifmw_test_operator_ansibletest_ansible_extra_vars }}" |
| 123 | + ansibleInventory: "{{ cifmw_test_operator_ansibletest_ansible_inventory }}" |
| 124 | + openStackConfigMap: "{{ cifmw_test_operator_ansibletest_openstack_config_map }}" |
| 125 | + openStackConfigSecret: "{{ cifmw_test_operator_ansibletest_openstack_config_secret }}" |
| 126 | + workflow: "{{ cifmw_test_operator_ansibletest_workflow }}" |
| 127 | + debug: "{{ cifmw_test_operator_ansibletest_debug }}" |
| 128 | +``` |
| 129 | + |
| 130 | +## Horizontest specific parameters |
| 131 | +* `cifmw_test_operator_horizontest_name`: (String) Value used in the `Horizontest.Metadata.Name` field. The value specifies the name of some resources spawned by the test-operator role. Default value: `horizontest-tests` |
| 132 | +* `cifmw_test_operator_horizontest_config`: (Dict) Definition of HorizonTest CR instance that is passed to the test-operator (see [the test-operator documentation](https://openstack-k8s-operators.github.io/test-operator/crds.html#horizontest-custom-resource)). Default value: |
| 133 | +``` |
| 134 | + apiVersion: test.openstack.org/v1beta1 |
| 135 | + kind: HorizonTest |
| 136 | + metadata: |
| 137 | + name: "{{ cifmw_test_operator_horizontest_name }}" |
| 138 | + namespace: "{{ cifmw_test_operator_namespace }}" |
| 139 | + spec: |
| 140 | + containerImage: "{{ cifmw_test_operator_horizontest_image }}:{{ cifmw_test_operator_horizontest_image_tag }}" |
| 141 | + adminUsername: "{{ cifmw_test_operator_horizontest_admin_username }}" |
| 142 | + adminPassword: "{{ cifmw_test_operator_horizontest_admin_password }}" |
| 143 | + dashboardUrl: "{{ cifmw_test_operator_horizontest_dashboard_url }}" |
| 144 | + authUrl: "{{ cifmw_test_operator_horizontest_auth_url }}" |
| 145 | + repoUrl: "{{ cifmw_test_operator_horizontest_repo_url }}" |
| 146 | + horizonRepoBranch: "{{ cifmw_test_operator_horizontest_horizon_repo_branch }}" |
| 147 | + imageUrl: "{{ cifmw_test_operator_horizontest_image_url }}" |
| 148 | + projectName: "{{ cifmw_test_operator_horizontest_project_name }}" |
| 149 | + user: "{{ cifmw_test_operator_horizontest_user }}" |
| 150 | + password: "{{ cifmw_test_operator_horizontest_password }}" |
| 151 | + flavorName: "{{ cifmw_test_operator_horizontest_flavor_name }}" |
| 152 | + logsDirectoryName: "{{ cifmw_test_operator_horizontest_logs_directory_name }}" |
| 153 | + debug: "{{ cifmw_test_operator_horizontest_debug }}" |
| 154 | + extraFlag: "{{ cifmw_test_operator_horizontest_extra_flag }}" |
| 155 | + projectNameXpath "{{ cifmw_test_operator_horizontest_project_name_xpath }}" |
| 156 | + horizonTestDir: "{{ cifmw_test_operator_horizontest_horizon_test_dir }}" |
| 157 | +``` |
| 158 | + |
| 159 | +## Examples |
| 160 | + |
| 161 | +### Execute the `test-operator` role multiple times within a single job |
| 162 | + |
| 163 | +If you want to run the `test-operator` role twice within a single job, make sure |
| 164 | +that for the second run, you specify a value for the `cifmw_test_operator_*_name` |
| 165 | +other than the default one (e.g., `tempest-tests`, `tobiko-tests`, ...): |
| 166 | + |
| 167 | +``` |
| 168 | +cifmw_test_operator_tempest_name: "post-update-tempest-tests" |
| 169 | +``` |
| 170 | + |
| 171 | +### Use test operator stages |
| 172 | +Test operator stages are meant to allow setting the order of stages,running stages |
| 173 | +of the same controller multiple times and using different hooks,vars and names for stages. |
| 174 | +``` |
| 175 | +cifmw_test_operator_stages: |
| 176 | + - name: basic-functionallity |
| 177 | + type: tempest |
| 178 | + test_vars: |
| 179 | + cifmw_test_operator_tempest_name: 'basic-functionality-tests' |
| 180 | + - name: ansibletest |
| 181 | + type: ansibletest |
| 182 | + - name: advanced-tests |
| 183 | + type: tempest |
| 184 | + test_vars: |
| 185 | + cifmw_test_operator_tempest_name: 'advanced-tests' |
| 186 | + - name: tobiko |
| 187 | + type: tobiko |
| 188 | + test_vars_file: /path/to/tobiko/override/test/file |
| 189 | +``` |
0 commit comments