Skip to content

no-wait option in tests.yaml #688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hemanthnakkina wants to merge 1 commit into
base: master
Choose a base branch
from
Open

no-wait option in tests.yaml #688

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions doc/source/addingcharmtests.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,38 @@ In the above case, focal-ussuri will be deployed using the --force parameter.
i.e. the `tests_options.force_deploy['focal-ussuri']` option applies to the
`focal-ussuri` bundle whether it appears in any of the bundle sections.

Skipping the post-deploy wait
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

By default, after deploying a bundle zaza waits for all applications to reach
the states specified in ``target_deploy_status`` (or active/idle if not
configured). This wait can be skipped on a per-bundle basis using the
``tests_options.no_wait_deploy`` option.

This is useful when a charm requires post-deploy configuration before it can
reach its target state, or when you want to proceed immediately to the
configure step without waiting.

In the ``tests.yaml`` the option is added as a list item::

charm_name: vault
gate_bundles:
- focal-ussuri

target_deploy_status:
vault:
workload-status: blocked
workload-status-message: Vault needs to be initialized

tests_options:
no_wait_deploy:
- focal-ussuri
Copy link
Copy Markdown
Member

@freyes freyes Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zaza supports adding a bundle with a prefix , e.g. https://github.com/openstack-charmers/charmed-openstack-tester/blob/master/tests/distro-regression/tests/tests.yaml#L5 , will this option honor it?

gate_bundles:
- security:focal-ussuri
- focal-ussuri
test_options:
  no_wait_deploy:
    - focal-ussuri

in a tests.yaml like this ^, the no-wait should only match the second bundle and not security:focal-ussuri


Comment on lines +188 to +202
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description example shows tests_options.no_wait_deploy: - smoke, but the implemented behavior (and this doc example) expects a list of bundle names (e.g. focal-ussuri). If smoke/dev/gate selectors were intended, the code needs to be adjusted; otherwise please update the PR description/example to avoid confusion.

Copilot uses AI. Check for mistakes.
In the above case, zaza will deploy the ``focal-ussuri`` bundle and immediately
proceed to the configure step without waiting for vault to enter the blocked
state. This is equivalent to passing ``--no-wait`` directly to
``functest-deploy``.
Comment on lines +205 to +206
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new docs state this option is equivalent to passing --no-wait to functest-deploy, but the implementation also skips the additional post-deploy block_until_all_units_idle settle step in func_test_runner.run_env_deployment. Either update the docs to describe the extra behavior (and drop the equivalence claim), or change the runner to still perform the model-settle wait even when skipping target_deploy_status waiting.

Suggested change
state. This is equivalent to passing ``--no-wait`` directly to
``functest-deploy``.
state or for the model to settle (the additional post-deploy
``block_until_all_units_idle`` wait is also skipped).

Copilot uses AI. Check for mistakes.

Augmenting behaviour of configure steps
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
82 changes: 60 additions & 22 deletions unit_tests/test_zaza_charm_lifecycle_func_test_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,11 +85,11 @@ def test_func_test_runner(self):
deploy_calls = [
mock.call(cwd + '/tests/bundles/bundle1.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'},
force=True, test_directory=None, trust=False,
wait=True, force=True, test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(cwd + '/tests/bundles/bundle2.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'},
force=True, test_directory=None, trust=False,
wait=True, force=True, test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
configure_calls = [
mock.call('newmodel', [
Expand Down Expand Up @@ -164,24 +164,24 @@ def test_func_test_runner_cmr(self):
cwd = os.getcwd()
deploy_calls = [
mock.call(cwd + '/tests/bundles/bundle1.yaml', 'm1',
model_ctxt={'default_alias': 'm1'}, force=False,
test_directory=None, trust=False,
model_ctxt={'default_alias': 'm1'}, wait=True,
force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(cwd + '/tests/bundles/bundle2.yaml', 'm2',
model_ctxt={'default_alias': 'm2'}, force=False,
test_directory=None, trust=False,
model_ctxt={'default_alias': 'm2'}, wait=True,
force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(
cwd + '/tests/bundles/bundle5.yaml',
'm3',
model_ctxt={'model_alias_5': 'm3', 'model_alias_6': 'm4'},
force=False, test_directory=None, trust=False,
wait=True, force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(
cwd + '/tests/bundles/bundle6.yaml',
'm4',
model_ctxt={'model_alias_5': 'm3', 'model_alias_6': 'm4'},
force=False, test_directory=None, trust=False,
wait=True, force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
configure_calls = [
mock.call('m1', [
Expand Down Expand Up @@ -256,12 +256,12 @@ def test_func_test_runner_with_before_script(self):
mock.call('newmodel', 'default_alias', test_directory=None)]
deploy_calls = [
mock.call(cwd + '/tests/bundles/bundle1.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'}, force=False,
test_directory=None, trust=False,
model_ctxt={'default_alias': 'newmodel'}, wait=True,
force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(cwd + '/tests/bundles/bundle2.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'}, force=False,
test_directory=None, trust=False,
model_ctxt={'default_alias': 'newmodel'}, wait=True,
force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
before_deploy_calls = [
mock.call('newmodel', [
Expand Down Expand Up @@ -319,7 +319,7 @@ def test_func_test_runner_smoke(self):
deploy_calls = [
mock.call(cwd + '/tests/bundles/bundle2.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'},
force=False,
wait=True, force=False,
test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)
Expand Down Expand Up @@ -352,12 +352,12 @@ def test_func_test_runner_dev(self):
cwd = os.getcwd()
deploy_calls = [
mock.call(cwd + '/tests/bundles/bundle3.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'}, force=False,
test_directory=None, trust=False,
model_ctxt={'default_alias': 'newmodel'}, wait=True,
force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(cwd + '/tests/bundles/bundle4.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'}, force=False,
test_directory=None, trust=False,
model_ctxt={'default_alias': 'newmodel'}, wait=True,
force=False, test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)

Expand Down Expand Up @@ -392,7 +392,7 @@ def test_func_test_runner_specify_bundle(self):
cwd + '/tests/bundles/maveric-filebeat.yaml',
'newmodel',
model_ctxt={'default_alias': 'newmodel'},
force=False,
wait=True, force=False,
test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)
Expand Down Expand Up @@ -429,7 +429,7 @@ def test_func_test_runner_specify_bundle_with_alias(self):
cwd + '/tests/bundles/maveric-filebeat.yaml',
'newmodel',
model_ctxt={'alias': 'newmodel'},
force=False,
wait=True, force=False,
test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)
Expand Down Expand Up @@ -459,7 +459,7 @@ def test_func_test_runner_specify_bundle_with_implicit_alias(self):
cwd + '/tests/bundles/maveric-filebeat.yaml',
'newmodel',
model_ctxt={'alias': 'newmodel'},
force=False,
wait=True, force=False,
test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)
Expand Down Expand Up @@ -498,15 +498,15 @@ def test_func_test_runner_cmr_specify_bundle_with_alias(self):
'newmodel1',
model_ctxt={'alias': 'newmodel1',
'another_alias': 'newmodel2'},
force=False,
wait=True, force=False,
test_directory=None, trust=False,
ignore_hard_deploy_errors=False),
mock.call(
cwd + '/tests/bundles/maverick-things.yaml',
'newmodel2',
model_ctxt={'alias': 'newmodel1',
'another_alias': 'newmodel2'},
force=False,
wait=True, force=False,
test_directory=None, trust=False,
ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)
Expand Down Expand Up @@ -604,3 +604,41 @@ def test_main_bundle_keep_model_ambiguous_case2(self):

def test_main_bundle_keep_model_ambiguous_case3(self):
self.__keep_model_ambiguous(False, True, True)

def test_func_test_runner_no_wait_deploy(self):
self.patch_object(lc_func_test_runner.utils, 'get_charm_config')
self.patch_object(lc_func_test_runner.utils, 'generate_model_name')
self.patch_object(lc_func_test_runner.prepare, 'prepare')
self.patch_object(lc_func_test_runner.before_deploy, 'before_deploy')
self.patch_object(lc_func_test_runner.deploy, 'deploy')
self.patch_object(lc_func_test_runner.configure, 'configure')
self.patch_object(lc_func_test_runner.test, 'test')
self.patch_object(lc_func_test_runner.destroy, 'destroy')
self.patch_object(
lc_func_test_runner.zaza.model,
'block_until_all_units_idle')
self.generate_model_name.return_value = 'newmodel'
self.get_charm_config.return_value = {
'charm_name': 'mycharm',
'gate_bundles': ['bundle1', 'bundle2'],
'tests': [
'zaza.charm_tests.mycharm.tests.SmokeTest'],
'tests_options': {
'no_wait_deploy': ['bundle1']
}}
lc_func_test_runner.func_test_runner()
cwd = os.getcwd()
deploy_calls = [
mock.call(cwd + '/tests/bundles/bundle1.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'},
wait=False, force=False, test_directory=None,
trust=False, ignore_hard_deploy_errors=False),
mock.call(cwd + '/tests/bundles/bundle2.yaml', 'newmodel',
model_ctxt={'default_alias': 'newmodel'},
wait=True, force=False, test_directory=None,
trust=False, ignore_hard_deploy_errors=False)]
self.deploy.assert_has_calls(deploy_calls)
# block_until_all_units_idle should only be called for bundle2
# (bundle1 has no_wait_deploy set)
self.block_until_all_units_idle.assert_called_once_with(
ignore_hard_errors=False, model_name='newmodel')
24 changes: 24 additions & 0 deletions unit_tests/test_zaza_charm_lifecycle_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,30 @@ def test_is_config_deploy_trusted_for_bundle(self):
}
self.assertTrue(lc_utils.is_config_deploy_trusted_for_bundle('x'))

def test_no_wait_deploy(self):
self.patch_object(lc_utils, 'get_charm_config')
# test that no options at all returns False
self.get_charm_config.return_value = {}
self.assertFalse(lc_utils.no_wait_deploy('x'))
# test that if options exist but no bundle
self.get_charm_config.return_value = {
'tests_options': {}
}
self.assertFalse(lc_utils.no_wait_deploy('x'))
self.get_charm_config.return_value = {
'tests_options': {
'no_wait_deploy': []
}
}
self.assertFalse(lc_utils.no_wait_deploy('x'))
# verify that it returns True if the bundle is mentioned
self.get_charm_config.return_value = {
'tests_options': {
'no_wait_deploy': ['x']
}
}
self.assertTrue(lc_utils.no_wait_deploy('x'))

def test_get_class(self):
self.assertEqual(
type(lc_utils.get_class('unit_tests.'
Expand Down
10 changes: 9 additions & 1 deletion zaza/charm_lifecycle/func_test_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,21 +142,29 @@ def run_env_deployment(env_deployment, keep_model=DESTROY_MODEL, force=False,
deployment.bundle)
errors_ = (ignore_hard_deploy_errors or
utils.ignore_hard_deploy_errors(deployment.bundle))
no_wait_ = utils.no_wait_deploy(deployment.bundle)
deploy.deploy(
os.path.join(
utils.get_bundle_dir(),
'{}.yaml'.format(deployment.bundle)),
deployment.model_name,
model_ctxt=model_aliases,
wait=not no_wait_,
force=force_,
trust=trust_,
test_directory=test_directory,
ignore_hard_deploy_errors=errors_)

# When deploying bundles with cross model relations, hooks may be
# triggered in already deployedi models so wait for all models to
# triggered in already deployed models so wait for all models to
# settle.
for deployment in env_deployment.model_deploys:
if utils.no_wait_deploy(deployment.bundle):
logging.info(
"Skipping post-deploy wait for {} "
"(no_wait_deploy is set)".format(
deployment.model_name))
continue
Comment on lines 158 to +167
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says we must “wait for all models to settle” for cross-model relations, but the new no_wait_deploy check skips the settle wait (block_until_all_units_idle) entirely for those deployments. Either update the explanatory comment to reflect the conditional behavior, or reconsider skipping this settle step since it’s specifically meant to avoid cross-model hook races.

Copilot uses AI. Check for mistakes.
logging.info("Waiting for {} to settle".format(
deployment.model_name))
errors_ = (ignore_hard_deploy_errors or
Expand Down
33 changes: 33 additions & 0 deletions zaza/charm_lifecycle/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -526,6 +526,39 @@ def ignore_hard_deploy_errors(
return False


def no_wait_deploy(
bundle_name, yaml_file=None, fatal=True):
"""Ask if the wait step should be skipped after deploying bundle.

Setting no_wait_deploy for a bundle means that zaza will not wait for
the applications to reach the states defined in target_deploy_status (or
active/idle) after deploying the bundle.
Comment on lines +531 to +535
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no_wait_deploy is described here as skipping only the target_deploy_status/active-idle wait, but in func_test_runner.run_env_deployment the same flag is also used to skip the post-deploy block_until_all_units_idle settle step. Please clarify the docstring to match the actual behavior (or adjust the runner if the settle wait should still occur).

Suggested change
"""Ask if the wait step should be skipped after deploying bundle.
Setting no_wait_deploy for a bundle means that zaza will not wait for
the applications to reach the states defined in target_deploy_status (or
active/idle) after deploying the bundle.
"""Ask if the deploy wait/settle steps should be skipped for a bundle.
Setting no_wait_deploy for a bundle means that zaza will not:
* wait for the applications to reach the states defined in
target_deploy_status (or active/idle) after deploying the bundle; or
* perform the post-deploy block_until_all_units_idle "settle" wait.

Copilot uses AI. Check for mistakes.

The tests_options section needs to look like:

tests_options:
no_wait_deploy:
- focal-ussuri

:param bundle_name: bundle to check in the no_wait_deploy list
:type bundle_name: str
:param yaml_file: the YAML file that contains the tests specification
:type yaml_file: Optional[str]
:param fatal: whether any errors cause an exception or are just logged.
:type fatal: bool
:returns: True if the config option is set for the bundle
:rtype: bool
:raises: OSError if the YAML file doesn't exist and fatal=True
"""
config = get_charm_config(yaml_file, fatal)
try:
return bundle_name in config['tests_options']['no_wait_deploy']
# Type error is if no_wait_deploy is present, but with no list
except (KeyError, TypeError):
pass
return False


def get_class(class_str):
"""Get the class represented by the given string.

Expand Down
Loading