Skip to content

CLOUD-929 - Initial pytest test migration#2058

Draft
jvpasinatto wants to merge 24 commits intomainfrom
pytest-complete
Draft

CLOUD-929 - Initial pytest test migration#2058
jvpasinatto wants to merge 24 commits intomainfrom
pytest-complete

Conversation

@jvpasinatto
Copy link
Contributor

@jvpasinatto jvpasinatto commented Sep 23, 2025

Due to the high volume of requests, we're unable to provide free service for this account. To continue using the service, please upgarde to a paid plan.

CHANGE DESCRIPTION

This PR sets up the initial groundwork for transitioning to pytest as our testing framework. It includes:

  • Conversion of init-deploy, finalizer and liveness tests to pytest
  • Addition of a GitHub Action to check Python code quality

Create virtual environment and install dependencies:
uv sync --locked

Run all tests:
uv run pytest

Run a specific test
uv run pytest e2e-tests/init-deploy

Enabling debug logging:
LOG_LEVEL=DEBUG uv run pytest

CHECKLIST

Jira

  • Is the Jira ticket created and referenced properly?
  • Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
  • Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

  • Is an E2E test/test case added for the new feature/change?
  • Are unit tests added where appropriate?
  • Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

  • Are all needed new/changed options added to default YAML files?
  • Are all needed new/changed options added to the Helm Chart?
  • Did we add proper logging messages for operator actions?
  • Did we ensure compatibility with the previous version or cluster upgrade process?
  • Does the change support oldest and newest supported MongoDB version?
  • Does the change support oldest and newest supported Kubernetes version?

@pull-request-size pull-request-size bot added the size/XXL 1000+ lines label Sep 23, 2025

# TODO: consider using Python for filtering instead of yq
yq_filter = f"""
del(.metadata.ownerReferences[].apiVersion) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be nice in a separated file perhaps



@pytest.fixture(scope="class")
def deploy_chaos_mesh(namespace):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could have separate file for helm installed dependencies and use a generic function with parameters to install a helm chart since subprocess is going to be used fot it

@egegunes egegunes added this to the v1.22.0 milestone Dec 15, 2025
Comment on lines 26 to 36
def kubectl_bin(*args, check: bool = True, input_data: str = "") -> str:
"""Execute kubectl command"""
cmd = ["kubectl"] + list(args)
logger.debug(" ".join(map(str, cmd)))
result = subprocess.run(cmd, check=check, capture_output=True, text=True, input=input_data)

if result.stderr:
logger.warning(f"kubectl error: {result.stderr}")
return result.stderr

return result.stdout
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be nice to add a -n <current-namespace> option and stop changing the kubectl context. This would match the approach used in kuttl tests and would let us run multiple tests in parallel on the same cluster. I don’t think it would cause any problems.

os.environ.setdefault("DELETE_CRD_ON_START", "1")
os.environ.setdefault("SKIP_DELETE", "0")
os.environ.setdefault("SKIP_BACKUPS_TO_AWS_GCP_AZURE", "1")
os.environ.setdefault("UPDATE_COMPARE_FILES", "0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPDATE_COMPARE_FILES isn't implemented in this PR


if result.stderr:
logger.warning(f"kubectl error: {result.stderr}")
return result.stderr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return result.stderr

If we have stderr and result.returncode is 0, I think we should still return stdout

create_infra() {
local ns="$1"

echo "$ns" > /tmp/pytest_current_namespace
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[shfmt] reported by reviewdog 🐶

Suggested change
echo "$ns" > /tmp/pytest_current_namespace
echo "$ns" >/tmp/pytest_current_namespace

@JNKPercona
Copy link
Collaborator

Test Name Result Time
arbiter passed 00:10:18
balancer passed 00:17:55
cross-site-sharded passed 00:16:45
custom-replset-name passed 00:09:34
custom-tls passed 00:13:30
custom-users-roles passed 00:09:17
custom-users-roles-sharded passed 00:10:34
data-at-rest-encryption passed 00:11:45
data-sharded passed 00:22:54
demand-backup failure 01:28:30
demand-backup-eks-credentials-irsa passed 00:00:09
demand-backup-fs passed 00:23:23
demand-backup-if-unhealthy passed 00:10:01
demand-backup-incremental-aws failure 00:12:18
demand-backup-incremental-azure passed 00:12:21
demand-backup-incremental-gcp-native passed 00:13:21
demand-backup-incremental-gcp-s3 passed 00:10:18
demand-backup-incremental-minio passed 00:25:26
demand-backup-incremental-sharded-aws failure 01:17:12
demand-backup-incremental-sharded-azure failure 01:30:13
demand-backup-incremental-sharded-gcp-native passed 00:17:09
demand-backup-incremental-sharded-gcp-s3 passed 00:17:09
demand-backup-incremental-sharded-minio failure 01:30:20
demand-backup-physical-parallel failure 01:30:14
demand-backup-physical-aws failure 00:12:16
demand-backup-physical-azure passed 00:12:51
demand-backup-physical-gcp-s3 passed 00:12:36
demand-backup-physical-gcp-native failure 01:30:08
demand-backup-physical-minio passed 00:21:29
demand-backup-physical-minio-native passed 00:26:41
demand-backup-physical-minio-native-tls passed 01:11:40
demand-backup-physical-sharded-parallel failure 01:30:15
demand-backup-physical-sharded-aws failure 00:54:12
demand-backup-physical-sharded-azure passed 00:17:25
demand-backup-physical-sharded-gcp-native passed 00:25:32
demand-backup-physical-sharded-minio passed 01:30:23
demand-backup-physical-sharded-minio-native passed 00:17:25
demand-backup-sharded failure 01:30:10
disabled-auth failure 00:29:14
expose-sharded failure 00:17:47
finalizer passed 00:06:47
ignore-labels-annotations failure 00:05:07
init-deploy failure 00:04:22
ldap passed 01:30:24
ldap-tls passed 00:12:19
limits passed 00:05:30
liveness passed 00:08:08
mongod-major-upgrade failure 00:43:03
mongod-major-upgrade-sharded passed 00:23:29
monitoring-2-0 failure 00:26:00
monitoring-pmm3 passed 01:08:48
multi-cluster-service passed 00:12:31
multi-storage passed 00:18:20
non-voting-and-hidden passed 00:48:21
one-pod passed 00:07:00
operator-self-healing-chaos failure 00:43:06
pitr failure 00:00:37
pitr-physical failure 00:00:36
pitr-sharded failure 00:00:34
pitr-to-new-cluster passed 00:43:07
pitr-physical-backup-source failure 00:00:33
preinit-updates failure 00:00:32
pvc-auto-resize failure 00:00:31
pvc-resize failure 00:00:26
recover-no-primary passed 00:31:01
replset-overrides passed 00:24:43
replset-remapping failure 00:00:33
replset-remapping-sharded passed 00:17:25
rs-shard-migration failure 00:00:29
scaling failure 00:00:32
scheduled-backup failure 00:00:35
security-context failure 00:00:30
self-healing-chaos failure 01:30:09
service-per-pod failure 00:09:41
serviceless-external-nodes passed 00:06:03
smart-update passed 00:07:29
split-horizon passed 00:14:06
stable-resource-version failure 01:30:14
storage passed 00:07:01
tls-issue-cert-manager passed 00:32:00
unsafe-psa failure 01:30:05
upgrade passed 00:09:32
upgrade-consistency passed 00:07:21
upgrade-consistency-sharded-tls passed 00:59:56
upgrade-sharded failure 00:08:40
upgrade-partial-backup failure 01:30:08
users passed 00:18:06
users-vault failure 01:30:12
version-service failure 01:30:03
Summary Value
Tests Run 89/89
Job Duration 07:22:52
Total Test Time 44:08:07

Commit: 9a7560b
Image: perconalab/percona-server-mongodb-operator:PR-2058-9a7560b9
Test report: report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants