Skip to content

xcloud-backend.md

Latest commit

 

History

History
217 lines (156 loc) · 8.94 KB

xcloud-backend.md

File metadata and controls

217 lines (156 loc) · 8.94 KB

ScyllaDB Cloud (xcloud) backend

Overview

The xcloud backend allows SCT to provision and test Scylla on ScyllaDB Cloud. This backend leverages the ScyllaDB Cloud REST API to create, configure, and manage clusters across different cloud providers (AWS and GCE).

Unlike other SCT backends where infrastructure is provisioned and operated directly, the xcloud backend provisions managed ScyllaDB clusters, where the database is already installed and configured.

Key features

  • Managed ScyllaDB Clusters: DB nodes are fully managed by ScyllaDB Cloud
  • Multi-Cloud Support: deploy clusters on AWS or GCE cloud providers
  • VPC Peering: private connectivity between SCT infrastructure and cloud clusters
  • Vector Search Support: deploy Vector Search nodes alongside DB nodes
  • Monitoring Integration: federation with ScyllaDB Cloud Prometheus endpoint
  • Automated Cleanup: keep-tag based resource cleanup mechanism

Prerequisites

API credentials

To use the xcloud backend, you need Siren user API credentials for ScyllaDB Cloud.

There are pre-created Siren users for different cloud environments (lab, staging, production).

Additionally, developers can configure their own Siren user in a ScyllaDB Cloud environment and generate API credentials, which can then be provided to SCT via a local file. Instructions for configuring test billing for a Siren user and generating API credentials can be found in ScyllaDB Cloud API access for SCT.

AWS/GCE access

Since loader and monitor nodes are provisioned directly on AWS/GCE (not a part of ScyllaDB Cloud cluster), the appropriate cloud provider credentials should be configured.

SSH connectivity to DB nodes

SSH access to cloud DB nodes is performed via a StrongDM bastion host. StrongDM hosts are configured only for pre-created Siren users.

Configuration

Required configuration parameters

Parameter Environment Variable Description
cluster_backend SCT_CLUSTER_BACKEND Must be set to xcloud
xcloud_provider SCT_XCLOUD_PROVIDER Cloud provider: aws or gce
scylla_version SCT_SCYLLA_VERSION Scylla version to deploy

Optional configuration parameters

Parameter Environment Variable Description Default
xcloud_env SCT_XCLOUD_ENV ScyllaDB Cloud environment (lab, staging or prod) lab
xcloud_credentials_path SCT_XCLOUD_CREDENTIALS_PATH Path to local credentials file By default the credentials of pre-created Siren users are used
xcloud_replication_factor SCT_XCLOUD_REPLICATION_FACTOR Cluster replication factor Based on node count
xcloud_vpc_peering SCT_XCLOUD_VPC_PEERING VPC peering configuration (see below) Enabled
n_vector_store_nodes SCT_N_VECTOR_STORE_NODES Number of Vector Search nodes 0 (VS nodes deployment is disabled)

VPC peering configuration

VPC peering enables private connectivity between SCT infrastructure (runner/loaders/monitors) and ScyllaDB Cloud cluster nodes. It is highly recommended to use VPC peering instead of public IPs, to reduce network latency and data transfer costs.

Configuration format:

xcloud_vpc_peering:
  enabled: true
  cidr_pool_base: '172.31.0.0/16'  # base CIDR for cluster networks
  cidr_subnet_size: 24              # subnet size for each cluster

Monitoring configuration

ScyllaDB Cloud clusters provide a Prometheus endpoint for metrics federation. SCT automatically configures monitoring when a monitor node is provisioned.

Running tests

Basic test example

# set required configuration parameters
export SCT_SCYLLA_VERSION=2025.3.0
export SCT_XCLOUD_ENV=lab
export SCT_XCLOUD_PROVIDER=aws

# run a simple provision test
hydra run-test longevity_test.LongevityTest.test_custom_time \
  --backend xcloud \
  --config test-cases/PR-provision-test.yaml --config configurations/network_config/test_communication_public.yaml

SCT configuration example

test-cases/custom/my-xcloud-test.yaml:

test_duration: 60
n_db_nodes: 3
n_loaders: 1
n_monitor_nodes: 1

instance_type_loader: 'c6i.large'
gce_instance_type_loader: 'e2-standard-2'

stress_cmd: [
  "cassandra-stress write cl=QUORUM duration=30m -schema 'replication(strategy=NetworkTopologyStrategy,replication_factor=3)' -mode cql3 native -rate threads=1000 -pop seq=1..5000000"
]

user_prefix: 'my-xcloud-test'

xcloud_vpc_peering:
  enabled: true
  cidr_pool_base: '172.31.0.0/16'
  cidr_subnet_size: 24

Listing cloud clusters

# list all environments using `list-resources` hydra command
hydra list-resources -b xcloud --get-all

# list specific environment only
hydra list-resources -b xcloud --get-all --xcloud-env lab

# list all environments using cleanup script in dry-run mode
./docker/env/hydra.sh python -m utils.cloud_cleanup.xcloud.clean_xcloud --dry-run

Reusing existing xcloud cluster

Existing ScyllaDB Cloud clusters can be reused similarly to other backends - by specifying the test ID of the test run that created the cluster. Find more details in reuse_cluster documentation.

Resources cleanup

ScyllaDB Cloud clusters created by SCT are named with pattern: {user_prefix}-db-cluster-{test_id}-keep-{hours}h

Automatic cleanup using keep-tag

Cloud clusters created by SCT include a keep-tag suffix in their name (e.g., -keep-2h). The tag indicates for how long the cluster should be retained before automated deletion. A periodic cleanup job deletes clusters older than the specified duration.

Keep duration is automatically set based on test duration and is calculated as: test_duration + prepare_stress_duration + margin

Executing cleanup script manually:

# dry-run mode to see what would be deleted (checks all environments by default)
./docker/env/hydra.sh python -m utils.cloud_cleanup.xcloud.clean_xcloud --dry-run

# cleanup of test clusters on specific environment
./docker/env/hydra.sh python -m utils.cloud_cleanup.xcloud.clean_xcloud --environments lab

# cleanup for all environments
./docker/env/hydra.sh python -m utils.cloud_cleanup.xcloud.clean_xcloud

Manual cleanup

Cloud clusters are created under a Siren account and are not tied to a specific user. Therefore, manual cleanup is performed based on the ID of the test that created the cluster.

# get test ID from an SCT run
export TEST_ID=$(cat ~/sct-results/latest/test_id)

hydra clean-resources --test-id $TEST_ID -b xcloud

Architecture details

Cluster components

When running tests with xcloud backend, SCT provisions:

  1. Cloud cluster (managed by ScyllaDB Cloud):

    • Database nodes
    • Vector Search nodes (optional)

  2. SCT Infrastructure (provisioned on AWS/GCE):

    • SCT runner node
    • Loader nodes
    • Monitor nodes

Network connectivity

┌────────────────────────────────────────────┐
│         ScyllaDB Cloud                     │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  │
│  │ DB Node  │  │ DB Node  │  │ DB Node  │  │
│  │ (private)│  │ (private)│  │ (private)│  │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  │
│       │ VPC Peering │             │        │
└───────┼─────────────┼─────────────┼────────┘
        │             │             │
        └─────────────┴─────────────┘
                      │
        ┌─────────────┴──────────┐
        │   SCT Infrastructure   │
        │  ┌──────────┐          │
        │  │  Loader  │          │
        │  │  Nodes   │          │
        │  └──────────┘          │
        │  ┌──────────┐          │
        │  │ Monitor  │          │
        │  │  Node    │          │
        │  └──────────┘          │
        │  ┌──────────┐          │
        │  │  SCT     │          │
        │  │  Runner  │          │
        │  └──────────┘          │
        └────────────────────────┘

Additional resources