Skip to content

README.md

Latest commit

 

History

History
50 lines (39 loc) · 1.57 KB

README.md

File metadata and controls

50 lines (39 loc) · 1.57 KB

Scaleway Infrastructure Deployment

This guide describes how to provision the Scaleway infrastructure components required by the BigData Republic Open Data Platform and prepare Trino object storage configuration.

Overview

You will:

  1. Create / retrieve Scaleway API credentials
  2. Apply Terraform to provision the object storage and the Kubernetes cluster
  3. Prepare Trino configuration values and ENV variables
  4. Export kubeconfig environment variables for subsequent platform deployment

Prerequisites

Install required tooling:

brew install terraform

1. Create Account & Collect Credentials

  1. Go to https://www.scaleway.com/ and create an account (if not already done)
  2. Generate secret key via https://console.scaleway.com/iam/api-keys
  3. When creating the key copy the Terraform > scaleway.auto.tfvars part and store it in a file scaleway.tfvars in the secrets folder

2. Deploy Infrastructure

cd infra/scaleway

terraform init
terraform apply --vars-file=../../secrets/scaleway.tfvars

3. Prepare Trino Configuration and ENV Variables

cd ../../configs/trino

S3_ENDPOINT=https://iceberg.s3.nl-ams.scw.cloud \
S3_REGION=nl-ams \
S3_AWS_ACCESS_KEY=<read from terraform output> \
S3_AWS_SECRET_KEY=<read from terraform output> \
envsubst < values-template.yaml > values-scaleway.yaml

Prepare ENV Variables used for the platform deployment:

export KUBE_CONFIG_PATH=$(realpath ../../secrets/scaleway-kubeconfig.yaml)
export KUBECONFIG=$KUBE_CONFIG_PATH

Next Steps

Continue with the platform deployment