diff --git a/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-cloudrun.md b/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-cloudrun.md new file mode 100644 index 0000000000..44bdf96c70 --- /dev/null +++ b/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-cloudrun.md @@ -0,0 +1,181 @@ +--- +title: "Installing on Cloud Run" +linkTitle: "Installing on Cloud Run" +weight: 3 +description: > + This page describes how to install `piped` on Cloud Run. +--- + +## Prerequisites + +### A registered `piped` + +- Make sure your `piped` is registered in the Control Plane and that you have its **PIPED_ID** and **PIPED_KEY**. +- If not, follow the guide to [register a new `piped`](../../../user-guide/managing-controlplane/registering-a-piped/). + +### SSH key for Git repositories + +- If your Git repositories are private, `piped` requires a private SSH key to access those repositories. +- Please check out [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository’s **Settings** page.) + +## Installation + +### Preparing the `piped` configuration file + +Prepare a `piped` configuration file like the following: + +```yaml +apiVersion: pipecd.dev/v1beta1 +kind: Piped +spec: + projectID: {PROJECT_ID} + pipedID: {PIPED_ID} + pipedKeyData: {BASE64_ENCODED_PIPED_KEY} + # Write in a format like "host:443" because the communication is done via gRPC. + apiAddress: {CONTROL_PLANE_API_ADDRESS} + + git: + sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY} + + repositories: + - repoId: {REPO_ID_OR_NAME} + remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git + branch: {GIT_BRANCH} + + # Optional + # Enable this piped to handle Cloud Run applications. + platformProviders: + - name: cloudrun-in-project + type: CLOUDRUN + config: + project: {GCP_PROJECT_ID} + region: {GCP_PROJECT_REGION} + + # Optional + # Uncomment this if you want to enable this piped to handle Terraform applications. + # - name: terraform-gcp + # type: TERRAFORM + + # Optional + # Uncomment this if you want to enable Secret Management. + # See: https://pipecd.dev/docs/user-guide/managing-application/secret-management/ + # secretManagement: + # type: KEY_PAIR + # config: + # privateKeyData: {BASE64_ENCODED_PRIVATE_KEY} + # publicKeyData: {BASE64_ENCODED_PUBLIC_KEY} +``` + +See the [configuration reference](../../../user-guide/managing-piped/configuration-reference/) for the full list of available fields. + +### Storing the configuration in Secret Manager + +Create a new secret in [Secret Manager](https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets) to store the configuration securely: + +```console +gcloud secrets create cloudrun-piped-config --data-file={PATH_TO_CONFIG_FILE} +``` + +Then make sure that Cloud Run has permission to access that secret as described in the [Cloud Run secret access guide](https://cloud.google.com/run/docs/configuring/secrets#access-secret). + +### Running `piped` on Cloud Run + +Prepare a Cloud Run service manifest as below. + +{{< tabpane >}} +{{< tab lang="yaml" header="Piped with Remote-upgrade" >}} +# Enable remote-upgrade feature of piped. +# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade +# This allows upgrading piped to a new version from the web console. + +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: piped + annotations: + run.googleapis.com/ingress: internal + run.googleapis.com/ingress-status: internal +spec: + template: + metadata: + annotations: + autoscaling.knative.dev/maxScale: "1" # This must be 1. + autoscaling.knative.dev/minScale: "1" # This must be 1. + run.googleapis.com/cpu-throttling: "false" # This is required. + spec: + containerConcurrency: 1 # This must be 1 to ensure piped works correctly. + containers: + - image: gcr.io/pipecd/launcher:{{< blocks/latest_version >}} + args: + - launcher + - --launcher-admin-port=9086 + - --config-file=/etc/piped-config/config.yaml + ports: + - containerPort: 9086 + volumeMounts: + - mountPath: /etc/piped-config + name: piped-config + resources: + limits: + cpu: 1000m + memory: 2Gi + volumes: + - name: piped-config + secret: + secretName: cloudrun-piped-config + items: + - key: latest + path: config.yaml +{{< /tab >}} + +{{< tab lang="yaml" header="Piped without Remote-upgrade" >}} +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: piped + annotations: + run.googleapis.com/ingress: internal + run.googleapis.com/ingress-status: internal +spec: + template: + metadata: + annotations: + autoscaling.knative.dev/maxScale: "1" + autoscaling.knative.dev/minScale: "1" + run.googleapis.com/cpu-throttling: "false" + spec: + containerConcurrency: 1 + containers: + - image: gcr.io/pipecd/launcher:{{< blocks/latest_version >}} + args: + - launcher + - --launcher-admin-port=9086 + - --config-file=/etc/piped-config/config.yaml + ports: + - containerPort: 9086 + volumeMounts: + - mountPath: /etc/piped-config + name: piped-config + resources: + limits: + cpu: 1000m + memory: 2Gi + volumes: + - name: piped-config + secret: + secretName: cloudrun-piped-config + items: + - key: latest + path: config.yaml +{{< /tab >}} +{{< /tabpane >}} + +Apply the Cloud Run service: + +```console +gcloud run services replace {PATH_TO_CLOUD_RUN_SERVICE_MANIFEST} +``` + +Once the service is created, Cloud Run will run the `piped` agent as a stateless service that connects to your PipeCD Control Plane and deploys applications according to your configuration. Make sure that the created secret is accessible from this `piped` service as described in the [Cloud Run secret access guide](https://cloud.google.com/run/docs/configuring/secrets#access-secret). + + diff --git a/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-fargate.md b/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-fargate.md new file mode 100644 index 0000000000..4e6b79c45a --- /dev/null +++ b/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-fargate.md @@ -0,0 +1,212 @@ +--- +title: "Installing on ECS Fargate" +linkTitle: "Installing on ECS Fargate" +weight: 4 +description: > + This page describes how to install `piped` as a task on an ECS cluster backed by AWS Fargate. +--- + +## Prerequisites + +### A registered `piped` + +- Make sure your `piped` is registered in the Control Plane and that you have its **PIPED_ID** and **PIPED_KEY**. +- If not, follow the guide to [register a new `piped`](../../../user-guide/managing-controlplane/registering-a-piped/). + +### SSH key for Git repositories + +- If your Git repositories are private, `piped` requires a private SSH key to access those repositories. +- Please check out [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository's **Settings** page.) + +## Installation + +### Preparing the `piped` configuration file + +Prepare a `piped` configuration file as follows: + +```yaml +apiVersion: pipecd.dev/v1beta1 +kind: Piped +spec: + projectID: {PROJECT_ID} + pipedID: {PIPED_ID} + pipedKeyData: {BASE64_ENCODED_PIPED_KEY} + # Write in a format like "host:443" because the communication is done via gRPC. + apiAddress: {CONTROL_PLANE_API_ADDRESS} + + git: + sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY} + + repositories: + - repoId: {REPO_ID_OR_NAME} + remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git + branch: {GIT_BRANCH} + + # Optional + # Enable this piped to handle ECS applications. + platformProviders: + - name: ecs-dev + type: ECS + config: + region: {ECS_RUNNING_REGION} + + # Optional + # Uncomment this if you want to enable this piped to handle Terraform applications. + # - name: terraform-dev + # type: TERRAFORM + + # Optional + # Uncomment this if you want to enable Secret Management. + # See: https://pipecd.dev/docs/user-guide/managing-application/secret-management/ + # secretManagement: + # type: KEY_PAIR + # config: + # privateKeyData: {BASE64_ENCODED_PRIVATE_KEY} + # publicKeyData: {BASE64_ENCODED_PUBLIC_KEY} +``` + +See the [configuration reference](../../../user-guide/managing-piped/configuration-reference/) for the full configuration. + +### Storing the configuration in AWS + +Store the above configuration data in AWS so it can be referenced from your Fargate task. You can use either [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) or [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html). + +{{< tabpane >}} +{{< tab lang="bash" header="Store in AWS Secrets Manager" >}} +aws secretsmanager create-secret --name PipedConfig \ + --description "Configuration of piped running as ECS Fargate task" \ + --secret-string `base64 piped-config.yaml` +{{< /tab >}} +{{< tab lang="bash" header="Store in AWS Systems Manager Parameter Store" >}} +aws ssm put-parameter \ + --name PipedConfig \ + --value `base64 piped-config.yaml` \ + --type SecureString +{{< /tab >}} +{{< /tabpane >}} + +If you use AWS Secrets Manager, make sure your task role or execution role has permission to read the secret. See [Required IAM permissions for Amazon ECS secrets](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-secrets.html) for more details. + +### Defining the task definition + +Prepare a task definition for your `piped` task. The following examples show how to configure `piped` to read its configuration from AWS Secrets Manager. + +{{< tabpane >}} +{{< tab lang="json" header="Piped with Remote-upgrade" >}} +# Enable remote-upgrade feature of piped. +# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade +# This allows upgrading piped to a new version from the web console. + +{ + "family": "piped", + "executionRoleArn": "{PIPED_TASK_EXECUTION_ROLE_ARN}", + "containerDefinitions": [ + { + "name": "piped", + "essential": true, + "image": "ghcr.io/pipe-cd/launcher:{{< blocks/latest_version >}}", + "entryPoint": [ + "sh", + "-c" + ], + "command": [ + "/bin/sh -c \"launcher launcher --config-data=$(echo $CONFIG_DATA)\"" + ], + "secrets": [ + { + "valueFrom": "{PIPED_SECRET_MANAGER_ARN}", + "name": "CONFIG_DATA" + } + ] + } + ], + "requiresCompatibilities": [ + "FARGATE" + ], + "networkMode": "awsvpc", + "memory": "512", + "cpu": "256" +} +{{< /tab >}} +{{< tab lang="json" header="Piped" >}} +# This just installs a piped with the specified version. +# Whenever you want to upgrade that piped to a new version or update its config data you have to restart it. + +{ + "family": "piped", + "executionRoleArn": "{PIPED_TASK_EXECUTION_ROLE_ARN}", + "containerDefinitions": [ + { + "name": "piped", + "essential": true, + "image": "ghcr.io/pipe-cd/piped:{{< blocks/latest_version >}}", + "entryPoint": [ + "sh", + "-c" + ], + "command": [ + "/bin/sh -c \"piped piped --config-data=$(echo $CONFIG_DATA)\"" + ], + "secrets": [ + { + "valueFrom": "{PIPED_SECRET_MANAGER_ARN}", + "name": "CONFIG_DATA" + } + ] + } + ], + "requiresCompatibilities": [ + "FARGATE" + ], + "networkMode": "awsvpc", + "memory": "512", + "cpu": "256" +} +{{< /tab >}} +{{< /tabpane >}} + +Register the task definition and run a `piped` task: + +```console +aws ecs register-task-definition --cli-input-json file://taskdef.json +aws ecs run-task --cluster {ECS_CLUSTER} --task-definition piped +``` + +Once the task is created, it will run continuously because of the `piped` execution. Since this task is run without [`startedBy`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_StartTask.html#API_StartTask_RequestSyntax), if `piped` stops, it will not automatically be restarted. To keep `piped` running, define an ECS service to control the task deployment. + +### Defining the ECS service + +The following is a sample ECS service definition to control the `piped` task deployment: + +```json +{ + "cluster": "{ECS_CLUSTER}", + "serviceName": "piped", + "desiredCount": 1, + "taskDefinition": "{PIPED_TASK_DEFINITION_ARN}", + "deploymentConfiguration": { + "minimumHealthyPercent": 0, + "maximumPercent": 100 + }, + "schedulingStrategy": "REPLICA", + "launchType": "FARGATE", + "networkConfiguration": { + "awsvpcConfiguration": { + "assignPublicIp": "ENABLED", + "...": "..." + } + } +} +``` + +Then create the ECS service: + +```console +aws ecs create-service \ + --cluster {ECS_CLUSTER} \ + --service-name piped \ + --cli-input-json file://service.json +``` + +When the service is running, ECS will ensure that exactly one `piped` task is running (because `desiredCount` is 1), keeping your `piped` agent available on Fargate. + diff --git a/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-google-cloud-vm.md b/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-google-cloud-vm.md new file mode 100644 index 0000000000..49c8c33134 --- /dev/null +++ b/docs/content/en/docs-v1.0.x/installation/install-piped/installing-on-google-cloud-vm.md @@ -0,0 +1,148 @@ +--- +title: "Installing on Google Cloud VM" +linkTitle: "Installing on Google Cloud VM" +weight: 2 +description: > + This page describes how to install `piped` on a Google Cloud VM. +--- + +## Prerequisites + +### A registered `piped` + +- Make sure your `piped` is registered in the Control Plane and that you have its **PIPED_ID** and **PIPED_KEY**. +- If not, follow the guide to [register a new `piped`](../../../user-guide/managing-controlplane/registering-a-piped/). + +### SSH key for Git repositories + +- If your Git repositories are private, `piped` requires a private SSH key to access those repositories. +- Please check out [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository's **Settings** page.) + +## Installation + +### Preparing the `piped` configuration file + +Prepare a `piped` configuration file as the following: + +```yaml +apiVersion: pipecd.dev/v1beta1 +kind: Piped +spec: + projectID: {PROJECT_ID} + pipedID: {PIPED_ID} + pipedKeyData: {BASE64_ENCODED_PIPED_KEY} + # Write in a format like "host:443" because the communication is done via gRPC. + apiAddress: {CONTROL_PLANE_API_ADDRESS} + + git: + sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY} + + repositories: + - repoId: {REPO_ID_OR_NAME} + remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git + branch: {GIT_BRANCH} + + # Optional + # Uncomment this if you want to enable this piped to handle Cloud Run applications. + # platformProviders: + # - name: cloudrun-in-project + # type: CLOUDRUN + # config: + # project: {GCP_PROJECT_ID} + # region: {GCP_PROJECT_REGION} + + # Optional + # Uncomment this if you want to enable this piped to handle Terraform applications. + # - name: terraform-gcp + # type: TERRAFORM + + # Optional + # Uncomment this if you want to enable Secret Management. + # See: https://pipecd.dev/docs/user-guide/managing-application/secret-management/ + # secretManagement: + # type: KEY_PAIR + # config: + # privateKeyData: {BASE64_ENCODED_PRIVATE_KEY} + # publicKeyData: {BASE64_ENCODED_PUBLIC_KEY} +``` + +See the [configuration reference](../../../user-guide/managing-piped/configuration-reference/) for the full configuration. + +### Creating a Secret Manager secret + +Create a new secret in [Secret Manager](https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets) to store the configuration securely: + +```shell +gcloud secrets create vm-piped-config --data-file={PATH_TO_CONFIG_FILE} +``` + +### Creating a service account and granting roles + +Create a service account for `piped` and grant it the required roles: + +```shell +gcloud iam service-accounts create vm-piped \ + --description="Used by piped running on Google Cloud VM" \ + --display-name="vm-piped" + +# Allow piped to access the created secret. +gcloud secrets add-iam-policy-binding vm-piped-config \ + --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \ + --role="roles/secretmanager.secretAccessor" + +# Allow piped to write its log messages to Google Cloud Logging. +gcloud projects add-iam-policy-binding {GCP_PROJECT_ID} \ + --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \ + --role="roles/logging.logWriter" + +# Optional +# If you want to use this piped to handle Cloud Run applications, +# grant additional roles as described in the Cloud Run IAM documentation: +# https://cloud.google.com/run/docs/reference/iam/roles#additional-configuration +# +# gcloud projects add-iam-policy-binding {GCP_PROJECT_ID} \ +# --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \ +# --role="roles/run.developer" +# +# gcloud iam service-accounts add-iam-policy-binding {GCP_PROJECT_NUMBER}-compute@developer.gserviceaccount.com \ +# --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \ +# --role="roles/iam.serviceAccountUser" +``` + +### Running `piped` on a Google Cloud VM + +Use `gcloud compute instances create-with-container` to run `piped` as a container on a VM. The following examples show how to configure the instance to read the configuration from Secret Manager. + +{{< tabpane >}} +{{< tab lang="console" header="Piped with Remote-upgrade" >}} +# Enable remote-upgrade feature of piped. +# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade +# This allows upgrading piped to a new version from the web console. + +gcloud compute instances create-with-container vm-piped \ + --container-image="ghcr.io/pipe-cd/launcher:{{< blocks/latest_version >}}" \ + --container-arg="launcher" \ + --container-arg="--config-from-gcp-secret=true" \ + --container-arg="--gcp-secret-id=projects/{GCP_PROJECT_ID}/secrets/vm-piped-config/versions/{SECRET_VERSION}" \ + --network="{VPC_NETWORK}" \ + --subnet="{VPC_SUBNET}" \ + --scopes="cloud-platform" \ + --service-account="vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" +{{< /tab >}} +{{< tab lang="console" header="Piped" >}} +# This just installs a piped with the specified version. +# Whenever you want to upgrade that piped to a new version or update its config data you have to restart it. + +gcloud compute instances create-with-container vm-piped \ + --container-image="ghcr.io/pipe-cd/piped:{{< blocks/latest_version >}}" \ + --container-arg="piped" \ + --container-arg="--config-gcp-secret=projects/{GCP_PROJECT_ID}/secrets/vm-piped-config/versions/{SECRET_VERSION}" \ + --network="{VPC_NETWORK}" \ + --subnet="{VPC_SUBNET}" \ + --scopes="cloud-platform" \ + --service-account="vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" +{{< /tab >}} +{{< /tabpane >}} + +After that, you should see on the PipeCD web `Settings` page that `piped` is connected to the Control Plane. You can also view `piped` logs as described in the [Compute Engine container logs documentation](https://cloud.google.com/compute/docs/containers/deploying-containers#viewing_logs). +