From 668001a50655aaba0d59200190b231bb6bd3492e Mon Sep 17 00:00:00 2001 From: Benedikt Rollik Date: Fri, 16 May 2025 14:51:57 +0200 Subject: [PATCH] feat(infr): add region selector step --- .../how-to/create-deployment.mdx | 11 ++++--- .../how-to/delete-deployment.mdx | 9 +++--- .../how-to/import-custom-model.mdx | 17 +++++----- .../how-to/manage-allowed-ips.mdx | 9 +++--- ...managed-inference-with-private-network.mdx | 18 ++++++----- .../how-to/monitor-deployment.mdx | 11 ++++--- pages/managed-inference/quickstart.mdx | 32 +++++++++++-------- 7 files changed, 59 insertions(+), 48 deletions(-) diff --git a/pages/managed-inference/how-to/create-deployment.mdx b/pages/managed-inference/how-to/create-deployment.mdx index ad5ed35260..f6a90ed12e 100644 --- a/pages/managed-inference/how-to/create-deployment.mdx +++ b/pages/managed-inference/how-to/create-deployment.mdx @@ -17,8 +17,9 @@ dates: - [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization 1. Click the **AI** section of the [Scaleway console](https://console.scaleway.com/), and select **Managed Inference** from the side menu to access the Managed Inference dashboard. -2. Click **Deploy a model** to launch the model deployment wizard. -3. Provide the necessary information: +2. From the drop-down menu, select the geographical region where you want to create your deployment. +3. Click **Deploy a model** to launch the model deployment wizard. +4. Provide the necessary information: - Select the desired model and quantization to use for your deployment [from the available options](/managed-inference/reference-content/). Scaleway Managed Inference allows you to deploy various AI models, either from the Scaleway catalog or by importing a custom model. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation. @@ -28,12 +29,12 @@ dates: - Choose the geographical **region** for the deployment. - Specify the GPU Instance type to be used with your deployment. -4. Enter a **name** for the deployment, and optional tags. -5. Configure the **network connectivity** settings for the deployment: +5. Enter a **name** for the deployment, and optional tags. +6. Configure the **network connectivity** settings for the deployment: - Attach to a **Private Network** for secure communication and restricted availability. Choose an existing Private Network from the drop-down list, or create a new one. - Set up **Public connectivity** to access resources via the public internet. Authentication by API key is enabled by default. - Enabling both private and public connectivity will result in two distinct endpoints (public and private) for your deployment. - Deployments must have at least one endpoint, either public or private. -6. Click **Deploy model** to launch the deployment process. Once the model is ready, it will be listed among your deployments. \ No newline at end of file +7. Click **Deploy model** to launch the deployment process. Once the model is ready, it will be listed among your deployments. \ No newline at end of file diff --git a/pages/managed-inference/how-to/delete-deployment.mdx b/pages/managed-inference/how-to/delete-deployment.mdx index 53169f93f4..39934c45f8 100644 --- a/pages/managed-inference/how-to/delete-deployment.mdx +++ b/pages/managed-inference/how-to/delete-deployment.mdx @@ -22,10 +22,11 @@ Once you have finished your inference tasks you can delete your deployment. This - [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization 1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays. -2. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon to access the deployment dashboard. -3. Click the **Settings** tab of your deployment to display additional settings. -4. Click **Delete deployment**. -5. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment. +2. From the drop-down menu, select the geographical region you want to manage. +3. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon to access the deployment dashboard. +4. Click the **Settings** tab of your deployment to display additional settings. +5. Click **Delete deployment**. +6. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment. Deleting a deployment is a permanent action and will erase all its associated data. diff --git a/pages/managed-inference/how-to/import-custom-model.mdx b/pages/managed-inference/how-to/import-custom-model.mdx index 3a23670902..c5f6040220 100644 --- a/pages/managed-inference/how-to/import-custom-model.mdx +++ b/pages/managed-inference/how-to/import-custom-model.mdx @@ -24,25 +24,26 @@ Scaleway provides a selection of common models for deployment from the Scaleway - [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) to perform actions in your Organization. 1. Click **Managed Inference** in the **AI** section of the side menu in the [Scaleway console](https://console.scaleway.com/) to access the dashboard. -2. Click **Deploy a model** to launch the model deployment wizard. -3. In the **Choose a model** section, select **Custom model**. If you have no model yet, click **Import a model** to start the model import wizard. -4. Choose an upload source: +2. From the drop-down menu, select the geographical region you want to manage. +3. Click **Deploy a model** to launch the model deployment wizard. +4. In the **Choose a model** section, select **Custom model**. If you have no model yet, click **Import a model** to start the model import wizard. +5. Choose an upload source: - **Hugging Face**: Pull the model from Hugging Face. - **Object Storage**: This feature is coming soon. -5. Enter your Hugging Face access token, which must have READ access to the repository. +6. Enter your Hugging Face access token, which must have READ access to the repository. [Learn how to generate a Hugging Face access token](https://huggingface.co/docs/hub/security-tokens). -6. Enter the name of the Hugging Face repository to pull the model from. +7. Enter the name of the Hugging Face repository to pull the model from. Ensure you have access to gated models if applicable. Refer to the [Hugging Face documentation](https://huggingface.co/docs/hub/en/models-gated) for details. -7. Choose a name for your model. The name must be unique within your Organization and Project and cannot be changed later. -8. Click **Verify import** to check your Hugging Face credentials and ensure model compatibility. +8. Choose a name for your model. The name must be unique within your Organization and Project and cannot be changed later. +9. Click **Verify import** to check your Hugging Face credentials and ensure model compatibility. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation. -9. Review the summary of your import, which includes: +10. Review the summary of your import, which includes: - Context size by node type. - Quantization options. - Estimated cost. diff --git a/pages/managed-inference/how-to/manage-allowed-ips.mdx b/pages/managed-inference/how-to/manage-allowed-ips.mdx index 0c31033ffa..c32e58f5af 100644 --- a/pages/managed-inference/how-to/manage-allowed-ips.mdx +++ b/pages/managed-inference/how-to/manage-allowed-ips.mdx @@ -28,13 +28,14 @@ Allowed IPs restrict the IPs allowed to access your Managed Inference endpoints. ## How to allow an IP address to connect to a deployment 1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays. -2. Click a deployment name or > **More info** to access the deployment dashboard. -3. Click the **Security** tab and navigate to the **Allowed IPs** section. A list of your allowed IP addresses displays. -4. Click **Add allowed IP**. The IP can be a single IP or an IP block. +2. From the drop-down menu, select the geographical region you want to manage. +3. Click a deployment name or > **More info** to access the deployment dashboard. +4. Click the **Security** tab and navigate to the **Allowed IPs** section. A list of your allowed IP addresses displays. +5. Click **Add allowed IP**. The IP can be a single IP or an IP block. The IP must be specified in CIDR format, i.e. `198.51.100.135/32` for a single IP or `198.51.100.0/24` for an IP block. -5. Enter a single IP address or a subnetwork. +6. Enter a single IP address or a subnetwork. To restore initial settings and allow connections from all IPs, delete all allowed IPs from the list. diff --git a/pages/managed-inference/how-to/managed-inference-with-private-network.mdx b/pages/managed-inference/how-to/managed-inference-with-private-network.mdx index c1e1e9a622..6d6fa62a07 100644 --- a/pages/managed-inference/how-to/managed-inference-with-private-network.mdx +++ b/pages/managed-inference/how-to/managed-inference-with-private-network.mdx @@ -29,25 +29,27 @@ Using a Private Network for communications between your Instances hosting your a ### Attaching a Private Network during deployment setup 1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays. -2. Navigate to the **Deployments** section and click **Create New Deployment**. The setup wizard displays. -3. During the [setup process](/managed-inference/how-to/create-deployment/), you access the **Networking** section. -4. You will be asked to **attach a Private Network**. Two options are available: +2. From the drop-down menu, select the geographical region you want to manage. +3. Navigate to the **Deployments** section and click **Create New Deployment**. The setup wizard displays. +4. During the [setup process](/managed-inference/how-to/create-deployment/), you access the **Networking** section. +5. You will be asked to **attach a Private Network**. Two options are available: - **Attach an existing Private Network**: Select from the list of available networks. - **Add a new Private Network**: Choose this option if you need to create a new network. -5. **Confirm your selection** and complete the deployment setup process. +6. **Confirm your selection** and complete the deployment setup process. ### Attaching a Private Network to an existing deployment 1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays. -2. Click a deployment name or > **More info** to access the deployment dashboard. -3. Go to the **Overview** tab and locate the **Endpoints** section. -4. Click **Attach Private Network**. Two options are available: +2. From the drop-down menu, select the geographical region you want to manage. +3. Click a deployment name or > **More info** to access the deployment dashboard. +4. Go to the **Overview** tab and locate the **Endpoints** section. +5. Click **Attach Private Network**. Two options are available: - **Attach an existing Private Network**: Select from the list of available networks. - **Add a new Private Network**: Choose this option if you need to create a new network. Alternatively, you can access the **Security tab** and attach a network from the **Private Network** section. -5. **Save your changes** to apply the new network configuration. +6. **Save your changes** to apply the new network configuration. ### Verifying the Private Network connection diff --git a/pages/managed-inference/how-to/monitor-deployment.mdx b/pages/managed-inference/how-to/monitor-deployment.mdx index 6dad20e243..17dec12511 100644 --- a/pages/managed-inference/how-to/monitor-deployment.mdx +++ b/pages/managed-inference/how-to/monitor-deployment.mdx @@ -24,8 +24,9 @@ This documentation page shows you how to monitor your Managed Inference deployme ## How to monitor your LLM dashboard 1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays. -2. Click a deployment name or > **More info** to access the deployment dashboard. -3. Click the **Monitoring** tab of your deployment. The Cockpit overview displays. -4. Click **Open Grafana metrics dashboard** to open your Cockpit's Grafana interface. -5. Authenticate with your [Grafana credentials](/cockpit/how-to/retrieve-grafana-credentials/). The Grafana dashboard displays. -6. Select your Managed Inference dashboard from the [list of your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/) to visualize your metrics. \ No newline at end of file +2. From the drop-down menu, select the geographical region you want to manage. +3. Click a deployment name or > **More info** to access the deployment dashboard. +4. Click the **Monitoring** tab of your deployment. The Cockpit overview displays. +5. Click **Open Grafana metrics dashboard** to open your Cockpit's Grafana interface. +6. Authenticate with your [Grafana credentials](/cockpit/how-to/retrieve-grafana-credentials/). The Grafana dashboard displays. +7. Select your Managed Inference dashboard from the [list of your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/) to visualize your metrics. \ No newline at end of file diff --git a/pages/managed-inference/quickstart.mdx b/pages/managed-inference/quickstart.mdx index d95f94faeb..1648b9e3e3 100644 --- a/pages/managed-inference/quickstart.mdx +++ b/pages/managed-inference/quickstart.mdx @@ -32,8 +32,9 @@ Here are some of the key features of Scaleway Managed Inference: ## How to create a Managed Inference deployment 1. Navigate to the **AI** section of the [Scaleway console](https://console.scaleway.com/), and select **Managed Inference** from the side menu to access the Managed Inference dashboard. -2. Click **Create deployment** to launch the deployment creation wizard. -3. Provide the necessary information: +2. From the drop-down menu, select the geographical region where you want to create your deployment. +3. Click **Create deployment** to launch the deployment creation wizard. +4. Provide the necessary information: - Select the desired model and the quantization to use for your deployment [from the available options](/managed-inference/reference-content/). Scaleway Managed Inference allows you to deploy various AI models, either from the Scaleway catalog or by importing a custom model. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation. @@ -43,24 +44,25 @@ Here are some of the key features of Scaleway Managed Inference: - Choose the geographical **region** for the deployment. - Specify the GPU Instance type to be used with your deployment. -4. Enter a **name** for the deployment, along with optional tags to aid in organization. -5. Configure the **network** settings for the deployment: +5. Enter a **name** for the deployment, along with optional tags to aid in organization. +6. Configure the **network** settings for the deployment: - Enable **Private Network** for secure communication and restricted availability within Private Networks. Choose an existing Private Network from the drop-down list, or create a new one. - Enable **Public Network** to access resources via the public Internet. API key protection is enabled by default. - Enabling both private and public networks will result in two distinct endpoints (public and private) for your deployment. - Deployments must have at least one endpoint, either public or private. -6. Click **Create deployment** to launch the deployment process. Once the deployment is ready, it will be listed among your deployments. +7. Click **Create deployment** to launch the deployment process. Once the deployment is ready, it will be listed among your deployments. ## How to access a Managed Inference deployment Managed Inference deployments have authentication enabled by default. As such, your endpoints expect a secret key generated with Scaleway's Identity and Access Management service (IAM) for authentication. 1. Click **Managed Inference** in the **AI** section of the side menu. The Managed Inference dashboard displays. -2. Click next to the deployment you want to edit. The deployment dashboard displays. -3. Click **Generate key** in the **Deployment connection** section of the dashboard. The token creation wizard displays. -4. Fill in the [required information for API key creation](/iam/how-to/create-api-keys/) and click **Generate API key**. +2. From the drop-down menu, select the geographical region where you want to manage. +3. Click next to the deployment you want to edit. The deployment dashboard displays. +4. Click **Generate key** in the **Deployment connection** section of the dashboard. The token creation wizard displays. +5. Fill in the [required information for API key creation](/iam/how-to/create-api-keys/) and click **Generate API key**. You have full control over authentication from the **Security** tab of your deployment. Authentication is enabled by default. @@ -69,8 +71,9 @@ Managed Inference deployments have authentication enabled by default. As such, y ## How to interact with Managed Inference 1. Click **Managed Inference** in the **AI** section of the side menu. The Managed Inference dashboard displays. -2. Click next to the deployment you want to edit. The deployment dashboard displays. -3. Click the **Inference** tab. Code examples in various environments display. Copy and paste them into your code editor or terminal. +2. From the drop-down menu, select the geographical region where you want to manage. +3. Click next to the deployment you want to edit. The deployment dashboard displays. +4. Click the **Inference** tab. Code examples in various environments display. Copy and paste them into your code editor or terminal. Prompt structure may vary from one model to another. Refer to the specific instructions for use in our [dedicated documentation](/managed-inference/reference-content/). @@ -79,10 +82,11 @@ Managed Inference deployments have authentication enabled by default. As such, y ## How to delete a deployment 1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays. -2. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon to access the deployment dashboard. -3. Click the **Settings** tab of your deployment to display additional settings. -4. Click **Delete deployment**. -5. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment. +2. From the drop-down menu, select the geographical region where you want to create your deployment. +3. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon to access the deployment dashboard. +4. Click the **Settings** tab of your deployment to display additional settings. +5. Click **Delete deployment**. +6. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment. Deleting a deployment is a permanent action, and will erase all its associated configuration and resources.