Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions pages/managed-inference/how-to/create-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ dates:
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization

1. Click the **AI** section of the [Scaleway console](https://console.scaleway.com/), and select **Managed Inference** from the side menu to access the Managed Inference dashboard.
2. Click **Deploy a model** to launch the model deployment wizard.
3. Provide the necessary information:
2. From the drop-down menu, select the geographical region where you want to create your deployment.
3. Click **Deploy a model** to launch the model deployment wizard.
4. Provide the necessary information:
- Select the desired model and quantization to use for your deployment [from the available options](/managed-inference/reference-content/).
<Message type="important">
Scaleway Managed Inference allows you to deploy various AI models, either from the Scaleway catalog or by importing a custom model. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation.
Expand All @@ -28,12 +29,12 @@ dates:
</Message>
- Choose the geographical **region** for the deployment.
- Specify the GPU Instance type to be used with your deployment.
4. Enter a **name** for the deployment, and optional tags.
5. Configure the **network connectivity** settings for the deployment:
5. Enter a **name** for the deployment, and optional tags.
6. Configure the **network connectivity** settings for the deployment:
- Attach to a **Private Network** for secure communication and restricted availability. Choose an existing Private Network from the drop-down list, or create a new one.
- Set up **Public connectivity** to access resources via the public internet. Authentication by API key is enabled by default.
<Message type="important">
- Enabling both private and public connectivity will result in two distinct endpoints (public and private) for your deployment.
- Deployments must have at least one endpoint, either public or private.
</Message>
6. Click **Deploy model** to launch the deployment process. Once the model is ready, it will be listed among your deployments.
7. Click **Deploy model** to launch the deployment process. Once the model is ready, it will be listed among your deployments.
9 changes: 5 additions & 4 deletions pages/managed-inference/how-to/delete-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,11 @@ Once you have finished your inference tasks you can delete your deployment. This
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization

1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
2. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Icon name="more" /> to access the deployment dashboard.
3. Click the **Settings** tab of your deployment to display additional settings.
4. Click **Delete deployment**.
5. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.
2. From the drop-down menu, select the geographical region you want to manage.
3. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Icon name="more" /> to access the deployment dashboard.
4. Click the **Settings** tab of your deployment to display additional settings.
5. Click **Delete deployment**.
6. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.

<Message type="important">
Deleting a deployment is a permanent action and will erase all its associated data.
Expand Down
17 changes: 9 additions & 8 deletions pages/managed-inference/how-to/import-custom-model.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,26 @@ Scaleway provides a selection of common models for deployment from the Scaleway
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) to perform actions in your Organization.

1. Click **Managed Inference** in the **AI** section of the side menu in the [Scaleway console](https://console.scaleway.com/) to access the dashboard.
2. Click **Deploy a model** to launch the model deployment wizard.
3. In the **Choose a model** section, select **Custom model**. If you have no model yet, click **Import a model** to start the model import wizard.
4. Choose an upload source:
2. From the drop-down menu, select the geographical region you want to manage.
3. Click **Deploy a model** to launch the model deployment wizard.
4. In the **Choose a model** section, select **Custom model**. If you have no model yet, click **Import a model** to start the model import wizard.
5. Choose an upload source:
- **Hugging Face**: Pull the model from Hugging Face.
- **Object Storage**: This feature is coming soon.
5. Enter your Hugging Face access token, which must have READ access to the repository.
6. Enter your Hugging Face access token, which must have READ access to the repository.
<Message type="note">
[Learn how to generate a Hugging Face access token](https://huggingface.co/docs/hub/security-tokens).
</Message>
6. Enter the name of the Hugging Face repository to pull the model from.
7. Enter the name of the Hugging Face repository to pull the model from.
<Message type="note">
Ensure you have access to gated models if applicable. Refer to the [Hugging Face documentation](https://huggingface.co/docs/hub/en/models-gated) for details.
</Message>
7. Choose a name for your model. The name must be unique within your Organization and Project and cannot be changed later.
8. Click **Verify import** to check your Hugging Face credentials and ensure model compatibility.
8. Choose a name for your model. The name must be unique within your Organization and Project and cannot be changed later.
9. Click **Verify import** to check your Hugging Face credentials and ensure model compatibility.
<Message type="tip">
For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation.
</Message>
9. Review the summary of your import, which includes:
10. Review the summary of your import, which includes:
- Context size by node type.
- Quantization options.
- Estimated cost.
Expand Down
9 changes: 5 additions & 4 deletions pages/managed-inference/how-to/manage-allowed-ips.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,14 @@ Allowed IPs restrict the IPs allowed to access your Managed Inference endpoints.
## How to allow an IP address to connect to a deployment

1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
2. Click a deployment name or <Icon name="more" /> > **More info** to access the deployment dashboard.
3. Click the **Security** tab and navigate to the **Allowed IPs** section. A list of your allowed IP addresses displays.
4. Click **Add allowed IP**. The IP can be a single IP or an IP block.
2. From the drop-down menu, select the geographical region you want to manage.
3. Click a deployment name or <Icon name="more" /> > **More info** to access the deployment dashboard.
4. Click the **Security** tab and navigate to the **Allowed IPs** section. A list of your allowed IP addresses displays.
5. Click **Add allowed IP**. The IP can be a single IP or an IP block.
<Message type="note">
The IP must be specified in CIDR format, i.e. `198.51.100.135/32` for a single IP or `198.51.100.0/24` for an IP block.
</Message>
5. Enter a single IP address or a subnetwork.
6. Enter a single IP address or a subnetwork.
<Message type="note">
To restore initial settings and allow connections from all IPs, delete all allowed IPs from the list.
</Message>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,25 +29,27 @@ Using a Private Network for communications between your Instances hosting your a
### Attaching a Private Network during deployment setup

1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
2. Navigate to the **Deployments** section and click **Create New Deployment**. The setup wizard displays.
3. During the [setup process](/managed-inference/how-to/create-deployment/), you access the **Networking** section.
4. You will be asked to **attach a Private Network**. Two options are available:
2. From the drop-down menu, select the geographical region you want to manage.
3. Navigate to the **Deployments** section and click **Create New Deployment**. The setup wizard displays.
4. During the [setup process](/managed-inference/how-to/create-deployment/), you access the **Networking** section.
5. You will be asked to **attach a Private Network**. Two options are available:
- **Attach an existing Private Network**: Select from the list of available networks.
- **Add a new Private Network**: Choose this option if you need to create a new network.
5. **Confirm your selection** and complete the deployment setup process.
6. **Confirm your selection** and complete the deployment setup process.

### Attaching a Private Network to an existing deployment

1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
2. Click a deployment name or <Icon name="more" /> > **More info** to access the deployment dashboard.
3. Go to the **Overview** tab and locate the **Endpoints** section.
4. Click **Attach Private Network**. Two options are available:
2. From the drop-down menu, select the geographical region you want to manage.
3. Click a deployment name or <Icon name="more" /> > **More info** to access the deployment dashboard.
4. Go to the **Overview** tab and locate the **Endpoints** section.
5. Click **Attach Private Network**. Two options are available:
- **Attach an existing Private Network**: Select from the list of available networks.
- **Add a new Private Network**: Choose this option if you need to create a new network.
<Message type="tip">
Alternatively, you can access the **Security tab** and attach a network from the **Private Network** section.
</Message>
5. **Save your changes** to apply the new network configuration.
6. **Save your changes** to apply the new network configuration.

### Verifying the Private Network connection

Expand Down
11 changes: 6 additions & 5 deletions pages/managed-inference/how-to/monitor-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ This documentation page shows you how to monitor your Managed Inference deployme
## How to monitor your LLM dashboard

1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
2. Click a deployment name or <Icon name="more" /> > **More info** to access the deployment dashboard.
3. Click the **Monitoring** tab of your deployment. The Cockpit overview displays.
4. Click **Open Grafana metrics dashboard** to open your Cockpit's Grafana interface.
5. Authenticate with your [Grafana credentials](/cockpit/how-to/retrieve-grafana-credentials/). The Grafana dashboard displays.
6. Select your Managed Inference dashboard from the [list of your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/) to visualize your metrics.
2. From the drop-down menu, select the geographical region you want to manage.
3. Click a deployment name or <Icon name="more" /> > **More info** to access the deployment dashboard.
4. Click the **Monitoring** tab of your deployment. The Cockpit overview displays.
5. Click **Open Grafana metrics dashboard** to open your Cockpit's Grafana interface.
6. Authenticate with your [Grafana credentials](/cockpit/how-to/retrieve-grafana-credentials/). The Grafana dashboard displays.
7. Select your Managed Inference dashboard from the [list of your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/) to visualize your metrics.
32 changes: 18 additions & 14 deletions pages/managed-inference/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ Here are some of the key features of Scaleway Managed Inference:
## How to create a Managed Inference deployment

1. Navigate to the **AI** section of the [Scaleway console](https://console.scaleway.com/), and select **Managed Inference** from the side menu to access the Managed Inference dashboard.
2. Click **Create deployment** to launch the deployment creation wizard.
3. Provide the necessary information:
2. From the drop-down menu, select the geographical region where you want to create your deployment.
3. Click **Create deployment** to launch the deployment creation wizard.
4. Provide the necessary information:
- Select the desired model and the quantization to use for your deployment [from the available options](/managed-inference/reference-content/).
<Message type="important">
Scaleway Managed Inference allows you to deploy various AI models, either from the Scaleway catalog or by importing a custom model. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation.
Expand All @@ -43,24 +44,25 @@ Here are some of the key features of Scaleway Managed Inference:
</Message>
- Choose the geographical **region** for the deployment.
- Specify the GPU Instance type to be used with your deployment.
4. Enter a **name** for the deployment, along with optional tags to aid in organization.
5. Configure the **network** settings for the deployment:
5. Enter a **name** for the deployment, along with optional tags to aid in organization.
6. Configure the **network** settings for the deployment:
- Enable **Private Network** for secure communication and restricted availability within Private Networks. Choose an existing Private Network from the drop-down list, or create a new one.
- Enable **Public Network** to access resources via the public Internet. API key protection is enabled by default.
<Message type="important">
- Enabling both private and public networks will result in two distinct endpoints (public and private) for your deployment.
- Deployments must have at least one endpoint, either public or private.
</Message>
6. Click **Create deployment** to launch the deployment process. Once the deployment is ready, it will be listed among your deployments.
7. Click **Create deployment** to launch the deployment process. Once the deployment is ready, it will be listed among your deployments.

## How to access a Managed Inference deployment

Managed Inference deployments have authentication enabled by default. As such, your endpoints expect a secret key generated with Scaleway's Identity and Access Management service (IAM) for authentication.

1. Click **Managed Inference** in the **AI** section of the side menu. The Managed Inference dashboard displays.
2. Click <Icon name="more" /> next to the deployment you want to edit. The deployment dashboard displays.
3. Click **Generate key** in the **Deployment connection** section of the dashboard. The token creation wizard displays.
4. Fill in the [required information for API key creation](/iam/how-to/create-api-keys/) and click **Generate API key**.
2. From the drop-down menu, select the geographical region where you want to manage.
3. Click <Icon name="more" /> next to the deployment you want to edit. The deployment dashboard displays.
4. Click **Generate key** in the **Deployment connection** section of the dashboard. The token creation wizard displays.
5. Fill in the [required information for API key creation](/iam/how-to/create-api-keys/) and click **Generate API key**.

<Message type="tip">
You have full control over authentication from the **Security** tab of your deployment. Authentication is enabled by default.
Expand All @@ -69,8 +71,9 @@ Managed Inference deployments have authentication enabled by default. As such, y
## How to interact with Managed Inference

1. Click **Managed Inference** in the **AI** section of the side menu. The Managed Inference dashboard displays.
2. Click <Icon name="more" /> next to the deployment you want to edit. The deployment dashboard displays.
3. Click the **Inference** tab. Code examples in various environments display. Copy and paste them into your code editor or terminal.
2. From the drop-down menu, select the geographical region where you want to manage.
3. Click <Icon name="more" /> next to the deployment you want to edit. The deployment dashboard displays.
4. Click the **Inference** tab. Code examples in various environments display. Copy and paste them into your code editor or terminal.

<Message type="note">
Prompt structure may vary from one model to another. Refer to the specific instructions for use in our [dedicated documentation](/managed-inference/reference-content/).
Expand All @@ -79,10 +82,11 @@ Managed Inference deployments have authentication enabled by default. As such, y
## How to delete a deployment

1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
2. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Icon name="more" /> to access the deployment dashboard.
3. Click the **Settings** tab of your deployment to display additional settings.
4. Click **Delete deployment**.
5. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.
2. From the drop-down menu, select the geographical region where you want to create your deployment.
3. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Icon name="more" /> to access the deployment dashboard.
4. Click the **Settings** tab of your deployment to display additional settings.
5. Click **Delete deployment**.
6. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.

<Message type="important">
Deleting a deployment is a permanent action, and will erase all its associated configuration and resources.
Expand Down
Loading