You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please read the [AWS Provider](../providers/aws/aws.md) documentation for more details about how the AWS provider works and how to get started.
33
33
34
-
:::success AWS Free Tier & Credits
35
-
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an elligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
34
+
:::tip[AWS Free Tier & Credits]
35
+
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an eligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
36
36
:::
37
37
38
38
## DigitalOcean
@@ -44,8 +44,8 @@ The Defang DigitalOcean Provider is available for Public Preview as of October 2
44
44
45
45
Please read the [DigitalOcean Provider](../providers/digitalocean/digitalocean.md) documentation for more details about how the DigitalOcean provider works and how to get started.
46
46
47
-
:::success DigitalOcean Credits
48
-
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an elligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
47
+
:::tip[DigitalOcean Credits]
48
+
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an eligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
49
49
:::
50
50
51
51
## GCP
@@ -56,11 +56,10 @@ The Defang GCP Provider is available for Public Preview as of December 2024.
56
56
57
57
Please check out the [GCP Provider](../providers/gcp/) page for more details.
58
58
59
-
:::success GCP Free Tier & Credits
60
-
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an elligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
59
+
:::tip[GCP Free Tier & Credits]
60
+
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an eligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
Copy file name to clipboardExpand all lines: docs/concepts/managed-storage/managed-object-storage.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ sidebar_position: 3000
6
6
7
7
# Managed Object Storage
8
8
9
-
:::infoNot Yet Supported
9
+
:::info[Not Yet Supported]
10
10
As of September 2024, Defang does not yet support managed Object Storage, but it is on our roadmap. If you are interested in Object Storage support, please vote on [this issue](https://github.com/DefangLabs/defang/issues/688).
Copy file name to clipboardExpand all lines: docs/concepts/managed-storage/managed-postgres.mdx
+4-2Lines changed: 4 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,6 +39,7 @@ You can also set the following optional environment variables to configure the m
39
39
You can connect to the managed Postgres instance using the name of your service as the hostname, `POSTGRES_USER`, `POSTGRES_DB`, and `POSTGRES_PASSWORD` environment variables.
40
40
41
41
### Example
42
+
42
43
:::info
43
44
For a smoother experience with Defang, we recommend using Postgres 14 for your container images. This version provides easier access and improved usability.
44
45
:::
@@ -83,7 +84,7 @@ When a project is deployed to a [production environment](/docs/concepts/deployme
83
84
84
85
The AWS Console can be used to restore a snapshot into a new instance of Postgres.
85
86
86
-
<!--
87
+
{/*
87
88
### Major Version Updating of Engine
88
89
89
90
To update the database engine you can update the image to a later version in your Compose file and apply it via ```defang compose up --provider=aws```. In the example below, we change from Postgres 15 to 16.
Copy file name to clipboardExpand all lines: docs/concepts/managed-storage/managed-storage.md
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,5 +6,4 @@ sidebar_position: 000
6
6
7
7
# Managed Storage
8
8
9
-
Defang helps you provision the infrastructure you need to run your services. That infrastructure is designed to scale in and out without persistent storage, so you can build highly scalable services. But Defang can also help you provision managed services to store and persist your data, like [caches](./managed-redis.md), [databases](./managed-postgres.md), and [object storage](./managed-object-storage.md).
10
-
9
+
Defang helps you provision the infrastructure you need to run your services. That infrastructure is designed to scale in and out without persistent storage, so you can build highly scalable services. But Defang can also help you provision managed services to store and persist your data, like [caches](./managed-redis.md), [databases](./managed-postgres.mdx), and [object storage](./managed-object-storage.md).
Copy file name to clipboardExpand all lines: docs/concepts/resources.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ services:
19
19
replicas: 3
20
20
resources:
21
21
reservations:
22
-
cpus: '1.0'
22
+
cpus: "1.0"
23
23
memory: 2048M
24
24
devices:
25
25
- capabilities: ["gpu"]
@@ -32,17 +32,17 @@ const service = new defang.DefangService("gpu-service", {
32
32
deploy: {
33
33
replicas: 3,
34
34
resources: {
35
-
reservations: {
36
-
cpu: 1.0,
37
-
memory: 2048,
38
-
devices: [{capabilities: ['gpu']}]
39
-
}
40
-
}
41
-
}
35
+
reservations: {
36
+
cpu: 1.0,
37
+
memory: 2048,
38
+
devices: [{capabilities: ["gpu"] }],
39
+
},
40
+
},
41
+
},
42
42
});
43
43
```
44
44
45
-
:::infoGPUs
45
+
:::info[GPUs]
46
46
If you require access to GPUs, you can specify this in the `deploy.resources.reservations.devices[0].capabilities` section of your service as in the examples above. You can learn more about this in the [Docker-Compose documentation](https://docs.docker.com/compose/gpu-support/). This is the only supported value in the `deploy.resources.reservations.devices` section.
47
47
:::
48
48
@@ -68,7 +68,7 @@ Defang uses `shm_size` to configure both the memory and disk space available to
68
68
The default `shm_size` values for each platform are as follows. More or less may be specified.
Copy file name to clipboardExpand all lines: docs/providers/aws/aws.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,8 +8,8 @@ sidebar_position: 000
8
8
9
9
Why should you use Defang with AWS? Defang allows you to easily create and manage full, scalable applications with AWS. Defang aims to make it easier to deploy your services to the cloud. Don't waste your time learning the ins and outs of AWS, deciding which of the 200+ services to use, and then writing the infrastructure code to deploy your services, and making sure they are properly secured. Defang does all of that for you.
10
10
11
-
:::success AWS Free Tier & Credits
12
-
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an elligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
11
+
:::tip[AWS Free Tier & Credits]
12
+
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an eligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
13
13
:::
14
14
15
15
## Getting Started
@@ -66,7 +66,7 @@ Defang can help you provision [managed storage](/docs/concepts/managed-storage/m
66
66
67
67
### Managed Postgres
68
68
69
-
When using [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.md), the Defang CLI provisions an RDS Postgres instance in your account.
69
+
When using [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.mdx), the Defang CLI provisions an RDS Postgres instance in your account.
Copy file name to clipboardExpand all lines: docs/providers/digitalocean/digitalocean.md
+8-3Lines changed: 8 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,8 +10,8 @@ sidebar_position: 010
10
10
The Defang DigitalOcean Provider is available for Public Preview as of October 2024.
11
11
:::
12
12
13
-
:::success DigitalOcean Credits
14
-
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an elligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
13
+
:::tip[DigitalOcean Credits]
14
+
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an eligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
15
15
:::
16
16
17
17
Why should you use Defang with DigitalOcean? Defang allows you to easily create and manage full, scalable applications with DigitalOcean. Defang aims to make it easier to deploy your services to the cloud. DigitalOcean is one of the most popular cloud providers in the world and with Defang, you can bypass the complexities of the DigitalOcean platform. Let Defang do it for you and spend more time working on what's important to you!
@@ -23,12 +23,15 @@ Why should you use Defang with DigitalOcean? Defang allows you to easily create
23
23
To get started with the Defang BYOC DigitalOcean Provider, first [install the latest version of the Defang CLI](../getting-started#authenticate-with-defang).
24
24
25
25
### Sign up for DigitalOcean
26
+
26
27
Next, make sure you have signed up for a [DigitalOcean account](https://try.digitalocean.com/freetrialoffer/).
27
28
28
29
### Authenticate with DigitalOcean
30
+
29
31
After signing up for your account, be sure to set up your [personal access token](https://docs.digitalocean.com/reference/api/create-personal-access-token/). Defang will need to find this value in your shell as the `DIGITALOCEAN_TOKEN` environment variable.
30
32
31
33
### Authenticate with DigitalOcean Spaces
34
+
32
35
You will also need a [DigitalOcean Spaces access key](https://docs.digitalocean.com/products/spaces/how-to/manage-access/). Defang will need to find this value in your shell as the `SPACES_ACCESS_KEY_ID`, and `SPACES_SECRET_ACCESS_KEY` environment variables.
33
36
34
37
### Configure your shell environment
@@ -44,6 +47,7 @@ The Defang CLI will automatically check if these envinonment variables are set b
44
47
### Deploy your project to DigitalOcean
45
48
46
49
Once you are ready to go, add the `--provider=digitalocean` to your command to tell the Defang CLI to use the DigitalOcean provider or set the `DEFANG_PROVIDER` environment variable to `digitalocean`.
50
+
47
51
```bash
48
52
$ defang compose up --provider=digitalocean
49
53
# or
@@ -73,9 +77,10 @@ Defang allows you to configure your services with [sensitive config values](http
73
77
### Future Improvements
74
78
75
79
The following features are still in development for DigitalOcean:
Copy file name to clipboardExpand all lines: docs/providers/gcp.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,8 +10,8 @@ sidebar_position: 3000
10
10
The Defang GCP Provider is available for Public Preview as of December 2024.
11
11
:::
12
12
13
-
:::success GCP Free Tier & Credits
14
-
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an elligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
13
+
:::tip[GCP Free Tier & Credits]
14
+
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an eligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
15
15
:::
16
16
17
17
Defang enables you to effortlessly develop and deploy full, scalable applications with GCP. It is designed to simplify deploying your services to the cloud. As one of the leading cloud providers globally, GCP offers powerful tools and resources, and with Defang, you can bypass the complexities of the GCP platform. Let Defang handle the heavy lifting so you can focus on what matters most to you!
@@ -66,10 +66,11 @@ Defang offers integration with managed, cloud-native large language model servic
66
66
### Future Improvements
67
67
68
68
The following features are in active development for GCP:
69
+
69
70
-[Configuration and management of secrets](/docs/concepts//configuration.md)
70
71
-[Networking and Load Balancing](/docs/concepts//networking.mdx)
Copy file name to clipboardExpand all lines: docs/tutorials/deploy-with-gpu.mdx
+48-38Lines changed: 48 additions & 38 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,27 +5,36 @@ sidebar_position: 500
5
5
6
6
# Deploy a GPU-Powered Application to AWS
7
7
8
-
This tutorial will show you how to create and deploy a GPU-powered application on AWS using Defang.
8
+
This tutorial will show you how to create and deploy a GPU-powered application on AWS using Defang.
9
9
10
10
We will walk you through the whole deployment process based on the [Mistral & vLLM](https://github.com/DefangLabs/samples/tree/main/samples/vllm) sample.
11
11
12
12
:::info
13
13
Note that GPU deployments are not permitted on the Defang Playground. You must [upgrade to a paid account](https://defang.io/pricing/) and [deploy to your own cloud account](https://docs.defang.io/docs/tutorials/deploy-to-your-cloud).
14
14
:::
15
15
16
-
<iframewidth="560"height="315"src="https://www.youtube.com/embed/kynFa2zU7hQ?si=qdV0xa6vkhMFJ6qv"title="YouTube video player"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"referrerpolicy="strict-origin-when-cross-origin"allowfullscreen></iframe>
For any of this to work, you'll need to have access to GPU instances in your AWS account. To do that you'll need to go to the "[Service Quotas](https://console.aws.amazon.com/servicequotas/home)" console in your AWS account. From there you can request access to spot GPU instances. You'll need to request 8 or more because the value is per vCPU and the smallest GPU instance has 8 vCPUs. The instance types you're requesting are "All G and VT spot instances".
27
36
28
-
:::warningTiming
37
+
:::warning[Timing]
29
38
This process can take a few days for AWS to approve.
30
39
:::
31
40
@@ -41,27 +50,29 @@ You'll need to clone the [Mistral & vLLM](https://github.com/DefangLabs/samples/
41
50
42
51
## Step 2 - Check your [Defang BYOC](../concepts/defang-byoc.md) settings
43
52
44
-
* Make sure you [install the latest version of the Defang CLI](../getting-started#install-the-defang-cli.md)
45
-
* Then, make sure you have properly [authenticated your AWS account](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). The Defang CLI makes use of AWS environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`, so make sure the correct values are set for those.
53
+
- Make sure you [install the latest version of the Defang CLI](../getting-started#install-the-defang-cli.md)
54
+
- Then, make sure you have properly [authenticated your AWS account](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). The Defang CLI makes use of AWS environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`, so make sure the correct values are set for those.
46
55
47
56
:::tip
48
57
If you have the AWS CLI installed (which is not required in order to use the Defang CLI), you can verify that you've authenticated to AWS by running `aws sts get-caller-identity` and see your account ID.
49
58
:::
50
59
51
60
## Step 3 - Prepare your Environment
52
-
* Log into your Defang account
61
+
62
+
- Log into your Defang account
63
+
53
64
```bash
54
65
defang login
55
66
```
56
67
57
-
* Set the HuggingFace token using the `defang config` command
68
+
- Set the HuggingFace token using the `defang config` command
69
+
58
70
```bash
59
71
defang config set --name HF_TOKEN
60
72
```
61
73
62
74
[Configuration](../concepts/configuration.md) stores your sensitive information such as API keys, passwords, and other credentials for you.
63
75
64
-
65
76
## Step 4 - Explore the Compose File
66
77
67
78
The `compose.yml` file is where you define your services and their configurations.
@@ -105,19 +116,19 @@ We start with the latest vLLM Docker image provided by [Mistral AI](https://docs
105
116
106
117
```yaml
107
118
mistral:
108
-
image: ghcr.io/mistralai/mistral-src/vllm:latest
119
+
image: ghcr.io/mistralai/mistral-src/vllm:latest
109
120
```
110
121
111
122
We specify that we require a GPU to run our application.
112
123
113
124
```yaml
114
125
deploy:
115
-
resources:
116
-
reservations:
126
+
resources:
127
+
reservations:
117
128
cpus: '2.0'
118
-
memory: 8192M
119
-
devices:
120
-
- capabilities: ["gpu"]
129
+
memory: 8192M
130
+
devices:
131
+
- capabilities: ["gpu"]
121
132
```
122
133
123
134
The Mistral model will be downloaded from HuggingFace. We need to have a HuggingFace Token to enable the installation, so we specify that we need to get the `HF_TOKEN` configuration value from Defang.
@@ -126,41 +137,40 @@ Specifying the `HF_TOKEN` in the `environment` section of the service in the `co
126
137
127
138
```yaml
128
139
environment:
129
-
- HF_TOKEN
140
+
- HF_TOKEN
130
141
```
131
142
132
143
### The UI Service
133
144
134
145
In this sample we also provide a simple UI to interact with the endpoint created by vLLM. The UI service is a Next.js application that runs on port 3000.
135
146
136
-
:::tip Networking
147
+
:::tip
137
148
You can see here how Defang's [networking](../concepts//networking.mdx) works. The `mistral` service is available at `http://mistral:8000`, exactly as it would be in a local `docker-compose` environment.
0 commit comments