-
Notifications
You must be signed in to change notification settings - Fork 9.8k
thomasgauvin: add docs for hyperdrive to connect to private databases #16766
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 5 commits
c25b01e
32e2dcf
c0c9d3b
4a2bfab
158ba97
c1101a0
33f3938
85fd095
30e1b52
78a6006
1ed0078
02edd73
bd4eb43
e81cae6
c606302
22eeea6
5029a3d
eba7b41
313ba57
1204dcf
1a57155
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| --- | ||
| pcx_content_type: concept | ||
| title: Connect to a private database using Tunnel | ||
| sidebar: | ||
| order: 4 | ||
| --- | ||
|
|
||
| import { TabItem, Tabs, Render } from "~/components"; | ||
|
|
||
| Hyperdrive can securely connect to your private databases using [Cloudflare Tunnel](/cloudflare-one/connections/connect-networks/) and [Cloudflare Access](/cloudflare-one/policies/access/). | ||
|
|
||
| ## How it works | ||
|
|
||
| When your database is isolated within a private network (such as a [virtual private cloud](https://www.cloudflare.com/learning/cloud/what-is-a-virtual-private-cloud) or an on-premise network), you must enable a secure connection from your network to Cloudflare. | ||
|
|
||
| - [Cloudflare Tunnel](/cloudflare-one/connections/connect-networks/) is used to establish the secure tunnel connection. | ||
| - [Cloudflare Access](/cloudflare-one/policies/access/) is used to restrict access to your tunnel such that only specific Hyperdrive configurations can access it. | ||
|
|
||
| <Render file="tutorials-before-you-start" product="workers" /> | ||
|
|
||
| ## Prerequisites: | ||
|
|
||
| - A database in your private network, [configured to use TLS/SSL](/hyperdrive/configuration/connect-to-postgres/#supported-tls-ssl-modes). | ||
| - A hostname on your Cloudflare account, which will be used to route requests to your database. | ||
|
|
||
| ## 1. Create a tunnel in your private network | ||
thomasgauvin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### 1.1. Create a tunnel | ||
|
|
||
| First, create a [Cloudflare Tunnel](/cloudflare-one/connections/connect-networks/) in your private network to establish a secure connection between your network and Cloudflare. Your network must be configured such that the tunnel has permissions to egress to the Cloudflare network and access the database within your network. | ||
|
|
||
| <Render file="tunnel/create-tunnel" product="cloudflare-one" /> | ||
|
|
||
| ### 1.2. Connect your database using a public hostname | ||
|
|
||
| Your tunnel must be configured to use a public hostname so that Hyperdrive can route requests to it. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| 1. In the **Public Hostnames** tab, choose a **Domain** and specify any subdomain or path information. This will be used in your Hyperdrive configuration to route to this tunnel. | ||
thomasgauvin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| 2. In the **Service** section, specify **Type** `TCP` and the URL of your database, such as `127.0.0.1:5432`. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| 3. Select **Save tunnel**. | ||
|
|
||
| ## 2. Create a service token | ||
|
|
||
| The service token will be used to restrict requests to the tunnel, and is needed for the next step. | ||
|
|
||
| 1. In [Zero Trust](https://one.dash.cloudflare.com), go to **Access** > **Service Auth** > **Service Tokens**. | ||
|
|
||
| 2. Select **Create Service Token**. | ||
|
|
||
| 3. Name the service token. The name allows you to easily identify events related to the token in the logs and to revoke the token individually. | ||
|
|
||
| 4. Set a **Service Token Duration** of `Non-expiring`. This prevents the service token from expiring to be used throughout the life of the Hyperdrive configuration. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| 5. Select **Generate token**. You will see the generated Client ID and Client Secret for the service token, as well as their respective request headers. | ||
|
|
||
| 6. Copy the Access Client ID and Access Client Secret. These will be used when creating the Hyperdrive configuration. | ||
|
|
||
| :::caution | ||
| This is the only time Cloudflare Access will display the Client Secret. If you lose the Client Secret, you must generate a new service token. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ::: | ||
|
|
||
| ## 3. Create an Access application to secure the tunnel | ||
|
|
||
| [Cloudflare Access](/cloudflare-one/policies/access/) will be used to verify that requests to the tunnel originate from Hyperdrive using the service token created above. | ||
|
|
||
| 1. In [Zero Trust](https://one.dash.cloudflare.com), go to **Access** > **Applications**. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My only concern is that these instructions become obsolete if ZT changes their dashboard experience... Maybe it would be better to point at their documentation for this? But that would also fragment the experience of this walkthrough. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried to re-use as much as possible. The issue lies in the fact that we need to provide opinionated guidance for some of these. For instance, we need to indicate that they need to specify Type TCP and something such as localhost:5432 when creating the tunnel. Here, we need to specify the service auth token that needs to be configured. |
||
|
|
||
| 2. Select **Add an application**. | ||
|
|
||
| 3. Select **Self-hosted**. | ||
|
|
||
| 4. In **Application Configuration** > **Application name**, enter any name for the application. | ||
|
|
||
| 5. In **Application Configuration** > **Session Duration**, select `No duration, expires immediately`. | ||
|
|
||
| 6. In **Application Configuration** > **Application domain**, enter the subdomain and domain that was previously set for the tunnel application. | ||
|
|
||
| 7. In **Application Appearance**, disable the `Enable App in App Launcher` setting. | ||
|
|
||
| 8. In **Identify providers**, disable the `Accept all available identify providers` setting and select `Deselect all` identify providers. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| 9. Select **Next**. | ||
|
|
||
| 10. Enter a name in the **Policy name** and set the **Action** to `Service Auth`. | ||
|
|
||
| 11. In **Configure rules**, create an **Include** rule. Specify a **Selector** of `Service Token` and the **Value** of the service token you created in step [2. Create a service token](#2-create-a-service-token). | ||
thomasgauvin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| 12. Select **Next**. | ||
|
|
||
| 13. Select **Add application** to create the Access application. | ||
|
|
||
| ## 4. Use the Wrangler CLI to create a Hyperdrive configuration | ||
|
|
||
| The Wrangler CLI will be used to create a Hyperdrive configuration using the Access application and Tunnel created above. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```sh | ||
| # wrangler v3.65 and above required | ||
| npx wrangler hyperdrive create <NAME-OF-HYPERDRIVE-CONFIGURATION-FOR-DB-VIA-TUNNEL> --host=<HOSTNAME-FOR-THE-TUNNEL> --user=<USERNAME-FOR-YOUR-DATABASE> --password=<PASSWORD-FOR-YOUR-DATABASE> --database=<DATABASE-TO-CONNECT-TO> --access-client-id=<YOUR-ACCESS-CLIENT-ID> --access-client-secret=<YOUR-SERVICE-TOKEN-CLIENT-SECRET> | ||
| ``` | ||
|
|
||
| This command will create a Hyperdrive configuration using the usual database information (database name, database host, database user, and database password). | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| In addition, it will also set the Access Client ID and the Access Client Secret of the Service Token. When Hyperdrive makes requests to the tunnel, requests will be intercepted by Access and validated using the credentials of the Service Token. | ||
|
|
||
| ## 5. Query your Hyperdrive configuration from a Worker (optional) | ||
|
|
||
| To test your Hyperdrive configuration to the database using Tunnel and Access, use the Hyperdrive configuration ID in your Worker and deploy it. | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Create a Hyperdrive binding | ||
|
|
||
| <Render file="create-hyperdrive-binding" product="hyperdrive" /> | ||
|
|
||
| ### Query your database using Postgres.js | ||
|
|
||
| Use Postgres.js to send a test query to validate that the connection has been successful. | ||
|
|
||
| <Render file="use-postgresjs-to-make-query" product="hyperdrive" /> | ||
|
|
||
| Now, deploy your Worker: | ||
|
|
||
| ```bash | ||
| npx wrangler deploy | ||
| ``` | ||
|
|
||
| If you successfully receive the list of `pg_tables` from your database when you access your deployed Worker, your Hyperdrive has now been configured to securely connect to a private database using [Cloudflare Tunnel](/cloudflare-one/connections/connect-networks/) and [Cloudflare Access](/cloudflare-one/policies/access/). | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| {} | ||
thomasgauvin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| You must create a binding for your Worker to connect to your Hyperdrive configuration. [Bindings](/workers/runtime-apis/bindings/) allow your Workers to access resources, like D1, on the Cloudflare developer platform. You create bindings by updating your `wrangler.toml` file. | ||
|
|
||
| To bind your Hyperdrive configuration to your Worker, add the following to the end of your `wrangler.toml` file: | ||
|
|
||
| ```toml | ||
| [[hyperdrive]] | ||
| binding = "HYPERDRIVE" | ||
| id = "<YOUR_DATABASE_ID>" # the ID associated with the Hyperdrive you just created | ||
| ``` | ||
|
|
||
| Specifically: | ||
|
|
||
| - The value (string) you set for the `name` (binding name) will be used to reference this database in your Worker. In this tutorial, name your binding `HYPERDRIVE`. | ||
| - The binding must be [a valid JavaScript variable name](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Grammar_and_types#variables). For example, `binding = "hyperdrive"` or `binding = "productionDB"` would both be valid names for the binding. | ||
| - Your binding is available in your Worker at `env.<BINDING_NAME>`. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| --- | ||
| {} | ||
| --- | ||
|
|
||
| Install the Postgres.js driver: | ||
|
|
||
| ```sh | ||
| npm install postgres | ||
| ``` | ||
|
|
||
| Create a new `sql` instance and pass the Hyperdrive parameters: | ||
|
|
||
| ```ts | ||
| import postgres from "postgres"; | ||
|
|
||
| export interface Env { | ||
| // If you set another name in wrangler.toml as the value for 'binding', | ||
| // replace "HYPERDRIVE" with the variable name you defined. | ||
| HYPERDRIVE: Hyperdrive; | ||
| } | ||
|
|
||
| export default { | ||
| async fetch(request: Request, env: Env, ctx: ExecutionContext) { | ||
| // NOTE: if `prepare: false` is passed when connecting, performance will | ||
| // be slower but still correctly supported. | ||
| const sql = postgres(env.HYPERDRIVE.connectionString); | ||
|
|
||
| try { | ||
| // A very simple test query | ||
| const result = await sql`select * from pg_tables`; | ||
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
thomasgauvin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| // Clean up the client, ensuring we don't kill the worker before that is | ||
| // completed. | ||
| ctx.waitUntil(sql.end()); | ||
|
|
||
| // Return result rows as JSON | ||
| return Response.json({ result: result }); | ||
| } catch (e) { | ||
| console.log(e); | ||
| return Response.json({ error: e.message }, { status: 500 }); | ||
| } | ||
| }, | ||
| } satisfies ExportedHandler<Env>; | ||
| ``` | ||
Uh oh!
There was an error while loading. Please reload this page.