Skip to content

[ZT] Tunnel token rotation #18577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 5, 2024
Merged

[ZT] Tunnel token rotation #18577

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ae323f4
tunnel rotation instructions
ranbel Dec 4, 2024
a20dfff
fix link
ranbel Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 162 additions & 2 deletions ...dflare-one/connections/connect-networks/configure-tunnels/remote-management.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,166 @@ To configure how `cloudflared` sends requests to your [public hostname](/cloudfl

## Tunnel permissions

A remotely-managed tunnel only requires the tunnel token to run. Anyone with access to the token will be able to run the tunnel. You can get a tunnel's token from the dashboard or via the [API](/api/operations/cloudflare-tunnel-get-a-cloudflare-tunnel-token).
A remotely-managed tunnel only requires the tunnel token to run. Anyone with access to the token will be able to run the tunnel.

Account members with Cloudflare Access and DNS [permissions](/cloudflare-one/roles-permissions/) will be able to create, delete, and configure all tunnels for the account.
### View the tunnel token

To get the token for a remotely-managed tunnel:

<Tabs syncKey="dashPlusAPI">
<TabItem label="Dashboard">
1. In [Zero Trust](https://one.dash.cloudflare.com/), go to **Networks** > **Tunnels**.
2. Select a `cloudflared` tunnel and select **Edit**.
3. Copy `cloudflared` installation command.
4. Paste the installation command into any text editor. The token value is of the form `eyJhIjoiNWFiNGU5Z...`

</TabItem>
<TabItem label="API">

Make a `GET` request to the [Cloudflare Tunnel token](/api/operations/cloudflare-tunnel-get-a-cloudflare-tunnel-token) endpoint:

```sh
curl https://api.cloudflare.com/client/v4/accounts/{account_id}/cfd_tunnel/{tunnel_id}/token \
--header "Authorization: Bearer <API_TOKEN>"
```
```sh output
{
"success": true,
"errors": [],
"messages": [],
"result": "eyJhIjoiNWFiNGU5Z..."
}
```

</TabItem>
</Tabs>

### Rotate a token without service disruption

Cloudflare recommends rotating the tunnel token at a regular cadence to reduce the risk of token compromise. You can rotate a token with minimal disruption to users as long as the tunnel is served by at least two [`cloudflared` replicas](/cloudflare-one/connections/connect-networks/deploy-tunnels/deploy-cloudflared-replicas/). To ensure service availability, We recommend performing token rotations outside of working hours or in a maintenance window.

To rotate a tunnel token:

1. Refresh the token on Cloudflare:

<Tabs syncKey="dashPlusAPI">
<TabItem label="Dashboard">
1. In [Zero Trust](https://one.dash.cloudflare.com/), go to **Networks** > **Tunnels**.
2. Select a `cloudflared` tunnel and select **Edit**.
3. Select **Refresh token**.
4. Copy the `cloudflared` installation command for your operating system. This command contains the new token.

</TabItem>
<TabItem label="API">

1. Generate a random base64 string (minimum size 32 bytes) to use as a tunnel secret:

```sh
openssl rand -base64 32
```

```sh output
AQIDBAUGBwgBAgMEBQYHCAECAwQFBgcIAQIDBAUGBwg=
```

2. Make a `PATCH` request to the [Cloudflare Tunnel](/api/operations/cloudflare-tunnel-update-a-cloudflare-tunnel) endpoint:
```sh
curl --request PATCH \
https://api.cloudflare.com/client/v4/accounts/{account_id}/cfd_tunnel/{tunnel_id} \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer <API_TOKEN>" \
--data '{
"name": "Example tunnel",
"tunnel_secret": "AQIDBAUGBwgBAgMEBQYHCAECAwQFBgcIAQIDBAUGBwg="
}'
```

```sh output {18}
{
"success": true,
"errors": [],
"messages": [],
"result": {
"id": "f70ff985-a4ef-4643-bbbc-4a0ed4fc8415",
"account_tag": "699d98642c564d2e855e9661899b7252",
"created_at": "2024-12-04T22:03:26.291225Z",
"deleted_at": null,
"name": "Example tunnel",
"connections": [],
"conns_active_at": null,
"conns_inactive_at": "2024-12-04T22:03:26.291225Z",
"tun_type": "cfd_tunnel",
"metadata": {},
"status": "inactive",
"remote_config": true,
"token": "eyJhIjoiNWFiNGU5Z..."
}
}
```
3. Copy the `token` value shown in the output.

</TabItem>
</Tabs>

After refreshing the token, `cloudflared` can no longer establish new connections to Cloudflare using the old token. However, existing connectors will remain active and the tunnel will continue serving traffic.

2. On half of your `cloudflared` replicas, update `cloudflared` to use the new token. For example, on a Linux host:

```sh
sudo cloudflared service install <TOKEN>
```

3. Restart `cloudflared`:

```sh
sudo systemctl restart cloudflared.service
```

4. Confirm that the service started correctly:
```sh
sudo systemctl status cloudflared
```

While these replicas are connecting to Cloudflare with the new token, traffic will automatically route through the other replicas.

5. Wait 10 minutes for traffic to route through the new connectors.

6. Repeat steps 2, 3, and 4 for the second half of the replicas.

The tunnel token is now fully rotated. The old token is no longer in use.

### Rotate a compromised token

If your tunnel token is compromised, we recommend taking the following steps:

1. Refresh the token using the dashboard or API. Refer to Step 1 of [Rotate a token without service disruption](#rotate-a-token-without-service-disruption).
2. [Delete all connections](/api/operations/cloudflare-tunnel-clean-up-cloudflare-tunnel-connections) between `cloudflared` and Cloudflare:
```sh
curl --request DELETE \
https://api.cloudflare.com/client/v4/accounts/{account_id}/cfd_tunnel/{tunnel_id}/connections \
--header "Authorization: Bearer <API_TOKEN>"
```

This will clean up any unauthorized connections and prevent users from connecting to your network.

3. On each `cloudflared` replica, update `cloudflared` to use the new token. For example, on a Linux host:

```sh
sudo cloudflared service install <TOKEN>
```
4. Restart `cloudflared`:

```sh
sudo systemctl restart cloudflared.service
```

5. Confirm that the service started correctly:
```sh
sudo systemctl status cloudflared
```

The tunnel token is now fully rotated. The old token is no longer in use.

### Account-scoped roles

Account members with [Cloudflare Access](/cloudflare-one/roles-permissions/) and [DNS](/fundamentals/setup/manage-members/roles/) permissions will be able to create, delete, and configure all tunnels for the account.
Loading