Skip to content

Commit 64e0f8b

Browse files
authored
docs: consolidate cache configurations (#1264)
1 parent 3af18fd commit 64e0f8b

File tree

4 files changed

+120
-80
lines changed

4 files changed

+120
-80
lines changed

docs/en/guide/cache.md

Lines changed: 57 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,22 +12,50 @@ With CSI Driver, you can use either a host directory or a PVC for cache storage.
1212
* If all worker nodes are used to run JuiceFS Mount Pods, and they host similar cache content (similar situation if you use distributed caching), Pod migration is not really a problem, and you can still use host directories for cache storage.
1313
* When using a PVC for cache storage, different JuiceFS PVs can isolate cache data. If the Mount Pod is migrated to another node, the PVC reference remains the same. This ensures that the cache is unaffected.
1414

15-
## Using host directories (`hostPath`) {#cache-settings}
15+
## Using host path (`hostPath`) {#cache-settings}
1616

17-
For Kubernetes nodes, a dedicated disk is often used as data and cache storage, be sure to properly configure the cache directory, or JuiceFS cache will by default be written to `/var/jfsCache`, which can easily eat up system storage space.
17+
By default, CSI Driver uses the standard JuiceFS Client cache directory `/var/jfsCache` on the host, if you intend to use data disk as cache storage, make sure the correct path is configured, otherwise cache can drain the system disk.
1818

19-
After the cache directory is set, it will be accessible in the Mount Pod via `hostPath`. You might also need to configure other cache-related options (like `--cache-size`) according to [Adjust mount options](./configurations.md#mount-options).
19+
Specify `--cache-dir` in mount options, preferably in ConfigMap, and then CSI Driver will handle the mounts accordingly:
20+
21+
```yaml {6} title="values-mycluster.yaml"
22+
...
23+
globalConfig:
24+
enabled: true
25+
mountPodPatch:
26+
- mountOptions:
27+
- cache-dir=/data/cache
28+
- cache-size=10T
29+
```
30+
31+
When Mount Pod starts, it will include the hostPath mounts:
32+
33+
```yaml {4,9}
34+
...
35+
volumeMounts:
36+
...
37+
- mountPath: /data/cache
38+
name: cachedir-0
39+
volumes:
40+
...
41+
- hostPath:
42+
path: /data/cache
43+
type: DirectoryOrCreate
44+
name: cachedir-0
45+
```
46+
47+
If you need to further customize cache related options, check out the option list in [JuiceFS Community Edition](https://juicefs.com/docs/community/command_reference/#mount) and [JuiceFS Cloud Service](https://juicefs.com/docs/cloud/reference/commands_reference/#mount).
2048
2149
:::note
2250
23-
* In CSI Driver, `cache-dir` parameter does not support wildcard character, if you need to use multiple disks as storage devices, specify multiple directories joined by the `:` character. See [JuiceFS Community Edition](https://juicefs.com/docs/community/command_reference/#mount) and [JuiceFS Cloud Service](https://juicefs.com/docs/cloud/reference/commands_reference/#mount).
51+
* In CSI Driver, `cache-dir` parameter does not support wildcard character, if you need to use multiple disks as storage devices, specify multiple directories joined by the `:` character.
2452
* For scenarios that involve intensive small writes, we usually recommend users to temporarily enable client write cache, but due to its inherent risks, this is advised against when using CSI Driver, because Pod lifecycle is significantly more unstable, and can cause data loss if Pod exists unexpectedly.
2553

2654
:::
2755

2856
### Using ConfigMap
2957

30-
Read [Custom directory](./configurations.md#custom-cachedirs).
58+
Demostrated in the above code snippets.
3159

3260
### Define in PV (deprecated)
3361

@@ -81,20 +109,41 @@ mountOptions:
81109

82110
## Use PVC as cache path
83111

84-
From 0.15.1 and above, JuiceFS CSI Driver supports using a PVC as cache directory. This is often used in hosted Kubernetes clusters provided by cloud services, which allows you to use a dedicated cloud disk as cache storage for CSI Driver.
112+
If you have higher demands for cache isolation, or cannot use hostPath for cache due to other reasons, consider using PVC as cache storage.
85113

86-
First, create a PVC according to your cloud service provider's manual, for example:
114+
PVC should be created in advance, and if you are using one of the following service providers, you can refer to their manual:
87115

88116
* [Amazon EBS CSI Driver](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html)
89117
* [Use the Azure Disks CSI Driver in Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/azure-disk-csi)
90118
* [Using the Google Compute Engine persistent disk CSI Driver](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver)
91119
* [DigitalOcean Volumes Block Storage](https://docs.digitalocean.com/products/kubernetes/how-to/add-volumes)
92120

121+
:::tip
122+
For custom volumes, make sure `mountPath` and `hostPath` doesn't contain duplicates, to avoid conflicts.
123+
:::
124+
93125
Assuming a PVC named `jfs-cache-pvc` is already created in the same namespace as the Mount Pod (which defaults to `kube-system`), use the following example to set this PVC as the cache directory for JuiceFS CSI Driver.
94126

95127
### Using ConfigMap
96128

97-
Read [Custom directory](./configurations.md#custom-cachedirs).
129+
The minimum required version is CSI Driver v0.25.1. Upon modification, application Pods need to be re-created for changes to take effect.
130+
131+
When multiple cache directories are used, make sure all items have the same available capacity, and then set `--cache-size` to the sum.
132+
133+
```yaml
134+
- cacheDirs:
135+
- type: PVC
136+
name: jfs-cache-pvc
137+
- type: HostPath
138+
path: /var/jfsCache
139+
mountOptions:
140+
- cache-size=204800
141+
- free-space-ratio=0.01
142+
# Optional, used when you need to customize individual PVCs
143+
pvcSelector:
144+
matchLabels:
145+
need-cachedirs: "true"
146+
```
98147

99148
### Define in PV (deprecated)
100149

docs/en/guide/configurations.md

Lines changed: 2 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -363,38 +363,9 @@ stringData:
363363
configs: "{gc-secret: /root/.config/gcloud}"
364364
```
365365

366-
### Customize cache directory {#custom-cachedirs}
366+
### Cache {#custom-cachedirs}
367367

368-
This section only covers basic usage. For more details on cache warmup and cleanup, see [Cache settings](./cache.md).
369-
370-
With CSI Driver, you can use either a host directory or a PVC for cache storage. Note that:
371-
372-
- When using a PVC as a cache path, ensure that the PVC is created in advance and is in the same namespace as the Mount Pod.
373-
- When using a custom volume, make sure its `mountPath` does not conflict with other `hostPath` directories.
374-
375-
#### Using ConfigMap
376-
377-
The minimum required version is CSI Driver v0.25.1. Upon modification, application Pods need to be re-created for changes to take effect.
378-
379-
```yaml
380-
- pvcSelector:
381-
matchLabels:
382-
need-cachedirs: "true"
383-
cacheDirs:
384-
- type: PVC
385-
name: jfs-cache-pvc
386-
- type: HostPath
387-
path: /var/jfsCache
388-
```
389-
390-
#### Using mount options
391-
392-
This method only supports `hostPath` cache directories.
393-
394-
```yaml
395-
mountOptions:
396-
- cache-dir=/mnt/jfsCache1:/mnt/jfsCache2
397-
```
368+
Cache usage is also closely related to resource definition, data warmup, and cache cleanup, navigate to [Cache](./cache.md) to learn more.
398369

399370
### Other features
400371

docs/zh_cn/guide/cache.md

Lines changed: 59 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -11,22 +11,50 @@ JuiceFS 有着强大的缓存设计,阅读[社区版文档](https://juicefs.co
1111
* 如果你的集群中,所有节点均用于运行 Mount Pod,那么由于每个宿主机都持有大致相同的缓存(或者使用了分布式缓存),那么 Pod 漂移的问题可能也并不构成影响,完全可以使用宿主机路径作为缓存。
1212
* 如果用 PVC 作为缓存存储,好处是不同 JuiceFS PV 可以隔离缓存数据、分别管理,并且就算 Mount Pod 随着业务被迁移到其他节点,由于 PVC 引用关系不变,所以缓存数据仍然可以访问。
1313

14-
## 使用宿主机目录作为缓存`hostPath`) {#cache-settings}
14+
## 使用宿主机路径`hostPath`) {#cache-settings}
1515

16-
Kubernetes 节点往往采用单独的数据盘作为缓存盘,因此使用 JuiceFS 时,一定要注意正确设置缓存路径,否则默认使用根分区的 `/var/jfsCache` 目录来缓存数据,极易耗尽磁盘空间
16+
默认情况下,CSI 驱动的缓存路径就是标准的 JuiceFS 客户端缓存路径 `/var/jfsCache`,考虑到 Kubernetes 节点往往采用单独的数据盘作为缓存盘,因此一定要注意正确设置缓存路径,否则使用根分区的 `/var/jfsCache` 目录来缓存数据,容易耗尽系统盘空间
1717

18-
设置缓存路径以后,Kubernetes 宿主机上的路径会以 `hostPath` 卷的形式挂载到 Mount Pod 中,因此还需要根据缓存盘参数,对缓存相关的[挂载参数](./configurations.md#mount-options)进行调整(如缓存大小)。
18+
在挂载参数中指定好 `--cache-dir`,CSI 驱动就会自动将目标路径映射到 Pod 内,比方说在 ConfigMap 中这样配置:
1919

20-
:::tip 注意
20+
```yaml {6} title="values-mycluster.yaml"
21+
...
22+
globalConfig:
23+
enabled: true
24+
mountPodPatch:
25+
- mountOptions:
26+
- cache-dir=/data/cache
27+
- cache-size=10T
28+
```
29+
30+
那么随着 Mount Pod 启动,CSI 驱动就会为其加入对应的宿主机路径挂载:
31+
32+
```yaml {4,9}
33+
...
34+
volumeMounts:
35+
...
36+
- mountPath: /data/cache
37+
name: cachedir-0
38+
volumes:
39+
...
40+
- hostPath:
41+
path: /data/cache
42+
type: DirectoryOrCreate
43+
name: cachedir-0
44+
```
2145
22-
* 在 CSI 驱动中,`cache-dir` 不支持填写通配符,如果需要用多个设备作为缓存盘,填写多个目录,以 `:` 连接。详见[社区版](https://juicefs.com/docs/zh/community/command_reference/#mount)[云服务](https://juicefs.com/docs/zh/cloud/reference/commands_reference/#mount)文档。
46+
如果还需要进一步调整其他缓存相关的挂载参数,请阅读对应的 JuiceFS 客户端参数列表,详见[社区版](https://juicefs.com/docs/zh/community/command_reference/#mount)与[云服务](https://juicefs.com/docs/zh/cloud/reference/commands_reference/#mount)文档。
47+
48+
:::tip
49+
50+
* 在 CSI 驱动中,`cache-dir` 不支持填写通配符 `*`,如果需要用多个设备作为缓存盘,必须直接填写多个目录,以 `:` 连接。
2351
* 对于大量小文件写入场景,我们一般推荐临时开启客户端写缓存,但由于该模式本身带来的数据安全风险,我们尤其不推荐在 CSI 驱动中开启 `--writeback`,避免容器出现意外时,写缓存尚未完成上传,造成数据无法访问。
2452

2553
:::
2654

2755
### 使用 ConfigMap
2856

29-
阅读[「调整挂载参数」](./configurations.md#custom-cachedirs)
57+
已经在上方代码块中进行示范
3058

3159
### 在 PV 中定义(不推荐)
3260

@@ -78,22 +106,43 @@ mountOptions:
78106
- cache-size=204800
79107
```
80108

81-
## 使用 PVC 作为缓存路径
109+
## 使用 PVC
82110

83-
JuiceFS CSI 驱动 0.15.1 及以上版本支持使用 PVC 作为缓存路径,该实践多用于托管 Kubernetes 集群的云服务商,让你可以使用单独的云盘来作为 CSI 驱动的缓存存储设备
111+
如果对隔离程度有要求,或者因为其他原因而无法使用宿主机路径作为缓存目的地,那么可以使用 PVC 作为缓存
84112

85-
首先,按照所使用的托管 Kubernetes 集群的云服务商的说明,创建 PVC,比如
113+
如果使用 PVC 作为缓存路径,PVC 需要提前创建,并确保和 Mount Pod 在同一个命名空间(namespace)下,如果你正在使用公有云托管服务,以下是部分常用服务商的文档
86114

87115
* [Amazon EBS CSI 驱动](https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/ebs-csi.html)
88116
* [在 Azure Kubernetes 服务(AKS)中使用 Azure 磁盘 CSI 驱动](https://learn.microsoft.com/zh-cn/azure/aks/azure-disk-csi)
89117
* [使用 Google Compute Engine 永久性磁盘 CSI 驱动](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver)
90118
* [阿里云 ACK 云盘存储卷](https://help.aliyun.com/document_detail/134767.html)
91119

120+
:::tip
121+
如果自定义了 Volume,确保 `mountPath` 和 `hostPath` 没有重复项目,避免冲突。
122+
:::
123+
92124
假设名为 `jfs-cache-pvc` 的 PVC 创建完毕,与 Mount Pod 在同一个命名空间下(默认 `kube-system`),参考下方示范,让 CSI 驱动使用该 PVC 作为缓存路径。
93125

94126
### 使用 ConfigMap
95127

96-
缓存相关配置均通过挂载参数进行调整,请阅读[「调整挂载参数」](./configurations.md#custom-cachedirs)。
128+
该功能最低需要 CSI 驱动版本 v0.25.1,修改后需重建业务 Pod 生效。
129+
130+
如果使用了多个缓存路径,需要注意使用大小相同的存储设备,并且将 `--cache-size` 设置为可用空间之和。
131+
132+
```yaml
133+
- cacheDirs:
134+
- type: PVC
135+
name: jfs-cache-pvc
136+
- type: HostPath
137+
path: /var/jfsCache
138+
mountOptions:
139+
- cache-size=204800
140+
- free-space-ratio=0.01
141+
# 可选字段,对特定 PVC 定制缓存配置时使用
142+
pvcSelector:
143+
matchLabels:
144+
need-cachedirs: "true"
145+
```
97146

98147
### 在 PV 中定义(不推荐)
99148

docs/zh_cn/guide/configurations.md

Lines changed: 2 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -361,38 +361,9 @@ stringData:
361361
configs: "{gc-secret: /root/.config/gcloud}"
362362
```
363363

364-
### 定制缓存路径 {#custom-cachedirs}
364+
### 缓存 {#custom-cachedirs}
365365

366-
本小节只介绍如何设置缓存路径,如有需要,请继续阅读[缓存](./cache.md)了解如何设置多个缓存目录,以及缓存的预热和清理。
367-
368-
在 CSI 驱动中,可以选择使用宿主机路径或者 PVC 作为 JuiceFS 客户端缓存,对于这两种场景,需要注意:
369-
370-
- 如果使用 PVC 作为缓存路径,PVC 需要提前创建,并确保和 Mount Pod 在同一个 namespace。
371-
- 如果自定义了 Volume,确保自定义 Volume 中的 `mountPath` 和 `hostPath` 没有重复,避免冲突。
372-
373-
#### 使用 ConfigMap
374-
375-
该功能最低需要 CSI 驱动版本 v0.25.1,修改后需重建业务 Pod 生效。
376-
377-
```yaml
378-
- pvcSelector:
379-
matchLabels:
380-
need-cachedirs: "true"
381-
cacheDirs:
382-
- type: PVC
383-
name: jfs-cache-pvc
384-
- type: HostPath
385-
path: /var/jfsCache
386-
```
387-
388-
#### 使用挂载参数
389-
390-
该方法只支持配置 `hostPath` 形式的缓存路径。
391-
392-
```yaml
393-
mountOptions:
394-
- cache-dir=/mnt/jfsCache1:/mnt/jfsCache2
395-
```
366+
缓存的使用还涉及资源管理、数据预热和清理等事项,因此请移步阅读[缓存](./cache.md)来详细了解。
396367

397368
### 其他功能定制
398369

0 commit comments

Comments
 (0)