Best practice for dragonfly on host path mount

[chloeyin]

Hi all, thanks to the greate project.

I'm using it to accelerate the k8s pulling process.
I've read the official doc and tried to deploy to the local cluster, mostly using the default value from chart. There are some questions on the best practice for the dragonfly deployment.

1. Now the daemonset is by default mounting an empty dir, is is suggested to mount host path to the daemonset, so that when deployment is upraged, or for any reason that it is deleted, the new pod can reuse the data?
2. Is is suggested to deploy the peer seed by also mounting the host path? Now I see it mounts an PVC if persistance enabled. If I disable it and mount a local path to the /var/lib/dragonfly/content, is there any potential problems?
   Does this all depends on the keep flag?

storage:
  # dir is the directory to store task's metadata and content.
  dir: /var/lib/dragonfly/
  # keep indicates whether keep the task's metadata and content when the dfdaemon restarts.
  keep: true

If host path is a recommaned way for seed and daemonset, how does dragonfly deal with the existant data in /var/lib/dragonfly/content when the pod was upgraded, I guess for seed and daemonset, they are doing the same thing?

Also I'm confused about cacheDir: /var/cache/dragonfly/dfdaemon/, seems not to be the data dir I see in pod(which is /var/lib/dragonfly/content)
Thanks in advance!

[Zephyrcf]

@fcgxz2003

[fcgxz2003]

dir is the root path for metadata/content and keep=true prevents it from being wiped on restart
implementation: https://github.com/dragonflyoss/client/blob/89794901985c4874b26a0c9af9915fcf67e01df5/dragonfly-client-storage/src/content_linux.rs#L49
For client, we use host storage by mounting a hostPath to persist data, while for seedClient, since there are fewer of them, we can enable persistence via pvc to ensure performance.

Whether you use hostPath or pvc, as long as keep=true, the data will not be cleared when the pod updateed or restarted.
cacheDir has been deprecated, I will submit a pr to remove it.

[fcgxz2003]

dragonflyoss/helm-charts#450

[chloeyin]

Thanks for confirming. For this part "we can enable persistence via pvc to ensure performance.", why does use pvc ensure performance? @fcgxz2003

[fcgxz2003]

Thanks for confirming. For this part "we can enable persistence via pvc to ensure performance.", why does use pvc ensure performance? @fcgxz2003

This is our best practice, pvc provides flexible block storage with guaranteed throughput via StorageClass.

[chloeyin]

Thanks.
I Still have another question that want to clarify @fcgxz2003 :
If the daemonset is upraged or the daemonset pod is recreated, with mount to host path option, the content is cached but new nodes cannot download from this peer.

Steps:
Deploy dragonfly with peer(disable the seed).

1. Node A downloads the content from source.
2. The daemonset is upgraded and the pod is recreated.
3. Another node B downloads the same content(same url).
4. This new node B cannot reuse the data from the node A even if node A still has the content.
5. Try to trigger a download from both nodes A and B at the same time:

• The log from node A is INFO all pieces are downloaded from local

• While the log from node B is INFO finished piece bfe7da461c5a3227c5fae108c55f27445150ae64b2a636f4cbe73c524ee11fa6-251 from source

Is it true that after pods/daemonset upgraded, the local cache can only be used by the node itself, and it cannot serve as a peer for other downloads? It seems it can serve as other nodes' peer only when it triggers a real download from other peer or source through scheduler.