Strimzi
What kind of storage should I choose #6431
-
|
Known NFS is not suitable： https://sbg.technology/2018/07/10/kafka-nfs/ |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
In general, you should use block storage and not file storage such as NFS. But which one, that depends on you and your environment. The cloud native block storage (as offered by Ceph and others) should work, but it might be challenging performance-wise since it often shares the same networks with Kafka etc. You could also use local storage which is a choice often used with on-prem bare metal clusters. Or some iSCSI / FibreChannel appliances if you have them. If you run in public clouds, the things such as AWS EBS volumes work also well. |
Beta Was this translation helpful? Give feedback.
-
|
I have considered local storage, because I think the performance of local storage is better, but in this way, Kafka cannot be started from other k8s nodes and some fault tolerance is lost. Thank you for your advice. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, that is absolutely correct. Local storage has its pros and cons. I think normally a lot depends on the alternatives. Sometimes, there isn't anything better. I'm not an expert on Ceph & Co - In the past, I heard about people for whom it performed very well but also those who say that they get only 1 or 2 MB/s from the storage. I have no clue what the difference might be. But obviously if you are in the second group, the local storage might be great despite its disadvantages :-). In any case, keep in mind that you need to use the Ceph block storage option and not the file storage one - the file storage is basically like NFS and would not work well. |
Beta Was this translation helpful? Give feedback.
In general, you should use block storage and not file storage such as NFS. But which one, that depends on you and your environment. The cloud native block storage (as offered by Ceph and others) should work, but it might be challenging performance-wise since it often shares the same networks with Kafka etc. You could also use local storage which is a choice often used with on-prem bare metal clusters. Or some iSCSI / FibreChannel appliances if you have them. If you run in public clouds, the things such as AWS EBS volumes work also well.