Skip to content

Avoid distributed block storage #21099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Avoid distributed block storage #21099

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/current/v25.4/recommended-production-settings.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,8 @@ We recommend provisioning volumes with {% include {{ page.version.version }}/pro

This is especially recommended if you are using local disks with no RAID protection rather than a cloud provider's network-attached disks that are often replicated under the hood, because local disks have a greater risk of failure. You can do this for the [entire cluster]({% link {{ page.version.version }}/configure-replication-zones.md %}#edit-the-default-replication-zone) or for specific [databases]({% link {{ page.version.version }}/configure-replication-zones.md %}#create-a-replication-zone-for-a-database), [tables]({% link {{ page.version.version }}/configure-replication-zones.md %}#create-a-replication-zone-for-a-table), or [rows]({% link {{ page.version.version }}/configure-replication-zones.md %}#create-a-replication-zone-for-a-partition).

- Avoid distributed block storage. [CockroachDB's internal architecture]({% link {{ page.version.version }}/architecture/overview.md %}) already handles [data distribution]({% link {{ page.version.version }}/architecture/distribution-layer.md %}) and fault tolerance using the [Raft consensus protocol]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft). Layering CockroachDB's data directory on top of another distributed file or block system will lead to severe performance and stability issues. This configuration causes double replication (wasting space), introduces unpredictable network latency into critical I/O paths, and leads to unreliable durability semantics. Always use volumes that guarantee high, consistent IOPS and throughput. For per-cloud storage recommendations, see [Cloud-specific recommendations](#cloud-specific-recommendations).

{{site.data.alerts.callout_info}}
Under-provisioning storage leads to node crashes when the disks fill up. Once this has happened, it is difficult to recover from. To prevent your disks from filling up, provision enough storage for your workload, monitor your disk usage, and use a [ballast file]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#automatic-ballast-files). For more information, see [capacity planning issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#capacity-planning-issues) and [storage issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#storage-issues).
{{site.data.alerts.end}}
Expand Down
Loading