You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/shared/v3-distributed-admin-custom-partitions/best-practices.md
+15-24Lines changed: 15 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,39 +33,30 @@ If points don't have a value for the tag, InfluxDB can't store them in the corre
33
33
34
34
## Avoid over-partitioning
35
35
36
-
As you plan your partitioning strategy, keep in mind that data can be
37
-
"over-partitioned"--meaning partitions are so granular that queries end up
38
-
having to retrieve and read many partitions from the object store, which
39
-
hurts query performance.
40
-
41
-
- Balance the partition time interval with the actual amount of data written
42
-
during each interval. If a single interval doesn't contain a lot of data,
43
-
it is better to partition by larger time intervals.
44
-
- Don't partition by tags that you typically don't use in your query workload.
45
-
- Don't partition by distinct values of high-cardinality tags.
46
-
Instead, [use tag buckets](#use-tag-buckets-for-high-cardinality-tags) to
47
-
partition by these tags.
36
+
As you plan your partitioning strategy, keep in mind that over-partitioning your data can hurt query performance. If partitions are too granular, queries may need to retrieve and read many partitions from the [Object store](/influxdb/cloud-dedicated/reference/internals/storage-engine/#object-store).
37
+
38
+
- Balance the partition time interval with the actual amount of data written during each interval. If a single interval doesn't contain a lot of data, partition by larger time intervals.
39
+
- Avoid partitioning by tags that you typically don't use in your query workload.
40
+
- Avoid partitioning by distinct values of high-cardinality tags. Instead, [use tag buckets](#use-tag-buckets-for-high-cardinality-tags) to partition by these tags.
48
41
49
42
## Limit the number of partition files
50
43
51
-
Avoid exceeding **10,000** total partition files.
44
+
Avoid exceeding **10,000** total partitions.
52
45
Limiting the total partition count can help manage system performance and costs.
53
46
54
-
While planning your strategy include the following steps to keep the total
55
-
partition count below 10,000 files over the next few years:
47
+
While planning your strategy, take the following steps to limit your total
48
+
partition count.
49
+
We currently recommend planning to keep the total partition count below 10,000.
56
50
57
51
-[Estimate the total partition count](#estimate-the-total-partition-count) for the lifespan of your data
58
-
- Take the following steps to limit the total partition count:
59
-
60
-
-**Set a [database retention period](/influxdb/cloud-dedicated/admin/databases/#retention-period)**
61
-
to prevent the number of files from growing unbounded.
62
-
-**Partition by month or year** to [avoid over-partitioning](#avoid-over-partitioning)
63
-
and creating too many partition files.
64
-
-**Don't partition on high cardinality tags** unless you also use [tag buckets](#use-tag-buckets-for-high-cardinality-tags)
52
+
-**Set a [database retention period](/influxdb/cloud-dedicated/admin/databases/#retention-period)**
53
+
to prevent the number of partitions from growing unbounded
54
+
-**Partition by month or year** to [avoid over-partitioning](#avoid-over-partitioning)
55
+
-**Don't partition on high cardinality tags** unless you also use [tag buckets](#use-tag-buckets-for-high-cardinality-tags)
65
56
66
57
### Estimate the total partition count
67
58
68
-
Use the following formula to estimate the total partition file count over the
59
+
Use the following formula to estimate the total partition count over the
-`total_partition_count`: The number of partition files in [Object storage](/influxdb/cloud-dedicated/reference/internals/storage-engine/#object-storage)
76
67
-`cardinality_of_partitioned_tag`: The number of distinct values for a tag
77
68
-`data_lifespan`: The [database retention period](/influxdb/cloud-dedicated/admin/databases/#retention-period), if set, or the expected lifetime of the database
78
-
-`partition_duration`: The partition time interval, defined by the [tine part template](/influxdb/cloud-dedicated/admin/custom-partitions/partition-templates/#time-part-templates)
69
+
-`partition_duration`: The partition time interval, defined by the [time part template](/influxdb/cloud-dedicated/admin/custom-partitions/partition-templates/#time-part-templates)
0 commit comments