Skip to content

Commit a977c95

Browse files
authored
Increase local breaker reserve memory (#133770)
Currently, we use a local breaker that over-reserves memory for each driver to reduce frequent calls to the global circuit breaker, which can be expensive. After reviewing some profiles, the maximum reserved amount appears too small and does not significantly reduce calls to the global breaker. The reserved amount should cover at least several pages of data (~256KB by default). This change proposes increasing the maximum reserved to two pages (512KB). One downside is that we may hit the circuit breaker earlier under extremely tight memory conditions, but the impact should be minimal. For example, on a node with 8 CPUs, the total reserved could be 12 * 0.5 = 6MB. In cases with many long-running queries that frequently wake and sleep, the extra over-reserved memory may be more noticeable. However, even with 100 drivers, the total maximum would be only 50MB.
1 parent 733d841 commit a977c95

File tree

1 file changed

+2
-2
lines changed
  • x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/data

1 file changed

+2
-2
lines changed

x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/data/BlockFactory.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@
1919

2020
public class BlockFactory {
2121
public static final String LOCAL_BREAKER_OVER_RESERVED_SIZE_SETTING = "esql.block_factory.local_breaker.over_reserved";
22-
public static final ByteSizeValue LOCAL_BREAKER_OVER_RESERVED_DEFAULT_SIZE = ByteSizeValue.ofKb(4);
22+
public static final ByteSizeValue LOCAL_BREAKER_OVER_RESERVED_DEFAULT_SIZE = ByteSizeValue.ofKb(8);
2323

2424
public static final String LOCAL_BREAKER_OVER_RESERVED_MAX_SIZE_SETTING = "esql.block_factory.local_breaker.max_over_reserved";
25-
public static final ByteSizeValue LOCAL_BREAKER_OVER_RESERVED_DEFAULT_MAX_SIZE = ByteSizeValue.ofKb(16);
25+
public static final ByteSizeValue LOCAL_BREAKER_OVER_RESERVED_DEFAULT_MAX_SIZE = ByteSizeValue.ofKb(512);
2626

2727
public static final String MAX_BLOCK_PRIMITIVE_ARRAY_SIZE_SETTING = "esql.block_factory.max_block_primitive_array_size";
2828
public static final ByteSizeValue DEFAULT_MAX_BLOCK_PRIMITIVE_ARRAY_SIZE = ByteSizeValue.ofKb(512);

0 commit comments

Comments
 (0)