Skip to content

Commit 8e9ee52

Browse files
committed
HDInsight LLAP cluster sizing guide
1 parent e7a0d10 commit 8e9ee52

File tree

2 files changed

+9
-7
lines changed

2 files changed

+9
-7
lines changed

articles/hdinsight/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -768,6 +768,8 @@
768768
href: ./hdinsight-hadoop-hive-out-of-memory-error-oom.md
769769
- name: Apache Tez application hangs
770770
href: ./interactive-query/interactive-query-troubleshoot-tez-hangs.md
771+
- name: Apache Hive LLAP sizing guidelines
772+
href: ./interactive-query/hive-llap-sizing-guide.md
771773
- name: Apache Hive LLAP query performance
772774
href: ./interactive-query/interactive-query-troubleshoot-query-performance.md
773775
- name: Slow reducer
Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -91,9 +91,9 @@ For D14v2 worker nodes, the recommended value for llap queue is **80**.
9191
Configuration: ***hive.llap.daemon.yarn.container.mb***
9292

9393
LLAP daemon is run as a YARN container on each worker node. The total memory size for LLAP daemon container depends on following factors,
94-
1. Configurations of YARN container size (yarn.scheduler.minimum-allocation-mb, yarn.scheduler.maximum-allocation-mb, yarn.nodemanager.resource.memory-mb)
95-
2. Number of Tez AMs on a node
96-
3. Total memory configured for all containers on a node and LLAP queue capacity
94+
* Configurations of YARN container size (yarn.scheduler.minimum-allocation-mb, yarn.scheduler.maximum-allocation-mb, yarn.nodemanager.resource.memory-mb)
95+
* Number of Tez AMs on a node
96+
* Total memory configured for all containers on a node and LLAP queue capacity
9797

9898
Memory needed by Tez Application Masters(Tez AM) can be calculated as follows.
9999
For HDInsight Interactive cluster, by default, there is one Tez AM per worker node that acts as a query coordinator. The number of Tez AMs can be configured based on a number of concurrent queries to be served.
@@ -134,10 +134,10 @@ For D14 v2, the recommended value is **12**.
134134
Configuration: ***hive.llap.io.memory.size***
135135

136136
LLAP daemon container memory consists of following components;
137-
1. Head room
138-
2. Heap memory used by executors (Xmx)
139-
3. In-memory cache per daemon (its off-heap memory size, not applicable when SSD cache is enabled)
140-
4. In-memory cache metadata size (applicable only when SSD cache is enabled)
137+
* Head room
138+
* Heap memory used by executors (Xmx)
139+
* In-memory cache per daemon (its off-heap memory size, not applicable when SSD cache is enabled)
140+
* In-memory cache metadata size (applicable only when SSD cache is enabled)
141141

142142
**Headroom size**:
143143
This size indicates a portion of off-heap memory used for Java VM overhead (metaspace, threads stack, gc data structures, etc.). Generally, this overhead is about 6% of the heap size (Xmx). To be on the safer side, this value can be calculated as 6% of total LLAP daemon memory size.

0 commit comments

Comments
 (0)