Skip to content

Commit fba9cc8

Browse files
Geek-Hesrowen
authored andcommitted
[SPARK-21813][CORE] Modify TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES comments
## What changes were proposed in this pull request? The variable "TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES" comment error, It shouldn't be 2^32-1, should be 2^31-1, That means the maximum value of int. ## How was this patch tested? Existing test cases Author: he.qiao <[email protected]> Closes apache#19025 from Geek-He/08_23_comments.
1 parent d7b1fcf commit fba9cc8

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,8 @@
5353
* retrieve the base object.
5454
* <p>
5555
* This allows us to address 8192 pages. In on-heap mode, the maximum page size is limited by the
56-
* maximum size of a long[] array, allowing us to address 8192 * 2^32 * 8 bytes, which is
57-
* approximately 35 terabytes of memory.
56+
* maximum size of a long[] array, allowing us to address 8192 * (2^31 - 1) * 8 bytes, which is
57+
* approximately 140 terabytes of memory.
5858
*/
5959
public class TaskMemoryManager {
6060

@@ -74,7 +74,7 @@ public class TaskMemoryManager {
7474
* Maximum supported data page size (in bytes). In principle, the maximum addressable page size is
7575
* (1L &lt;&lt; OFFSET_BITS) bytes, which is 2+ petabytes. However, the on-heap allocator's
7676
* maximum page size is limited by the maximum amount of data that can be stored in a long[]
77-
* array, which is (2^32 - 1) * 8 bytes (or 16 gigabytes). Therefore, we cap this at 16 gigabytes.
77+
* array, which is (2^31 - 1) * 8 bytes (or about 17 gigabytes). Therefore, we cap this at 17 gigabytes.
7878
*/
7979
public static final long MAXIMUM_PAGE_SIZE_BYTES = ((1L << 31) - 1) * 8L;
8080

0 commit comments

Comments
 (0)