Skip to content

Commit d15dbfb

Browse files
kiszkRobert Kruszewski
authored andcommitted
[SPARK-23882][CORE] UTF8StringSuite.writeToOutputStreamUnderflow() is not expected to be supported
## What changes were proposed in this pull request? This PR excludes an existing UT [`writeToOutputStreamUnderflow()`](https://github.com/apache/spark/blob/master/common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java#L519-L532) in `UTF8StringSuite`. As discussed [here](apache#19222 (comment)), the behavior of this test looks surprising. This test seems to access metadata area of the JVM object where is reserved by `Platform.BYTE_ARRAY_OFFSET`. This test is introduced thru apache#16089 by NathanHowell. More specifically, [the commit](apache@27c102d) `Improve test coverage of UTFString.write` introduced this UT. However, I cannot find any discussion about this UT. I think that it would be good to exclude this UT. ```java public void writeToOutputStreamUnderflow() throws IOException { // offset underflow is apparently supported? final ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); final byte[] test = "01234567".getBytes(StandardCharsets.UTF_8); for (int i = 1; i <= Platform.BYTE_ARRAY_OFFSET; ++i) { new UTF8String( new ByteArrayMemoryBlock(test, Platform.BYTE_ARRAY_OFFSET - i, test.length + i)) .writeTo(outputStream); final ByteBuffer buffer = ByteBuffer.wrap(outputStream.toByteArray(), i, test.length); assertEquals("01234567", StandardCharsets.UTF_8.decode(buffer).toString()); outputStream.reset(); } } ``` ## How was this patch tested? Existing UTs Author: Kazuaki Ishizaki <[email protected]> Closes apache#20995 from kiszk/SPARK-23882.
1 parent 5c11465 commit d15dbfb

File tree

1 file changed

+0
-16
lines changed

1 file changed

+0
-16
lines changed

common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java

Lines changed: 0 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -515,22 +515,6 @@ public void soundex() {
515515
assertEquals(fromString("世界千世").soundex(), fromString("世界千世"));
516516
}
517517

518-
@Test
519-
public void writeToOutputStreamUnderflow() throws IOException {
520-
// offset underflow is apparently supported?
521-
final ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
522-
final byte[] test = "01234567".getBytes(StandardCharsets.UTF_8);
523-
524-
for (int i = 1; i <= Platform.BYTE_ARRAY_OFFSET; ++i) {
525-
new UTF8String(
526-
new ByteArrayMemoryBlock(test, Platform.BYTE_ARRAY_OFFSET - i, test.length + i))
527-
.writeTo(outputStream);
528-
final ByteBuffer buffer = ByteBuffer.wrap(outputStream.toByteArray(), i, test.length);
529-
assertEquals("01234567", StandardCharsets.UTF_8.decode(buffer).toString());
530-
outputStream.reset();
531-
}
532-
}
533-
534518
@Test
535519
public void writeToOutputStreamSlice() throws IOException {
536520
final ByteArrayOutputStream outputStream = new ByteArrayOutputStream();

0 commit comments

Comments
 (0)