YAML - ScannerException on block scalar "\n"

[benken-parasoft]

I'm copying this from https://bitbucket.org/snakeyaml/snakeyaml-engine/issues/67/scannerexception-on-block-scalar-n

The text “\n” does not serialize correctly.

Trivial reproduction example (Jackson 3):

StringWriter writer = new StringWriter();
YAMLFactory yamlFactory = YAMLFactory.builder()
        .enable(YAMLWriteFeature.MINIMIZE_QUOTES)
        .build();
try (JsonGenerator generator = yamlFactory.createGenerator(ObjectWriteContext.empty(), writer)) {
    generator.writeString("\n");
}
System.out.println(writer.toString());  // prints "--- |2+"
try (JsonParser parser = yamlFactory.createParser(ObjectReadContext.empty(), writer.toString())) {
    parser.nextToken();  // fails!
}

Stack trace:

tools.jackson.dataformat.yaml.JacksonYAMLParseException: while scanning a block scalar
 in reader, line 1, column 5:
    --- |2+
        ^
 the leading empty lines contain more spaces (2) than the first non-empty line.
 in reader, line 3, column 1:

    ^

 at [Source: (StringReader); line: 1, column: 4]
    at tools.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:308)
...
Caused by: while scanning a block scalar
 in reader, line 1, column 5:
    --- |2+
        ^
 the leading empty lines contain more spaces (2) than the first non-empty line.
 in reader, line 3, column 1:

    ^

    at org.snakeyaml.engine.v2.scanner.ScannerImpl.scanBlockScalar(ScannerImpl.java:1578)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.fetchBlockScalar(ScannerImpl.java:960)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.fetchLiteral(ScannerImpl.java:938)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:396)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.checkToken(ScannerImpl.java:193)
    at org.snakeyaml.engine.v2.parser.ParserImpl$ParseDocumentContent.produce(ParserImpl.java:558)
    at org.snakeyaml.engine.v2.parser.ParserImpl.lambda$produce$1(ParserImpl.java:232)
    at java.base/java.util.Optional.ifPresent(Optional.java:178)
    at org.snakeyaml.engine.v2.parser.ParserImpl.produce(ParserImpl.java:232)
    at org.snakeyaml.engine.v2.parser.ParserImpl.peekEvent(ParserImpl.java:206)
    at org.snakeyaml.engine.v2.parser.ParserImpl.next(ParserImpl.java:214)
    at tools.jackson.dataformat.yaml.YAMLParser.nextEvent(YAMLParser.java:446)
    at tools.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:306)
    ... 4 more

This reproduces with Jackson 3.0.0-rc6. I tried snakeyaml-engine versions 2.10, 2.9, and 2.8 but they all fail on this case. This type of issue does NOT happen when using snakeyaml 2.4 and Jackson 2.19.

I am trying to migrate from snakeyaml to snakeyaml-engine but am blocked where a lot of my cases fail because of this issue. I am serializing various complex objects and arrays and fail any time there is a String with empty lines (one or more “\n”).

[benken-parasoft]

I can also simulate this same issue by calling lower level snakeyaml-engine APIs directly:

StringWriter writer = new StringWriter();
DumpSettings settings = DumpSettings.builder().build();
Emitter emitter = new Emitter(settings, new WriterWrapper(writer));
emitter.emit(new StreamStartEvent());
emitter.emit(new DocumentStartEvent(true, Optional.empty(), Map.of()));
emitter.emit(new ScalarEvent(Optional.empty(), Optional.empty(), new ImplicitTuple(true, true), "\n", ScalarStyle.LITERAL));
emitter.emit(new DocumentEndEvent(false));
emitter.emit(new StreamEndEvent());
System.out.println(writer.toString());  // prints "--- |2+"
LoadSettings loadSettings = LoadSettings.builder().build(); 
ParserImpl parser = new ParserImpl(loadSettings, new StreamReader(loadSettings, writer.toString()));
while (parser.hasNext()) {  // fails!
    parser.next();
}

Stack trace:

while scanning a block scalar
 in reader, line 1, column 5:
    --- |2+
        ^
 the leading empty lines contain more spaces (2) than the first non-empty line.
 in reader, line 3, column 1:

    ^

    at org.snakeyaml.engine.v2.scanner.ScannerImpl.scanBlockScalar(ScannerImpl.java:1578)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.fetchBlockScalar(ScannerImpl.java:960)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.fetchLiteral(ScannerImpl.java:938)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:396)
    at org.snakeyaml.engine.v2.scanner.ScannerImpl.checkToken(ScannerImpl.java:193)
    at org.snakeyaml.engine.v2.parser.ParserImpl$ParseDocumentContent.produce(ParserImpl.java:558)
    at org.snakeyaml.engine.v2.parser.ParserImpl.lambda$produce$1(ParserImpl.java:232)
    at java.base/java.util.Optional.ifPresent(Optional.java:178)
    at org.snakeyaml.engine.v2.parser.ParserImpl.produce(ParserImpl.java:232)
    at org.snakeyaml.engine.v2.parser.ParserImpl.hasNext(ParserImpl.java:226)

Offhand, it looks as if the ParserImpl is broken, unless perhaps the original ScalarEvent was written incorrectly.

[asomov]

It may be indeed an issue in SnakeYAML Engine

[cowtowncoder]

Yes, sounds like this might be something that needs addressing at Snakeyaml-engine level.

[benken-parasoft]

The regression is now fixed in the latest snakeyaml-engine 2.11-SNAPSHOT. Should this issue be closed as invalid? Or perhaps jackson-dataformat-yaml should be updated to depend on 2.11 once available?

[cowtowncoder]

Let's only close once new non-snapshot version is available and YAML module has been updated.

[cowtowncoder]

In the meantime, updated snakeyaml-engine dep to 2.10 (was 2.8).

[asomov]

@cowtowncoder I would like to migrate SnakeYAML Engine to Java 11 (in the next version 2.11). Should it be a problem for this project?

[cowtowncoder]

No, not problematic: Jackson 3.0 requires Java 17. 2.x will remain Java 8, although some other modules require JDK 11 already. But 2.x does not use SnakeYAML engine.