Parsing and printing of strings

[j-j-m]

I had noticed in the JSON benchmark that the printing code is commented out...

Without realizing this, I have been trying to use variants of this parser printer contained in the benchmark to do various things

let string = ParsePrint {
  "\"".utf8
  Many(into: "") { string, fragment in
    string.append(contentsOf: fragment)
  } decumulator: { string in
    string.map(String.init).reversed().makeIterator()
  } element: {
    OneOf {
      Prefix(1...) { $0.isUnescapedJSONStringByte }.map(.string)

      escape
    }
  } terminator: {
    "\"".utf8
  }
}

The consistent error I run into is

A "Prefix" parser's predicate satisfied the first element printed by the next printer.

During a round-trip, the "Prefix" parser would have parsed this element, which means the data handed to the next printer is in an invalid state.

After Isolating the above parser I see it suffers from the same issue.

var input = "\"hello, world\""[...].utf8
try string.parse(input) // <--- works

var output: Substring.UTF8View = .init()
try string.print("hello, world", into: &output)  // <--- doesn't work

I guess what I am after is has anyone found a way around this? Is a working JSON, generic string Parser/Printer possible currently?

[cham-s]

Edit: 06/29/2023

😮 I just noticed that issues previously mentioned have been fixed since version 0.12.0.

The swift-parsing-benchmark parsers uses the new shiny struct declaration for ParserPrinter with parsers composed using
the body property result builder.

I did a couple of tests by parsing and printing JSON values, they're not failing anymore 😀.
Big thanks to the team for your continuous effort. You guys are awesome! 👍
The type safe JSONValue will be very valuable to developers.

Previous post:

Hi everyone, it looks like I'm not the only interested about the JSON parser-printer from the benchmark target.
After some debugging sessions I think I have located, exactly where the printer failed and I was actually wondering
why it was failing.

Note: Although commenting a part from the Parsing library code makes the JSON parer-printer works.
This not an actual solution, but is it to show where the printer fails.

First I would like to explain my underderstanding of how the string parser-printer works, then illustrate it with an actual example.
Feel free to point out any mistake I make.

How the string parser-printer works

I would like to focus on the printing part because the parsing part works as intended (at least to me).
The string printer is composed of the following parts:

It's a Many parser-printer with

• A literal parser, here the " characterised in Swift by:

  "\"".utf8 

• A decumulator, a closure that returns an iterator, in our case an iterator composed of a single character reversed string (in short stringified array of characters).

   string.map(String.init).reversed().makeIterator() // ReversedCollection<[String]>.Iterator

• An element, in our case this is a OneOf2 composed of:

  • Prefix while character is an unescaped JSON String byte
  • escape (If the character is an escape character this parser is chosen over the first one)
    Note: In printing Prefix and escape predicates are use to perform validation and drive the logic of printing.
    Because each parser possesses an underlying predicate.

• A terminator: In our case the literal parser "

  "\"".utf8 

Le chef patissier string printer uses all these ingredients to bake a nice input 🧁.

Example

Let's say we want to perform the following operation.

let input = ""[...].utf8
try string.print("hello", into: &input)

Note: For each step we will be referring to the current

• output, the character we want to print
• input, all characters successfully printed to the input so far

How the string parser-printer does its job

Since printing is performed in reverse order.

1- It tries to print (prepends) using the terminator printer the literal " into the input.

• Before printing

output = "\""

input = ""

• After printing attempt

output = "\""

input = "\""

In our case printing succeeds. ✅

2- The decumulator is a printer with the string hello splited and reversed and is reprensented as an iterator

ReversedCollection<["o", "l", "l", "e", "h"]>.Iterator

In general terms the Many printer tries to print as many elements as it can with the help of the iterator that provides
each character if any is available.

// Note: The actual logic is more nuanced the full code can be found in ```Sources/Parsing/ParserPrinters/Many.swift```

// Tries to print the first element
guard let first = iterator.next() else {
  // throws
}
try self.element.print(first, into: &input)

• Before printing

output = "o"

input = "\""

• After printing attempt

output = "o"

input = "o\""

Here printing also succeeds. ✅

Now the logic enters the loop since there are more characters left to print "l" "l" "e" "h"

 while let element = iterator.next() { 
  // In our case there is no explicit separator
  try self.separator.print(into: &input)
  try self.element.print(element, into: &input)
}
// 

Before printing

output = "l"

input = "o\""

• After printing attempt
  ❌ Unfortunately printing fails with the following output

Swift/ErrorType.swift:200: Fatal error: Error raised at top level: error: multiple failures occurred

error: round-trip expectation failed

A "Prefix" parser that parses at least 4 elements was given only 2 elements to print.

"o\""

error: expected   

error: expected 

error: expected 


error: expected 


error: expected 

error: expected /

error: expected \

error: expected "

error: round-trip expectation failed

A "Prefix" parser's predicate satisfied the first element printed by the next printer.

Why it fails

After some debugging sessions I think I have an understanding of why it fails which will lead to my final question.

Let's recall what the current input and output are and the what is the shape of the element parser from the string parser-printer.

Before printing

output = "l"

input = "o\""

let string = ParsePrint {
  "\"".utf8
  Many(into: "") { string, fragment in
    string.append(contentsOf: fragment)
  } decumulator: { string in
    string.map(String.init).reversed().makeIterator()
  } element: {
    OneOf {
      Prefix(1...) { $0.isUnescapedJSONStringByte }.map(.string)
      escape
    }
  } terminator: {
    "\"".utf8
  }
}

// element parser
    OneOf {
      Prefix(1...) { $0.isUnescapedJSONStringByte }.map(.string)
      escape
}

Printing is a done in reversed order compared to parsing so escape printer is tried first from the OneOf2 printer and failed which is the intended behaviour because
our current output is output = "l" and it's not an escaped character.

First Error

Actualy the escaped parser-printer tries its best to print but it failed because upon conversion it returns a 2 elements Substring.UTF8View.

let lUTF8View = "l".unicodeScalars.first
.map { String(UInt32($0), radix: 16)[...].utf8 }! // Substring.UTF8View

Substring(lUTF8View) // "6c"

lUTF8View.count // 2

And unicode takes 4 elements.

  let unicode = ParsePrint(.unicode) {
    Prefix(4) { $0.isHexDigit }
  }

This is why we get the first error ouput
A "Prefix" parser that parses at least 4 elements was given only 2 elements to print.

Final error

On to the final error now.

A "Prefix" parser's predicate satisfied the first element printed by the next printer.

During a round-trip, the "Prefix" parser would have parsed this element, which means the data handed to the next printer is in an invalid state.

To get to the point the failure happens inside the file Sources/Parsing/ParserPrinters/Prefix at line 171
here is the code snippet.

swift-parsing/Sources/Parsing/ParserPrinters/Prefix.swift

Lines 170 to 187 in 4d80541

	if count != maximum {
	guard input.first.map(predicate) != true
	else {
	throw PrintingError.failed(
	summary: """
	round-trip expectation failed
	A "Prefix" parser's predicate satisfied the first element printed by the next printer.
	During a round-trip, the "Prefix" parser would have parsed this element, which means \
	the data handed to the next printer is in an invalid state.
	""",
	input: input
	)
	}
	}
	}
	input.prepend(contentsOf: output)

Basically it says in order to prepend the current output "l" into input the result after evaluation of the predicate must be different from true.
In other word nil or false. This is what makes printing failed because the first of input is o and it evaluates to true when applying the predicate because o is an ``isUnescapedJSONStringByte```

guard input.first.map(predicate) != true

The current context is

// for reference
extension UTF8.CodeUnit {
  fileprivate var isUnescapedJSONStringByte: Bool {
    self != .init(ascii: "\"") && self != .init(ascii: "\\") && self >= .init(ascii: " ")
  }
}

output = "l"
input = "o\""
maximum = nil
count = 1
predicate = (UTF8.CodeUnit) -> Bool //  { $0.isUnescapedJSONStringByte }

So since count != nil it enters the if block
Then there is a guard checking the first element of input printed so for with the predicate
the result is

// Let's recall
input = "o\""

// So
"o\""[...].utf8.first.map { $0.isUnescapedJSONStringByte } // Optional(true)

The question

So my question is the following:
Does anyone know why in our context evaluating the first element of the input printed so far with the current
predicate as true is considered an error?

When we omit this guard by commenting it we have a working JSON parser-printer

171 //        guard input.first.map(predicate) != true
172 //        else {
173 //          throw PrintingError.failed(
174 //            summary: """
175 //              round-trip expectation failed
176 //
177 //              A "Prefix" parser's predicate satisfied the first element printed by the next printer.
178 //
179 //              During a round-trip, the "Prefix" parser would have parsed this element, which means \
180 //              the data handed to the next printer is in an invalid state.
181 //              """,
182 //            input: input
183 //          )
184 //        }

I've tried the round trip property and it works for

var hello = "\"hello\""[...].utf8
try string.print(try string.parse(&hello), into: &hello)
print(Substring(hello)) // "hello"

var input = #"""
{
  "hello": true,
  "goodbye": 42.42,
  "whatever": null,
  "fruits": ["🥥", "🥭", "🍒"],
  "xs": [1, "hello", null, false],
  "ys": {
    "0": 2,
    "1": "goodbye\n"
  }
}
"""#[...].utf8
try object.print(try object.parse(&input), into: &input)
print(Substring(input)) // {"fruits":["🥥","🥭","🍒"],"goodbye":42.42,"hello":true,"whatever":null,"xs":[1.0,"hello",null,false],"ys":{"0":2.0,"1":"goodbye\n"}}

Not a real solution

But I don't think that this is the solution since it might affect other parser-printers and the guard makes sure printing is done correctly.
At least I hope it will lead us to the right direction for the solution.

Sorry it was this long 😅, maybe it shows how much the JSON parser would be beneficial to the Swift community.