[xquery.xml]: list of unsupported serialization parameters is incomplete

[joewiz]

Describe the problem

The section of the XQuery documentation article describing unimplemented serialization parameters, https://github.com/eXist-db/documentation/blob/master/src/main/xar-resources/data/xquery/xquery.xml#L162-L168 (rendered version at https://exist-db.org/exist/apps/doc/xquery#unsupported-features), should be expanded to include use-character-maps. Perhaps there are others?

Source code: https://github.com/eXist-db/exist/blob/develop/exist-core/src/main/java/org/exist/xquery/util/SerializerUtils.java#L300

Previous discussion:

• Untranslated numerical entity   in UTF-8 serialization Question + Patch exist#187 (comment)
• https://markmail.org/message/hjhrkmkv6qqugol4

Spec:

• https://www.w3.org/TR/xslt-xquery-serialization-31/#XML_USE-CHARACTER-MAPS

Please provide the following

• exist-db version: 5.2.0
• documentation version: 5.1.0

[joewiz]

Here's the full list of serialization parameters (some of which are only relevant to certain serialization methods) - perhaps we confirm which ones aren't implemented and thus should be added to the documentation listing unimplemented options:

• version
• html-version
• encoding
• indent
  • and suppress-indentation
• cdata-section-elements
• omit-xml-declaration
  • and standalone
• doctype-system
  • and doctype-public
• undeclare-prefixes
• normalization-form
• media-type
• use-character-maps - implementation requested in [feature] Implement use-character-maps serialization option exist#3237
• byte-order-mark
• escape-uri-attributes
• include-content-type
• item-separator - implementation requested in Implement item-separator serialization option exist#1483 (works now for adaptive, but not for other methods)
• allow-duplicate-names
• json-node-output-method

Somewhat related: eXist-db/exist#2394

[marmoure]

while ago we worked on PR that adds serialization options
eXist-db/exist#3990

• version
• html-version
• encoding
• indent
  • and suppress-indentation
• cdata-section-elements
• omit-xml-declaration
  • and standalone
• doctype-system
  • and doctype-public
• undeclare-prefixes
• normalization-form
• media-type
• use-character-maps - implementation requested in [feature] Implement use-character-maps serialization option exist#3237
• byte-order-mark
• escape-uri-attributes
• include-content-type
• item-separator - implementation requested in Implement item-separator serialization option exist#1483 (works now for adaptive, but not for other methods)
• allow-duplicate-names
• json-node-output-method

@joewiz this makes the whole list implemented for fn:serialize
and the docs were updated accordingly, do you think more should be done here?

[joewiz]

@marmoure It sounds like you're sure that all of the serialization parameters are supported for all output methods. This would be amazing. Perhaps you could check how eXist performs on the serialization tests in XQTS. Then, if there are failing tests, we could identify the problem parameters/output methods in this article. Would that be possible?

[marmoure]

@joewiz a list of tests were already written to ensure the function work
https://github.com/eXist-db/exist/blob/develop/exist-core/src/test/xquery/xquery3/serialize.xql
will these be enough?

[joewiz]

@marmoure Those tests naturally covered only the use-character-map parameter. The tests I linked to cover the full range of options listed above. I think running the XQTS tests may be the best way to determine which options eXist supports and which, if any, remain unsupported.

[marmoure]

@joewiz i ran the serialization tests we do have A lot of failures and Errors

[adamretter]

@joewiz Whilst eXist-db has support for all of the serialization options, @marmoure has clearly identified that the XQTS tests show many failures - eXist-db/exist#4580.

These failures are in many different areas and have likely been present for a very long time. I think it would be impractical to include the full output from the XQTS into the page and expect the average user to make sense of it - as some of the failures can even be compound issues or unrelated due to bugs in other parts of eXist-db which are used by an XQTS test.

I think it would be best to remove the list of unsupported serialization parameters from the page as this issue originally suggested. If necessary, we could add a sentence about there being possible bugs, as full XQTS compliance has yet to be reached - but then in general I think that statement applies to most parts of eXist-db anyway.

What do you think?

[joewiz]

@adamretter This sounds reasonable. Separate from this task, then, perhaps we can discuss analyzing the results of the XQTS suite tests and creating issues to resolve the errors.