[bugfix] fn:transform: Conversion and treeIndex problems

[nverwer]

Description

The fn:transform function sometimes produced NullPointerExceptions. This happened for documents containing nodes of type org.exist.dom.memtree.ElementImpl and other subtypes of org.exist.dom.memtree.NodeImpl.

The function org.exist.xquery.functions.fn.transform.Convert.ToSaxon.ofNode(Node) uses org.exist.xquery.functions.fn.transform.TreeUtils.treeIndex(Node) to get a 'path' (as a list of indexes) of the Node in its containing document. This 'path' is used to find the node in a newly built document that is suitable for use by Saxon.
Using the 'path' that the treeIndex function produces assumes that it starts at the document node.

However, some nodes do not have a containing document. Specifically, in org.exist.dom.memtree.NodeImpl.getParentNode():

if (parent.getNodeType() == DOCUMENT_NODE && !((DocumentImpl) parent).isExplicitlyCreated()) {
            /*
                All nodes in the MemTree will return an Owner document due to how the MemTree is implemented,
                however the explicitlyCreated flag tells us whether there "really" was a Document Node or not.
                See https://github.com/eXist-db/exist/issues/1463
             */
            return null;

In this case, the 'path' returned by treeIndex does not contain the index (0) for the root node, and org.exist.xquery.functions.fn.transform.TreeUtils.xdmNodeAtIndex(XdmNode, List<Integer>) will return null when a node is expected.

The proposed fix tests for this case. Going up the ancestor chain, when a node that is not a document node, and that does not have a parent is found, the index 0 for this root element is inserted in the 'path'.

Testing

The NullPointerException happened in a rather complicated XQuery with documents composed from several sources.
I have not yet been able to make a simple test case, but I will keep trying.

[dizzzz]

@wolfgangmm please can you check?

[nverwer]

It appears there is another (different) problem with the way fn:transform handles its input, see #5682.

[nverwer]

I have added two more commits.

• Show the location when an error occurs during an XSL transformation (because my client who uses fn:transform wanted this).
• Fix an error when transforming a document containing 'NodeProxy` nodes into an XdmValue for Saxon.

The second commit fixes another bug in the handling of NodeProxy instances.

[nverwer]

There was another problem with the the 'path' returned by treeIndex.
The indexes are determined by counting previous-siblings.
It turns out that a org.exist.dom.persistent.StoredNode returns attributes of an element as previous siblings of the element's children.
When later the tree is traversed using the indexes, attributes are not in the children list.

I think that what StoredNode does is wrong, but decided to not try to solve it there (yet).
Instead, I made a work-around in TreeUtils.

[DrRataplan]

Should this method not be adapted? This loops over children(), are these not including attributes?

[line-o]

Thanks @nverwer again for your contribution. We will pull it in once we have at least one positive and one negative test for it.
Any pointers you could give how to test this scenario would be much appreciated.

[line-o]

Or maybe it is easier to test in XQuery?

[duncdrum]

@nverwer thank you for the time you spend on this. We would like to include this PR in a patch release of exist. Could you rebase the PR please so, we can start with a cleaner commit history? I will try to help you with adding a test case. Did you succeed in the meantime in wiggling things down? Could you share the xsl that triggered the NPE?

[nverwer]

@nverwer Could you rebase the PR please so, we can start with a cleaner commit history?

What should I rebase this on? EDIT: I see, it is develop-6.x.x.

About a test case: We had the problem several months ago in a very large application. I could not make a concise test case back then, and it would be even more difficult now. Designing a test case requires a thorough understanding of the internal representations of XML in eXist-db, which I do not have.

I apologize for not coming up with a useful test. I am aware that eXist can only advance by the efforts of the community around it.
My main concern is to keep our application running, and this PR has helped us to use fn:transform instead of transform:transform.

[nverwer]

After spending some time trying to rebase, it all seems to be terribly messed up, even resulting in automatically closing this PR. There are close to a 100 files with merge conflicts, most of which are totally unrelated to this PR. I have no idea why this went horribly wrong. I will give it one more try before giving up.

[nverwer]

I tried to apply the changes to develop-6.x.x and will reopen this PR.

[duncdrum]

@nverwer thank you, gave me quite a scare there for a minute

[nverwer]

@nverwer thank you, gave me quite a scare there for a minute
I should not have pushed prematurely. 😨
I could not reconstruct exactly what was in the original pull request, but it does include the fixes for the NPE. I added the code that allows maps and arrays to be passed as parameters to an XSL transformation. I probably should have made that into a separate pull request. 😬