SDK Schematron Memory Performance Question

[rkottmann]

Hi,

we face some memory issues using XSLT based Schematron compilers (both schxslt and ISO Schematron) without optimzations we need between 3-4 GB RAM after loading it using Saxon with Java. With Optimazation around 1GB and using https://github.com/phax/ph-schematron also stable around 1 GB. This behaviour is for all your SDK versions starting from 0.6.

However, our expectation would be that it requires less than half a GB.

Hence my question: How do you execute Schematron validation in TED CVS ? Do you observe similar memory consumption?

Knowing how you do it, would help us to get as close as possible to your way of validation.

Thx in advance.

ciao,

renzo

[bertrand-lorentz]

Hello,
What you describe roughly matches what I've seen:
On my laptop, for a testing tool using ISO Schematron and Saxon with Java, I had to increase the heap size from 4GB to 6GB ("-Xmx6G")

TED CVS uses ph-schematron in "pure" mode, which does require less memory.
TED CVS is designed to be able to handle multiple SDK versions, so it has the rules for several SDK loaded in memory. In addition it also has the translations of messages in up to 24 languages. So I don't have any precise numbers for it, but it seems able to handle at least one SDK version with 1GB of memory.

[rkottmann]

Thx. Your information helps a lot. Currently, we test an idea to solve this high memory consumption in general i.e. for all kinds of Schematron compiler. We will share our findings if you are interested.

[bertrand-lorentz]

Of course, I'd be interested in any improvement to the Schematron validation. We would be happy to reduce our hosting bill 😄