Semantic validation of MusicXML documents

[infojunkie]

Sometimes validating MusicXML syntax via XSD is not enough. For example, any element <sound tocoda="some-label"/> implies that there has to be another (and only one) element <sound coda="some-label"/> elsewhere in the document.

I asked Stack Overflow what existing XML technologies can help with that sort of validation. I received some authoritative answers advising to use Schematron or XSD 1.1. Another option is to use XSLT to generate a conformance report.

This ticket is about starting a tests folder in this repo where such semantic tests can be added. At this point in time, it is not clear which tech (Schematron or XSD 1.1) is better supported in terms of tools, and better suited to the task at hand.

For what it's worth, MEI schema supports embedded validation using Schematron and possibly other schemata via the contraintSpec element.

[infojunkie]

Tool support:

• Apache Xerces2 2.12.2 (Java) supports XSD 1.1.
• xmlschema (Python) supports XSD 1.1.
• ph-schematron (Python) supports ISO Schematron.
• SchXslt2 (XSLT) is a ISO Schematron-to XSLT 3.0 transpiler.
• PySchematron (Python) supports ISO Schematron.
• XQS (XQuery) supports ISO Schematron.

[infojunkie]

@rjjaret is this something you'd be interested to collaborate on?

[rjjaret]

Yes, this looks like a good one for me to work on.

[infojunkie]

That's great to hear, thanks! In the spirit of keeping our work in the open, let's first agree on specific use-cases we want to support and that would benefit our community:

I can suggest some use-cases:

• Ability to run validation tests in GitHub workflows. To ensure the tech stack is reproducible and to test the validation tests themselves. Initially, I imagine we would host the tests in this repo and run them against a set of specially crafted MusicXML files that are expected to pass/fail each validation.
• Ability to run validation tests from console. This would mean the tech dependencies to run the validation tests are well-documented and reasonably reproducible across Linux/macOS/Windows.
• Ability to run validation tests programmatically. This would mean that music software maintainers can bundle these validation tests in their own software or unit tests to ensure that they produce valid MusicXML.
• Supporting MXL. Not a priority imo, but an essential feature on the medium term.

Would love to hear your thoughts and suggestions! Other interested members too, of course.

[rjjaret]

The original link is sending to an explanation of a slur :). (https://music-encoding.org/guidelines/v5/elements/slur.html) Can you send the one you intended?

[infojunkie]

The original link is sending to an explanation of a slur :). (https://music-encoding.org/guidelines/v5/elements/slur.html) Can you send the one you intended?

There's no direct linking to constraintSpec, but you can find it in the XML declaration at https://music-encoding.org/guidelines/v5/elements/slur.html#declaration

[rjjaret]

Hey Karim. It says in the MusicXML ReadMe that I need to sign a license agreement before I can submit a pull request. Can you (or someone else) send me that? It's just for the initial task requested in the ticket - creating a tests folder structure and ReadMe.

[dspreadbury]

Rob, if you haven't yet formally joined the Music Notation Community Group, that's the step you're missing. Go here and click 'Join or Leave This Group' on the right-hand side. You'll sign the CLA as part of the process of joining the group.

[infojunkie]

Thanks both!

[rjjaret]

So I had joined a while back using a different email from my git email. (I'm assuming I signed the CLA then.) I just now added my git email as an additional email to my Notation Group account, so both are there. I've also linked my git account to my group account. Is there anything else I need to do to create a branch? I don't have that option in the branch dropdown. Unless that's not the protocol.

[infojunkie]

Rob, best if you can fork the repo and submit a PR from your fork. With time, as we iron out the interactions, we'll invite members of the community to work directly in this repo.

Also, I confirm I see you at https://www.w3.org/community/music-notation/participants

[rjjaret]

Hi all,

I've been looking through and playing with the tools a bit. Karim - no other use cases come to mind.

Perhaps a next step would be to come up with some validation needs, in addition to the example at the start of the thread, that can't be addressed with XSD. Then we can judge how the models each handle these.

[infojunkie]

Some other use cases:

• The same labeling issue also exists in segno / dalsegno pairs
• Number of beats in a measure correspond to the current time signature
• <backup> and <forward> don't cross measure boundaries
• musicxmlTestSuite has a number of invalid files. Most of them are caught by XSD validation, but at least 33e-Spanners-OctaveShifts-InvalidSize.xml has semantically incorrect octave-shift values and 45f-Repeats-InvalidEndings.xml has duplicate endings (<ending number="2"> in measures 3 and 4).

I'll add more as I review my projects and more examples come to mind. @lemzwerg , @mscuthbert do you have other suggestions here?

[lemzwerg]

@infojunkie, I've continuously extended and revised the MusicXML tests for LilyPond (while working on musicxml2ly). I'm not completely done yet – it's always far more work than expected...

Please have a look at

https://gitlab.com/lilypond/lilypond/-/tree/master/input/regression/musicxml