Effort involved with extending the model

[BnSmth]

Hello,

I'm a complete beginner to OCR (I did a CS degree about 10 years ago so I feel like an almost complete beginner to ML as I haven't done it since my dissertation). I've been developing an app to teach myself to sightread (and mainly learn to supercharge my development with claude code).

As part of the app, I have been looking at pulling in scans of music that I have laying around and sticking them in to the app which then detects how well the song is being played. I have been very impressed with homr on the whole and it was remarkably easy to get hooked into my codebase.

One thing that I have noticed is that while it is very good at picking up staves, clefs, bar lines, time signatures, notes, rests etc, it doesn't deal at all with other markings such as note relations (slurs etc), dynamics, articulation and ornamentation.

I'm slightly tempted to use this as a project to relearn some ML and see how good claude is at teaching me something new. Is this an incredibly stupid idea and there is a very obvious reason for these not to have been included in this model?

Thank you for the great work that you have done on this, I have really enjoyed using it in my little project.

[liebharc]

Hi!

Your background sounds similar to mine. I found oemer and TrOMR about 2-3 years ago and wanted to make an app which supports children in learning their instrument. I would be curious to see your app, if you are willing to share it.

homr with version 0.6 has some support of articulations. There is work to be done, but first make sure that you are on the latest main. Working on the model can get very time-consuming, but there might be a good starting point to get into this: Slurs&ties are currently disabled because they work very poorly. As of now, it's unknown why they work so poorly. In the best case, someone just needs to fix the XML generator code. In worse case, someone would need to check the training data and rerun training - which is still doable as the training data contains slurs&ties. If you want to try this, then the best start with looking at the homr internal vocabulary. As soon as you understood that, you can see if the transformer model detects slurs&ties correctly. If the transformer output is correct, then the issue is in the XML generator, otherwise you would need to continue investigating the training code. And at the end of that journey you likely have enough understanding of the system to think about extending the supported symbols.

Of course, this is just one way, which I believe allows you to get into the direction you mentioned, but you can also pursue other directions.

LLMs like Claude&Co can help you there. I use them as well. As usual with them, you just have to be very careful as they send you down a wrong path very frequently.