Intelligent Metronome

[corrados]

This issue has been migrated from Sourceforge. Created: 2020-04-03
Creator: Petra-Kathi

An idea to minimize latency differences:

The server can detect latencies of individual clients by clock
comparisons of respectively transmitted packets, so it should be able to
advise the individual clients to create metronome ticks (not audible
with the other participants) with individual latencies / time shifts of
some (ten) milliseconds , so the strokes are heard with best relative
timing.

The individual stroke timing should be derived from observation of
the ping timing over the last, say, 5 seconds, so it would do some
averaging against individual extraordinary retarded packets.

The participant with the slowest line should then get the stroke first, and the
other ones appropriately retarded by some milliseconds. The result
should be a better aligned combined sound, even though that would be
retarded by about the amount of the latency of the slowest participant
(single direction latency), but rather consistently.

[vocobox]

The idea is fun but would probably be a mess. Bad latency might maybe also unstable, meaning that your local metronome has a speed varying with the network latency. It also implies adding lot of sychronization information for this protocol to work instead of the current "best effort" mode of jamulus.

I personally like the approach of Jamulus described in Volker paper as following the "keep it simple principle".

Handling latency smartly is at the heart of Ninjam. I think this is good to have distinct software having distinct functions.

But it's just my opinion :)

[corrados]

I totally agree with you. I do not like the idea of having a metronome in Jamulus. If you need a metronome, something else is not configured correctly or you simply cannot jam in realtime because of some physical restriction (too far away from each other, your internet provider does not have low ping times, etc.).

[Petra-Kate]

The idea is fun but would probably be a mess. Bad latency might maybe also unstable, meaning that your local metronome has a speed varying with the network latency. It also implies adding lot of sychronization information for this protocol to work instead of the current "best effort" mode of jamulus.

The idea could be implemented in various levels of "intelligence": 1st level: Just implement an independent metronome ticker on the server as the central coordination point, and thus spreading a timing normal unconditionally as even as possible to every participant, and presumably split the average metronome latency by two, but by much more as far as the individual participants of a session are concerned. Especially so if they experience about the same individual latency relative to the server. 2nd level: Implement a short calibration phase, to be invoked willingly: measure *average* response times for a short period (like 3 s) based on individual member returns of pulsed test signals to each of them. Modify subsequent individual metronome transmits according to this (static) measurement to equalize expectable latency differences. Will not catch sudden quasi-permanent changes in network effects between calibration runs. 3rd level: Add a local, non-calibrated time stamp information to UDP packets sent to the server. Single precision integer (with overrun detection) would be completely satisfying, IMHO. Derive changing *relative* latencies from comparing their drift relative to each other (assumption: individual computer clock is sufficiently precise for the duration of a rehearsal). Do this on a several seconds averaging, so flatten out unmanageable jitter effects. Derive slightly deviating latencies and adjust relative metronome beats accordingly.

I personally like the approach of Jamulus described in Volker paper <http://llcon.sourceforge.net/PerformingBandRehearsalsontheInternetWithJamulus.pdf> as following the "keep it simple principle".

I agree completely to the KISS strategy but I wouldn't think that my suggestion violates it. I'd consider the increase in complexity for the suggestions above as quite small, compared to the compexity of the existing Jamulus as it is today. Cheers, Petra

[vanlaerav]

Hi, metronome is helping a lot! We use this..

The way we do is:

We add a member “clicky” who is running on a laptop in the same network as the server. Only the drummer is listening to “clicky” and we are trying to listen tintje drummer (it was allready hard in our rehearsal studio 😂). It works fine, except for the drummer, it is some times really difficult. But it prevents us to slow down during the song.

What a really great program Jamulus is!!! We use it during Corona for one month, it is really fun and works!!!

[Petra-Kate]

Hi, metronome is helping a lot! We use this.. We add a member “clicky” who is running on a laptop in the same network as the server.

This is about the method we use presently as well, since one of us (me) ist typically in the same network as the server. I use a smartphone metronome lying on the keyboard. But if we are more spread (begining next week) this method will double the latency, hence my suggestion. I should mention, perhaps, that we try to organize classical music over Jamulus, therefore no such leading person with us usually, and quite some pauses that tend to pull us slow, since at the end of the pauses each is trying to listen to the others. Cheers, Petra

[vanlaerav]

The metronome is helping us for rock songs above 110

We decease in tempo... but with the version 3.5 and behringer IO we manage to keep the delay under the 29ms. And we live in Hilversum, Loosdrecht, Ankeveen around 10 kilometer apart..

So only the drum, is maintain the tempo...

[corrados]

With my band we can play rock songs with tempos like 170. We all only hear our own signal which comes back from the server. That way we keep in sync.
The thing which kills the synchronization is if players have a pretty high latency and do direct monitoring instead of listening to their own signal coming back from the server.
If I enter a Jamulus server and play with people, I instantly feel if they have setup their hardware properly and then we play tight together. If people listen to their direct signal, you instantly start to hear that they do not play tight to what you play and it actually does not make fun playing together. This is my personal experience.

[mirabilos]

If you need a metronome, something else is not configured correctly or you simply cannot jam in realtime because of some physical restriction

This is not entirely correct. A metronome also helps participants to stay on tempo, because otherwise, the listen to each other and thus the slight, almost non-noticeable delay adds up to something really noticeable. (I haven’t yet used Jamulus, working on a proper Debian package for it first, but the person who recommended it to me on the OSAMC mailing list noted that in her tests of it.)

[trebmuh]

As a musician, you never ever need a metronome to rehearse or to play within a band. The metronome is useful for practicing your instrument alone.

As a music teacher, I always tells my student to use it when they practice but never when they play in a band. What you need when playing in a band is to listen to the other musicians you're playing a song with, and not to listen to a little plastic and electronic who knows nothing about music. In other words, using a metronome in a band is just killing the musicality. Mostly because it's annoying for people who don't need it (audio pollution), and the people who think they need it, needs in fact to listen to other musicians (and to work their instrument at home, even maybe with a metronome).

Not to mention that it'll not cope with any of the songs you'll play which need tempo variation (even if slight BPM variations to enlight and strengthen small mood change in a song).

Having a metronome on jamulus is a good recipe for a musical experience disaster in my opinion.

[...] for rock songs [...] So only the drum, is maintain[ing] the tempo...

That is pretty much how any rock band is functioning all over the world 😉

[Petra-Kate]

..., and the people who think they need it, needs in fact to listen to other musicians

This is exactly what should be *avoided* while using Jamulus: As I stated already before, and some others in here confirmed, listening to the fellow musicians will inadvertently lead to a slow-down due to the always present latency. As it is small, you *think* you can orient yourself by listening to your fellow musicians, but you can't. This is even more so if there are some pauses in your voice - which may be rather seldom in rock/pop but is quite frequent in classic music. In addition, with most classic pieces there ain't such a thing as a drummer. :-) (Okay, there are exceptions like Bolero, but I would like to play some other pieces as well ...) Cheers, Petra

[vocobox]

1st level: Just implement an independent metronome ticker on the server
as the central coordination point, and thus spreading a timing normal
unconditionally as even as possible to every participant, and presumably
split the average metronome latency by two, but by much more as far as
the individual participants of a session are concerned.

@Petra-Kate I don't understand what you mean by "split the average metronome latency by two". We are enslaved by latency, we do not have control on it. Could you clarify that point?

listening to the fellow musicians will inadvertently lead to a slow-down due to the
always present latency. As it is small, you think you can orient
yourself by listening to your fellow musicians, but you can't.

👍 very interesting. How would you estimate this theoretically? I guess my below reasoning is wrong. Would you be able to make something accurate?

Assume the worst bandmate latency is L - expressed in millisecond, the speed is S in BPM, and the duration is D minutes. Is this right to estimate the song will be stretch by LxSxD milliseconds? With this reasonning, a 3 minutes song at 60 BPM with a 30 ms latency would last 5.4 more seconds.

[Petra-Kate]

1st level: Just implement an independent metronome ticker on the server as the central coordination point, and thus spreading a timing normal unconditionally as even as possible to every participant, and presumably split the average metronome latency by two, but by much more as far as the individual participants of a session are concerned. @Petra-Kate <https://github.com/Petra-Kate> I don't understand what you mean by "split the average metronome latency by two". We are enslaved by latency, we do not have control on it. Could you clarify that point?

For the calculations I just assume some numbers: Average latency 50 ms, jitter 5 ms (playing across Germany, with someone situated in a "retarded" area :-)) If the person with a 50 ms latency is providing the metronome it will arrive at the others players 100 ms later (due to backward latency of 50 ms and then 50 ms forward latency again. In addition the jitter will increase, once again due to double passes, to averaged probably 7.5 ms If we have an autonomous metronome at the server it will send its strokes independently, so they will arrive with the players about 50 ms later (depending on the individual latencies, of course), but with all players delayed mostly similiar. So the span of latency is split roughly by half. Jitter will be the one experienced by any player individually as well, so it remains the old value.

listening to the fellow musicians will inadvertently lead to a slow-down due to the always present latency. As it is small, you /think/ you can orient yourself by listening to your fellow musicians, but you can't. 👍 very interesting. How would you estimate this theoretically? I guess my below reasoning is wrong. Would you be able to make something accurate? Assume the worst bandmate latency is L - expressed in millisecond, the speed is S in BPM, and the duration is D minutes. Is this right to estimate the song will be stretch by LxSxD milliseconds? With this reasonning, a 3 minutes song at 60 BPM with a 30 ms latency would last 5.4 more seconds.

I'd like to reason on the basis of relative deceleration: Let's assume we have 120 BPM, hence 2 beats per second (PBS). If each beat is delayed by 30 ms we have a retardation of 60 ms every second, or roughly 1/20 of the present speed. The effect will decrease over time since the number of beats per unit time will decrease, but nevertheless speed is constantly slowing down. I'm too lazy to do an exact mathematical calculation right now, but you will arrive at a (near-to) stand-still very soon: After about half a score sheet you will arrive at approximately half the original tempo. And this is excactly what we were observing when trying to listen to each other as we are used to from live playing. Of course this is not practically correct as every musician will listen at least partially to his/her inner metronome, but the more you tend to listen to your fellows the stronger this effect is. (Btw.: I guess this is one of the principal duties of the conductor during the concerto: to define an optical lead to the orchestra by his stroking. Luckily the velocity of light is slightly larger than that of sound. :-) Mind the fact that we have a delay of 3 ms/m in a live performance, so for large orchestras with distances of 20 - 30 m between left-most and right-most player this should play some role already, at least for really fast passages.) Cheers, Petra

[trebmuh]

This is exactly what should be avoided while using Jamulus: As I stated already before, and some others in here confirmed, listening to the fellow musicians will inadvertently lead to a slow-down due to the always present latency. As it is small, you think you can orient yourself by listening to your fellow musicians, but you can't.

Then one is not using the right tools and/or it's misconfigured. If in a band playing configuration one musician have to stop listening to other musicians, then it's a fail. Playing in a band is about listening to other.

This is even more so if there are some pauses in your voice - which may be rather seldom in rock/pop but is quite frequent in classic music. In addition, with most classic pieces there ain't such a thing as a drummer. :-) (Okay, there are exceptions like Bolero, but I would like to play some other pieces as well ...)

Yes, that is right, there is not a drummer in classical music, but there is a "conductor"... and we were talking about a rock band here. Moreover, you're giving me an example to proove my point even more with this example, in classic, they is more musicians and the music is even harder to play. And guess what ? Nobody is listening to a metronome 😉