UDP fragmentation

[softins]

https://github.com/corrados/jamulus/blob/017796919c814a74adcf916a6f200e7c157e58f1/ChangeLog#L19

Going down from 200 to 150 won't remove UDP fragmentation. In a Wireshark trace a couple of weeks ago, a list of 198 servers took 8 fragments, which when reassembled had a payload size of 6780 bytes. The fragments in that instance appeared to have an MTU of only 778 bytes, which is something over which neither end has control, as it may be due to an intermediate hop. Even if the path had the maximum possible MTU of 1500 bytes, it would still take around 4 or 5 fragments to send a list of 200 servers.

The only way really to solve it would be for the server also to listen on a TCP port (could use the same port number). A client wanting the server list could then connect to the central server on TCP, fetch the list and then close the connection. Fragmentation should no longer be a problem then.

For backward compatibility, a client could fall back to UDP if the TCP connection was refused, and the central server would still serve UDP requests for the server list if asked.

(Of course, it is only broken or misconfigured routers or computers that do not correctly handle fragmented UDP packets)

[corrados]

Wie will have multiple server lists in the near future. Each list size can even be further reduced to, e.g., 100 or less servers. This will solve the issue.

[corrados]

Users reported not long ago that NorthAmerica list worked but not the Default. At that time the NorthAmerica list already had about 100 entries so I assume that this will work.

[streaps]

IMHO protocol implementation issues shouldn't dictate the size of the server list and force a fragmentation into multiple central servers. This is really a bad design. Before I read this issue I already wondered (as a user who discovered Jamulus) why there is a limit of 200.

I don't know the the code and the protocol, so I don't know how hard or easy it is to change it (compatibility to older clients and servers,...). There is also QUIC as an alternative to TCP and plain UDP. QUIC also would make it possible to connect with a web client in the future.

[corrados]

This is really a bad design.

Well, it worked remarkably well the last 14 years ;-).

shouldn't dictate the size of the server list

The large server list does not only have UDP fragmentation issues but others as well. E.g. the client pings all the servers in the list every some seconds. This is a fundamental thing. It is much better to have multiple small lists.
Fortunately, pljones voluteers to host two more lists. So we will have four lists with the next Jamulus release (hopefully this weekend) and maybe even more in the future.

[streaps]

This is really a bad design.

Well, it worked remarkably well the last 14 years ;-).

I wonder why there is an open issue about it, if it still works well. ;)

[corrados]

Well, it seems you are a bit impatient ;-)

[lefty665]

I made this comment a couple of days ago on sourceforge but was redirected here.

Hi Guys, Help please. I'm in the US, and getting erratic connect lists. Sometimes it's full lists, sometimes partial lists, and most often no lists. Loading with --showallservers the servers with ping times, and not all have them, look a lot like the partial lists, when they show up at all. Most times it works the first time after a boot. Simply closing Jamulus and reopening it does not re-enable server lists. Once it quits giving lists it's done until a reboot. Older 8 core computer @4gHz, 16gb ram, but poky DSL, although far faster than spec minimums. Any suggestions, ports I should look at, etc are appreciated.

Other than connection issues I'm a fan, Jamulus has let me keep up with band members and musician friends while we're sequestered. Thank you.

ps. a fellow on sourceforge embraced the idea of turning off the stateful packet inspection portion of a users firewall and indicated he was going to add it to the Wiki Jamulus post. Doesn't seem like a great idea. https://sourceforge.net/p/llcon/discussion/533517/thread/0e9aa52428/?page=1

Thanks for your help.

[gilgongo]

@lefty665 The forthcoming update (3.5.4) implements a feature designed to help with the issue of seeing fewer servers than you should. This may or may not cure it, but it would be good if you could test it out. Watch out for the release announcement.

As to the firewall issue, I see they have replied to that, but essentially you will be safe without SPI unless you are doing something pretty non-standard with your network. In which case you would know whether you need SPI or not.

[corrados]

Loading with --showallservers the servers with ping times, and not all have them, look a lot like the partial lists, when they show up at all.

Do you always see the full list if you use --showallservers (ignoring that not all servers have a ping time)?

[gilgongo]

Do you always see the full list if you use --showallservers

That's certainly my own experience. Would be interesting to know about others.

[corrados]

That's certainly my own experience.

If this is the case, then we are in the wrong Github Issue here. Because if you see the full list with --showallservers, you are not affected by the UDP fragmentation issue (see the title of this Issue) since you can receive the complete list. It is then an issue with the ping messages which are suppressed.

[gilgongo]

Ah OK. I thought the two things were connected.

[lefty665]

Loading with --showallservers the servers with ping times, and not all have them, look a lot like the partial lists, when they show up at all.

Do you always see the full list if you use --showallservers (ignoring that not all servers have a ping time)?

No, it will show a full list once, occasionally twice, then a blank data list until it is rebooted. If I had to guess it would more likely be a ping issue. Jamulus apparently generates a lot of pings. Could it be overrunning a buffer or exceeding a threshold? But, that's just guesses from what I can see, not an informed opinion from getting my nose in the code. Thank you for your response, and for Jamulus.

[lefty665]

@lefty665 The forthcoming update (3.5.4) implements a feature designed to help with the issue of seeing fewer servers than you should. This may or may not cure it, but it would be good if you could test it out. Watch out for the release announcement.
As to the firewall issue, I see they have replied to that, but essentially you will be safe without SPI unless you are doing something pretty non-standard with your network. In which case you would know whether you need SPI or not.

I'll look for 3.54, thank you.

In reference to the firewall issue, I don't see that response, please be kind enough to direct me to it. - I found your response on sourceforge. My response is "horsefeathers". Your explanation of web threats is naïve at best. Having both hardware/router and software/os firewalls is entry level web security.

I do see from the wiki that recommending disabling SPI is now included. SPI is a fundamental feature of many better router firewalls. It is concerning that you would recommend disabling it to a user community that is often not well versed in web threats. In addition to potential damage to user computers your recommendation to trusting users creates potential liability for damages they may incur. I encourage you to remove that recommendation.

Tell me please, is Jamulus "doing something pretty non-standard with your network" traffic that would run afoul of SPI? Thanks. https://en.wikipedia.org/wiki/Stateful_firewall

[gilgongo]

SPI is a fundamental feature of many better router firewalls.

That discussion isn't relevant to this ticket. Please continue it on the forum thread.