[Feature discussion] Socket.io extension

[arthuro555]

Description

The p2p extension is cool but it has some limitations and is not the fastest. Some limitations are that if you want a server architecture, people can potentially steal your connection UID and take control of the server, plus you need to basically run the server in a headless browser (not the best in resources management). It also has the annoying 250 simultaneous connection limit.

Solution suggested

We could adapt the asynchronous to synchronous queueing system of the peer to peer extension for socket.io and socket.io-server, to let users write their own servers (only catch is servers would only work in a node environment) and clients. This would allow making real lan games, reliable server client architectures, an architecture like Minecraft (if you multiplayer host a server) etc.

Alternatives considered

We could also use websockets but socket io has fallbacks in case they are not supported and adds auto reconnect on unreliable connections.

Prerequisites

We need two things to be done before implementing this issue:

• Let people use the p2p extension for a while to be sure it's event system is reliable.
• NPM dependencies ([Final Review in Progress] Add dependencies and properties to Extensions #1717) for socket.io-server

[bil-ash]

My opinion-
The current approach of using peerjs(UDP) for multiplayer is quite OK.This can even power 3 v/s 3 or 1 v/s 5 games like Mini Militia because one user can connect to 5 others without any server and connection glitch.
Socket.io isn't suitable for LAN games,real-time multiplayer or anything (like PUBG) which requires too many players(100) in a single room(since it operates over TCP).
For such cases where a server is required, maybe geckos.io can be used which has a similar API to socket.io and has built in support for rooms .

Also, eagerly waiting for your Firebase extension PR to get approved and thanks for the peerjs extension.

[arthuro555]

Since gecko and socket.io are so similar, why not provide both at once? The difference is basically TCP vs UDP, and both always had had their defaults. UDP may be faster but it is less reliable and more vulnerable (more vulnerable as it is easier to spoof the IP the packet came from). I think I could easily put in the connect to server action a toggle to choose which one to use.

Socket.io also has been throughly tested, is trusted by big enterprises, and isn't dependent on newer technologies as it has fallbacks for older browsers, so trashing it away because another library claims to have a little bit less latency might be a bit of an excessive reaction ;)

Thanks you very much for your suggestion of using gecko tho as it might indeed be interresting as an alternative backend for providing UDP.

[4ian]

Thanks both! These discussions are very interesting.

Since gecko and socket.io are so similar, why not provide both at once?

Remember that while giving the "choice" seems naively better, it's also more things to maintain, the double of stuff to test, more confusion for the users... in some cases, "less is more" as we say :)
For such an extension, it might be important to be opinionated from the beginning, so I encourage the discussion to continue to understand things like performance (remember we're making games, something that is ok for a chat might not be ok for a fast paced action game which might also need things like interpolation of positions to be smooth), reliability, existing community around the library, ease of implementation :)

Network is notoriously something complicated - very few, if any, game engine managed to make it simple. That's a huge challenge for GDevelop, so we need to document ourselves a lot and understand what are the main challenges before! :)

[bil-ash]

So may be we should wait for some time(ideally, a month) to see how well the peerjs extension ( as mentioned above by @arthuro555 ) works and problems(if any) .
Also,I am planning to make something using both ajax (gdevelop network extension) as well peerjs . Lets see what are the outcomes and learnings from the attempt.

[arthuro555]

Geckos io made a handy little table comparing meet js, socket io and geckos io:
https://github.com/geckosio/geckos.io#when-to-use-socketio-geckosio-or-peerjs
Geckos is only better at sending fast and many updates, but this makes it way less reliable.
Geckos also requires signaling servers as it uses WebRTC, which might be tedious to set up.

For now, I would therefore stick to socket.io due to it's wider application range. If we get a high demand for UDP/faster real time networking we can always look back into gecko.

[bil-ash]

Geckos is only better at sending fast and many updates, but this makes it way less reliable.

It provides a way of sending reliable messages which can be used for turn-based multiplayer whereas the default non-reliable mode can be used for realtime multiplayer.

Geckos also requires signaling servers as it uses WebRTC, which might be tedious to set up.

It does not require external signaling servers,that's built-in. The README mentions about external ICE servers which defaults to google-provided public ICE servers, thats also the case for peerjs . Also, after reading your comment and searching on npm, I came across this ICE server for node. (I haven't tested this server with geckos.io.)

Geckos io made a handy little table comparing meet js, socket io and geckos io:
https://github.com/geckosio/geckos.io#when-to-use-socketio-geckosio-or-peerjs

There is an open issue regarding the table.

[arthuro555]

It provides a way

Capable of something != Good at that thing. Gecko explicitly says that it shouldn't be used with the reliable option as a default and should only be used as a last resort.

thats also the case for peerjs

The question is not about what p2p does but what library we want to use for socket networking ;)

In general I personally prefer to rely on proven technology. Gecko is not very used (in comparison to socket.io) and is therefore more prone to bugs/issues, and those could make GDevelop look unprofessional.

Socket.io is definitely good enough for anything any GDevelop user would want to make, and is a safer bet.

Don't get me wrong gecko is not bad or anything, it is just that a choice has to be made, and sockets.io is just more reliable, more likely to stay maintained, and reliable connections are easier to understand than unreliable ones. Imagine how many users would be confused as to why some of the events they send never arrive!

If someone makes a game big enough and know what they are doing and are trying to optimize to the maximum, and switching to UDP would become worth it, they probably would have the resources to implement it themselves.

[4ian]

Does this mean though that with socket.io, we close the door to "fast action paced" games, only officially supporting turn by turn or games with no issue of latency?
Not necessarily good/bad (being opinionated about the use case can help doing something that works very well, rather than trying something too generic), just trying to understand what these choices would imply in terms of capabilities.

[Keevle]

This is just people being ignorant to actually look into things and following the AAA crowd that always says tcp is slow @4ian many fast paced games use TCP(Websockets) Krunker
Surviv (Recently purchased by Kongregate games) to name 2 off of the top of my head,You can check them out yourself and when you do youll see that you dont need to rush to use webrtc to actually have a fast paced games,As a IO game developer i can tell you now that No dev uses Webrtc for any games in production its only toyed with in experimental projects.Im not saying Webrtc will remain in this state of forever experimental,Just that the available options of using Websocket arent as slow as most people who dont make or play them think

[bil-ash]

At the beginning,I would like to state that I haven't made any game using either socket.io/geckos.io, so I do not have any first-hand experience.

I was leaning towards webrtc because I came across some articles -one of the published by chrome team which state that current websocket API isn't suitable for very fast realtime updates. Also, construct 3 (most probably construct 2 also) uses webrtc for multiplayer and webrtc has pretty good browser support

However,majority(in this thread) seems to be on socket.io side, so let it be socket.io . If required, I will make a geckos.io extension later on.

[arthuro555]

Yes I know that geckos is potentially faster for games with many updates, but reliability is probably more important for most users.
Most users will try to do something like an increment variable event to increment a variable counter. If one packet misses, there will be a mess of desynchronized counters!
While he ko technically can have a reliable mode, it is way less optimized as it still uses an unreliable UDP data channel in the background.

To sum it up, my main argument in favor of socket.io is that we want an easy solution for users. Reliability is something they expect.
While UDP is in theory faster, it only is in case of loss of unnecessary packets. UDP is only good for people who are experienced and know what they are doing and really want to optimize at the maximum and have a very huge game. Making it available would cause users to use it as it says it is faster without realizing the consequences and introducing weird bugs they wouldn't understand (why doesn't my remote event always trigger when I want it to???).

My second argument is that socket.io is a big library heavily used by big tech companies (Trello for example), so even if it would stop being maintained one of those companies would fork it for sure and continue maintaining it. It has also been around longer and has therefore more stability.

Finally my last argument is that really fast games with many event calls might not work well at all due to the GDevelop event loop. If we use the same system as for p2p, there will always be a limit of 1 event that can be processed every frame. To really optimize at the maximum you would need to handle events with asynchronous JavaScript anyways, so it wouldn't be that difficult to also add gecko as include through the DOM or by just pasting the JavaScript file into a JavaScript event.and use it.

[4ian]

If one packet misses, there will be a mess of desynchronized counters!

This is something that is more complex I think. Synchronization loss will happen, whatever your solution is. It's not a "if" question, it's a "when" question :)
Which means that your game/server should be robust against this and periodically resynchronize important stuff - it's maybe (probably, certainly?) a complexity that we want to solve for our users in a "easy to use" multiplayer extension.

Seems like some games, like in Valve Source engine, are having an authoritative server sending very frequently the state of the world (i.e: position of objects basically, like the players). See "lag compensation" and "input prediction" on this article: https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking

Another article: https://medium.com/@qingweilim/how-do-multiplayer-games-sync-their-state-part-1-ab72d6a54043

really fast games with many event calls might not work well at all due to the GDevelop event loop

We should not consider the events sheet as a bottleneck. There are many way of handling more than one event, with a custom event type that would repeat for each received event, with something like a condition that "unstack" the received event each time a condition is called, etc.. in another words something to make a loop allowing to treat more than one event.

Note that what I wrote here is not telling if we should go for one library or the other. Just, that it's more complicated than just "choosing a fast library or a reliable one" (all of this is if we want an extension that is higher level than the current "peer to peer" extension and that allows to build multiplayer games easily. If we just want to "expose" the API of socket.io/gecko, it's easy but will certainly not be helpful to users that will be frustrated by the fact that it's too low level and they can't do their dream multiplayer games).