Low framerate when mirroring

[fazo96]

Hi, I'm using this software from linux to mirror my monitor, but the framerate is really low (around 1 FPS I guess) while it's very high when streaming from OSX.

What can I do to improve the framerate? From the readme you list some server software with a column showing mirroring support, so I guess this client should support mirroring, but from the code it looks like it screenshots the desktop and sends it as a photo instead.

[loretoparisi]

@fazo96 as far I can see there is a PhotoThread class that does

Thread.sleep(Math.round(0.9*timeout));

that is the sleep while sending a new frame:

airplay.photoRaw(image,NONE);
Thread.sleep(Math.round(0.9*timeout));

so you can try to change this.

[jamesdlow]

That line isn't how fast it sends the image, that tells it to resend the image again just before the TCP connection times out.

It tries to send the frames if it's changing as fast as possible. The reason why it's slow is that it's using the AirPlay image protocol to send individual PNGs of the screen.

The true AirPlay mirror protocol creates a HTTP server and MPEG stream of the screen, which this library does not support.

[loretoparisi]

@jamesdlow ah right! thanks for the clarification then, so it's basically doing a frame by frame image mirroring rather than a video stream.

[fazo96]

Thank you guys for the help 👍

Is it possible to implement the actual desktop streaming protocol into this library? I'm quite skilled in Java and @jamesdlow's code looks very readable, but I know nothing about airplay. I could look into implementing the functionality but it has to be well documented somewhere (I'm no good at reverse engineering) and I need to find a good enough way to record the desktop.

[loretoparisi]

@fazo96 the original and somewhat unofficial docs it's here https://nto.github.io/AirPlay.html#screenmirroring-httprequests
There are two endpoint for stream GET and POST, so it could be a starting point.
The streaming is Apple HLS, also this node implementation seems to have the HLS implementation, but I didn't try - https://github.com/zfkun/node-airplay/blob/master/airplay/hls.js

If you look at the code it creates a HTTP local server to handle the stream

HLSServer.prototype.start = function ( port ) {
    if ( !this.started ) {
        this.started = !0;

        this.baseURI = 'http://' + IP_LOCAL + ( port === 80 ? '' : ':' + port );

        this.server = http.createServer( this.httpHandler.bind( this ) );
        this.server.listen( port, IP_LOCAL );

        this.emit( 'start', { host: IP_LOCAL, port: port } );
    }

    return this;
}

[loretoparisi]

the author seems to be very prolific about AirPlay streaming and he has a Cocoa version too for HLS streaming - https://github.com/zfkun/iairplay

[jamesdlow]

Yes, basically the original AirPlay protocol is over HTTP. You either send an image or URL to webserver serving audio or video. The URL can be on a different server to the client sending the request as in when you AirPlay a video from YouTube on your phone, the AirPlay request is sending the URL of the MP4 on YouTube's servers. You can also receive playback information on the client device, but I'm not sure if that's done as a server or as information received back over the original HTTP request.

The AirPlay protocol changed recently with the new AppleTV, so I'm not sure how much is different to the above description.

In the case of mirroring though you have a web server running on the host computer which serves a stream of the desktop. It would be entirely possible to integrate it into this library, but I personally don't have the time right now to do it.