Interoperability with frame gen methods (in-game and driver level)

[lyndonguitar]

Select Topic Area

Feature Request

Body

Feature Request: Improved Support for Frame Generation in Mixed Refresh Rate/FPS Streaming Scenarios

Use Case:

I frequently use Sunshine to stream games from a host machine with a high refresh rate monitor (typically 144Hz+). To reach high frame rates in modern games, I often enable frame generation techniques such as DLSS Frame Generation, FSR-FG, AMD Fluid Motion Frames (AFMF), or NVIDIA Smooth Motion to boost performance from ~60fps to 100+fps.

However, my client devices, whether a 4K60Hz TV, a 60Hz phone, a Steam Deck LCD, or under bandwidth-saving constraints, are often limited to 60Hz/fps. This 100Hz+ -> 60Hz host-to-client setup is likely a common scenario for many Sunshine/Moonlight users too.

Pain Point:

The pain point comes when using aforementioned frame gen methods, where when streamed with Sunshine with the use case above, caps the stream to 60hz. With Frame gen still kicking in and the FPS capped to 60, there is some conflict. It results in a base frame rate of 30 frame-gen’d to 60. This results in increased input latency and reduced visual quality.

Disabling frame generation solves the issue (restoring an all-native 60fps stream), but only for those that has in-game FG methods to turn off. (Regardless, You'd have to do this everytime if you play back and forth between host and client). For driver-level features like AFMF or Smooth Motion, it requires manually adjusting driver settings each time too.

Feature Proposal:

If possible, I’d love to see Sunshine improve interoperability with frame generation features.
Some potential ideas:

Driver-level Frame Gen Blocker:
A toggle in Sunshine to temporarily block or prevent driver-level frame generation (e.g., AFMF, NVIDIA Smooth Motion). In my experience, AFMF doesn't activate when overlays are present, and there are times that Smooth Motion doesn't kick in, but I don't know exactly the trigger. Perhaps Sunshine can leverage that possibility to suppress these features during streaming sessions. I have also experienced cases where in-game frame gen (DLSS/FSR) can easily be turned off on at will, so that is also another possibility.

Real Frame Capture Only (Ignore Fake Frames):
An option to allow Sunshine to stream only the actual frames generated by the game, ignoring the interpolated frames like what's apparently happening already in this issue report: (see issue #3621). For example, if the game outputs 60fps but is frame-gen’d to 120fps, Sunshine should just capture the real 60fps frames for streaming to 60Hz clients. and this goes without saying that the host FPS shouldn't be capped to 60 in this case (so we wouldn't get 30 fake/30 real frames)

PS. although, even if it ignore fake frames, there would still be undeniably an input lag delay due to the cost of processing frame gen, so a "frame gen" blocker feature is most ideal.

I’m not a developer—just a long-time user— so I don't know if this is easy to implement or maybe even impossible, but I believe if possible, this would help significantly in delivering a smoother user experience when using Sunshine and modern frame generation methods.