The Concept: "Super Frames" (NCTAA)
As an RX 9070 XT owner playing at 120Hz or 240Hz, especially older games, I am proposing a thing I nicknamed "Super Frames."
The core idea is for a "new" anti-aliasing pipeline pipeline from a Causal (Past-to-Present) model to a Non-Causal (Bidirectional) model by utilizing a 1-frame temporal shift (equivalent to a Triple Buffer latency penalty).
The Problem with Causal Filters:
Current temporal solutions (TAA/FSR/DLSS) rely solely on T-1 and T and this leads to persistent "ghosting" and "smear" during disocclusion because the history buffer lacks ground-truth data for newly revealed pixels. The AI is essentially "guessing" the background.
The "Super Frames" Architecture:
Temporal Buffer: The display output is intentionally delayed by exactly one frame (4.16ms at 240Hz).
Bidirectional History Validation: The AI uses this "Future" Frame (T+1) as a Ground Truth Reference to validate the history samples of Frame T.
4D Voxel Integration: By triangulating T-1, T, and T+1, the engine achieves sub-pixel stability and absolute ghosting elimination. Since the future position of an object is known, the "hole" left behind in the current frame can be filled with 100% accuracy using the buffer.
Performance & Latency Trade-off:
At 240Hz, a 1-frame "Look-Ahead" buffer adds only 4.16ms of latency.
I'm not sure if this is something that can be added to AMD FidelityFX FSR Super Resolution in the form of triple buffering. We are essentially using a "offline" way of anti-aliasing, by timeshifting everything 1 frame.
And this can obviously be mixed with upscaling and frame insertion. And if the GPU and CPU have some headroom (not cranked to 100%) then vsync would be viable (no VRR) and would make this buttery smooth too.
They key idea here is unlike with current dual buffer where the future frame (or pixel vector) is predicted. The "AI" can actually know as it has the actual "future" frame.
I'm unsure how feasible this is. But rendering one frame ahead (aka delaying by 1 frame before display) the overhead should be less than say 4xSSAA or even 2xSSA, with the added bonus of having three depth etc. buffers (T-1, T, T+1) to do upscaling and/or generate frame insertion from.
I don't have the skill/or knowhow to to properly explain this, let alone try to maske a proof of concept, so I hope somebody here understand what I mean, and know how to maybe test it?
Tripplebuffer is not knew, but this way of doing it maybe is? There would be more cache/memory overhead as the GPU woud have to hold three frame/depth/geometry buffers at once (T-1, T, T+1).
And I guess competitive gamers would not want 1 frame extra latency. But for "main stream games" (think the likes of the upcoming GTA VI), if this temporal triple buffer would allow better AA, and combines well/can be used with the upscaling and frame insertion stuff too, then good looking (consitent?) FPS is achievable.
Temporal AA always annoyed me due to the "smear", but with high FPS and lower resolution and triple buffers, combined with FSR upscaling etc. The smear should be avoidable now?
In https://github.com/GPUOpen-LibrariesAndSDKs/FidelityFX-SDK/blob/main/Kits/FidelityFX/docs/techniques/super-resolution-temporal.md
I see it says "FSR2 is a temporal algorithm, and therefore requires access to data from both the current and previous frame."
So with my idea I guess it would instead be "requires access to data from both the current and previous two frames" or "requires access to data from the current and previous, and next frame" (depending on how one think about it).
Backwards compatibility:
Only, my idea is not just for super resolution, but also as a antialising methoid (override?)
One could use it with older games, and use the pixel's motion over the three frames to "almost" perfectly antialise it, assumming one triple buffer the frame data (not just the bitmap frame buffer). But I guess one could use AI on even just tripple framebuffers too for those really old games? (I'm just brainstorming here, I'm clueless with game engine pipelines.)
The Concept: "Super Frames" (NCTAA)
As an RX 9070 XT owner playing at 120Hz or 240Hz, especially older games, I am proposing a thing I nicknamed "Super Frames."
The core idea is for a "new" anti-aliasing pipeline pipeline from a Causal (Past-to-Present) model to a Non-Causal (Bidirectional) model by utilizing a 1-frame temporal shift (equivalent to a Triple Buffer latency penalty).
The Problem with Causal Filters:
Current temporal solutions (TAA/FSR/DLSS) rely solely on T-1 and T and this leads to persistent "ghosting" and "smear" during disocclusion because the history buffer lacks ground-truth data for newly revealed pixels. The AI is essentially "guessing" the background.
The "Super Frames" Architecture:
Temporal Buffer: The display output is intentionally delayed by exactly one frame (4.16ms at 240Hz).
Bidirectional History Validation: The AI uses this "Future" Frame (T+1) as a Ground Truth Reference to validate the history samples of Frame T.
4D Voxel Integration: By triangulating T-1, T, and T+1, the engine achieves sub-pixel stability and absolute ghosting elimination. Since the future position of an object is known, the "hole" left behind in the current frame can be filled with 100% accuracy using the buffer.
Performance & Latency Trade-off:
At 240Hz, a 1-frame "Look-Ahead" buffer adds only 4.16ms of latency.
I'm not sure if this is something that can be added to AMD FidelityFX FSR Super Resolution in the form of triple buffering. We are essentially using a "offline" way of anti-aliasing, by timeshifting everything 1 frame.
And this can obviously be mixed with upscaling and frame insertion. And if the GPU and CPU have some headroom (not cranked to 100%) then vsync would be viable (no VRR) and would make this buttery smooth too.
They key idea here is unlike with current dual buffer where the future frame (or pixel vector) is predicted. The "AI" can actually know as it has the actual "future" frame.
I'm unsure how feasible this is. But rendering one frame ahead (aka delaying by 1 frame before display) the overhead should be less than say 4xSSAA or even 2xSSA, with the added bonus of having three depth etc. buffers (T-1, T, T+1) to do upscaling and/or generate frame insertion from.
I don't have the skill/or knowhow to to properly explain this, let alone try to maske a proof of concept, so I hope somebody here understand what I mean, and know how to maybe test it?
Tripplebuffer is not knew, but this way of doing it maybe is? There would be more cache/memory overhead as the GPU woud have to hold three frame/depth/geometry buffers at once (T-1, T, T+1).
And I guess competitive gamers would not want 1 frame extra latency. But for "main stream games" (think the likes of the upcoming GTA VI), if this temporal triple buffer would allow better AA, and combines well/can be used with the upscaling and frame insertion stuff too, then good looking (consitent?) FPS is achievable.
Temporal AA always annoyed me due to the "smear", but with high FPS and lower resolution and triple buffers, combined with FSR upscaling etc. The smear should be avoidable now?
In https://github.com/GPUOpen-LibrariesAndSDKs/FidelityFX-SDK/blob/main/Kits/FidelityFX/docs/techniques/super-resolution-temporal.md
I see it says "FSR2 is a temporal algorithm, and therefore requires access to data from both the current and previous frame."
So with my idea I guess it would instead be "requires access to data from both the current and previous two frames" or "requires access to data from the current and previous, and next frame" (depending on how one think about it).
Backwards compatibility:
Only, my idea is not just for super resolution, but also as a antialising methoid (override?)
One could use it with older games, and use the pixel's motion over the three frames to "almost" perfectly antialise it, assumming one triple buffer the frame data (not just the bitmap frame buffer). But I guess one could use AI on even just tripple framebuffers too for those really old games? (I'm just brainstorming here, I'm clueless with game engine pipelines.)