Skip to content

Commit bb1314e

Browse files
cubanismoianelliottusnvlducoddhack
authored
Add the VK_EXT_present_timing extension (#1364)
* WIP: Add the VK_EXT_present_timing extension This extension allows an application that uses the VK_KHR_swapchain extension to obtain information about the presentation engine's display, to obtain timing information about each present operation, and to schedule a present to happen at a specific time. Applications can use this to minimize various visual anomalies (e.g., stuttering). * Major rebase and updates to the original proposal This change rebases the original extension on main. It also brings many modifications from discussions within the Khronos SI group and this MR: - Add a proposal document - Rely on VK_KHR_present_id to identify presents - Split out VK_PRESENT_MODE_FIFO_LATEST_READY_EXT - Add physical device and surface capabilities - Define "presentation" as a set of well-defined multiple steps - Explicitly describe the internal results queue interactions - Add many per-present parameters, allowing to remove swapchain state - Refine VRR support - Cleanup appendix and update contributors - Replace present slop with a flag to round to nearest refresh cycle - Refine mechanisms for feedback about time domains and timing properties updates - Allow timing results to be returned incomplete and out of order * Address latest review feedback - Time Domains - Rename vkGetSwapchainTimeDomainsEXT -> vkGetSwapchainTimeDomainPropertiesEXT - Add a version counter to the time domains for consistency with timing properties - Change VkSwapchainTimeDomainPropertiesEXT to hold a list of time domains - Change parameters of vkGetPastPresentationTimingEXT to use new structs * Mark some issues as resolved. Also fix some boilerplate language / copyright. * Add interaction with VK_PRESENT_MODE_FIFO_LATEST_READY_EXT * Fix latest spec build issues - Copyright date fix - Finish renaming minPresentDuration -> presentDuration - Add missing VkStructureType and fix spec language * Fix new consistency checks after rebase * Remove unused VkStructureType Also reorder structure types to be consistent with most other extensions. Fix some structures using old VkTimeDomainEXT instead of promoted VkTimeDomainKHR. * Change reserved swapchain create flag bit after rebase * Add flags to control present timing queries Update code examples accordingly * First pass at remaining issues - Removed VK_PRESENT_STAGE_IMAGE_LATCHED_BIT_EXT as we couldn't agree on its semantics. Replaced with VK_PRESENT_STAGE_REQUEST_DEQUEUED_BIT_EXT which should be clearer, as "dequeue"'ing a request is already mentioned in the spec. - Swapchains that require presents to be made before being able to return timing properties should now return VK_NOT_READY from vkGetSwapchainTimingPropertiesEXT - Renamed "variableRefreshDelay" to "refreshInterval". Spec wording is still a bit fuzzy, but this allows to express scenarios where the vblank interval is decoupled from the display's true refresh rate, like Android's ARR. - VkPastPresentationTimingEXT now also reports the targetTime that was provided by the app. This is a usability thing that avoids the need to keep track of previous present times in the app. - Removed targetPresentStage: it didn't make sense from API perspective. Presentation engine now makes a "best effort" approach to hit VK_PRESENT_STAGE_IMAGE_FIRST_PIXEL_VISIBLE_BIT_EXT with the target present time. This is unfortunately the best we can do at the moment due to multiple platforms restrictions. As a result, presentAtNearestRefreshCycle is now merely a hint instead of a requirement. - Added description for the remaining issues in the proposal - Move vkGetSwapchain*PropertiesEXT optional counter parameter at the end. This is closer to the overall API style and fixes issues with vulkan.hpp generation. - Reference VK_KHR_present_id2 instead of VK_KHR_present_id - Cleanup VkPresentTimingInfo Remove the VkPresentTimeEXT union, as it does not fit the style of Vulkan Remove boolean members and replace them with a new flags type. Cleanup some spec language and VUs. * Fix new [NOTE] styling * Typo fixes * Proofreading pass. Fixed a few inconsistencies in member names, parameter orders, etc. * Add an 'unknown' value for VkSwapchainTimingPropertiesEXT::refreshInterval * Fix oversight for target present times using a stage-local time domain When using a VK_TIME_DOMAIN_PRESENT_STAGE_LOCAL_EXT time domain to specify absolute present times, it was impossible to specify which stage to reference. * Fix proposal header to match new style * Proofreading pass. Explicitly state that targetTime may also be a duration in a few places. * Clarify vkGetPastPresentationTimingEXT behavior for returned present stage results. Also remove the by-present-id requests and add proper VUs for presentStageCount. * Fix present timing's swapchain createInfo bit position. The extension did not preempt a createInfo bit back then, and it hasn't come up frequently. I will reserve bit 9 in a separate PR. * Fix old style writing identified by CI * Fix presentAt device features to be optional in the xml * Mark ratified --------- Co-authored-by: Ian Elliott <[email protected]> Co-authored-by: Lionel Duc <[email protected]> Co-authored-by: Jon Leech <[email protected]>
1 parent 69b9214 commit bb1314e

File tree

12 files changed

+2013
-102
lines changed

12 files changed

+2013
-102
lines changed

‎appendices/VK_EXT_present_timing.adoc‎

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
// Copyright (c) 2017-2024 Khronos Group.
2+
//
3+
// SPDX-License-Identifier: CC-BY-4.0
4+
5+
include::{generated}/meta/{refprefix}VK_EXT_present_timing.adoc[]
6+
7+
=== Other Extension Metadata
8+
9+
*Last Modified Date*::
10+
2024-10-09
11+
*IP Status*::
12+
No known IP claims.
13+
*Contributors*::
14+
- Ian Elliott, Google
15+
- James Jones, NVIDIA
16+
- Jeff Juliano, NVIDIA
17+
- Daniel Rakos, AMD
18+
- Daniel Stone, Collabora
19+
- Daniel Vetter, Intel
20+
- Aric Cyr, AMD
21+
- Faith Ekstrand, Intel
22+
- Nicolai Hähnle, AMD
23+
- Alon Or-Bach, Samsung
24+
- Niklas Smedberg, Unity Technologies
25+
- Tobias Hector, AMD
26+
- Lionel Duc, NVIDIA
27+
- Lina Versace, Google
28+
- Sebastian Wick, Red Hat
29+
- Jakob Bornecrantz, Collabora
30+
- David Kvasnica, NVIDIA
31+
32+
=== Description
33+
34+
This device extension allows an application that uses the
35+
`apiext:VK_KHR_swapchain` extension to obtain information about the
36+
presentation engine's display, to obtain timing information about each
37+
present, and to schedule a present to happen no earlier than a desired time.
38+
An application can use this to minimize various visual anomalies (e.g.
39+
stuttering).
40+
41+
Traditional game and real-time animation applications need to correctly
42+
position their geometry for when the presentable image will be presented to
43+
the user.
44+
To accomplish this, applications need various timing information about the
45+
presentation engine's display.
46+
They need to know when presentable images were actually presented, and when
47+
they could have been presented.
48+
Applications also need to tell the presentation engine to display an image
49+
no sooner than a given time.
50+
This allows the application to avoid stuttering, so the animation looks
51+
smooth to the user.
52+
53+
include::{generated}/interfaces/VK_EXT_present_timing.adoc[]
54+
55+
=== Issues
56+
57+
1) How does the application determine refresh duration, quanta for change,
58+
whether FRR vs. VRR, etc.
59+
60+
*PROPOSED*: the query returns two values: 1) a refresh-cycle duration
61+
(pname:refreshDuration), and 2) an indication whether the timing is
62+
currently fixed (FRR) or variable (VRR).
63+
If pname:refreshDuration is zero, the platform cannot supply these
64+
values until after at least one flink:vkQueuePresentKHR has been done,
65+
from this time (e.g. if flink:vkQueuePresentKHR has been previously
66+
called for this swapchain, at least one additional call must be made).
67+
After calling flink:vkQueuePresentKHR, the query can be repeated until
68+
pname:refreshDuration is non-zero, at which point the FRR vs. VRR
69+
indication will also be valid.
70+
71+
If the presentation engine's pname:refreshDuration is a fixed value,
72+
the application's image present duration (IPD) should be a multiple of
73+
pname:refreshDuration.
74+
That is, the quanta for changing the IPD is pname:refreshDuration.
75+
For example, if pname:refreshDuration is 16.67ms, the IPD can be
76+
16.67ms, 33.33ms, 50.0ms, etc.
77+
78+
If the presentation engine's pname:refreshDuration is variable,
79+
pname:refreshDuration is the minimum value of the application's IPD, and
80+
the IPD can be larger by any quanta that is meaningful to the application.
81+
For example, if the pname:refreshDuration is 10ms (i.e. the maximum
82+
refresh rate is 100Hz), the application can choose an IPD of 11ms,
83+
13.33ms, 13.5ms, or 66.0ms; any value greater than or equal to 10ms is
84+
valid.
85+
There may be negative consequences for choosing an IPD that is too
86+
high, as the presentation engine may actually have a practical maximum
87+
pname:refreshDuration, where it needs to display the previous image
88+
again, and during this time the presentation engine might delay
89+
displaying a newly-presented image.
90+
91+
FRR displays on at least one platform (Wayland) are not necessarily
92+
fixed; but can change over time.
93+
For example, if a full-screen video player application is visible, the display
94+
may operate at a 24Hz refresh cycle; and then later switch to 60Hz when
95+
multiple windows are visible.
96+
97+
VRR displays on some platforms can also be seen as having different
98+
characteristics over time.
99+
For example, if an application's window is full-screen-exclusive (i.e. no other
100+
window or window system component is visible), the display can look like a VRR
101+
display (however that is defined).
102+
If the application's window is not full-screen-exclusive (e.g. a normal
103+
multi-window case), the display can look like an FRR display (i.e. because the
104+
compositor is trying to treat all windows in a consistent manner).
105+
A different issue will deal with how the timing characteristics can change
106+
over time.
107+
108+
109+
2) Do we return min/max values for refresh duration for VRR?
110+
111+
*PROPOSED*: return only the minimum value of refreshDuration for a VRR.
112+
113+
VRR displays have a minimum and maximum refresh rate, and therefore a minimum
114+
and maximum refreshDuration.
115+
It has been asserted that the display effectively does not have a minimum
116+
refresh rate.
117+
That is because if an application does not present soon enough, the display
118+
hardware will automatically re-display the previous image.
119+
However, when the display does that, an application cannot present a new image
120+
for a certain period of time.
121+
It is unclear about whether that period is large enough to cause visual
122+
artifacts.
123+
124+
125+
3) How to deal with changes in timing properties?
126+
127+
*RESOLVED*: The slink:VkPastPresentationTimingPropertiesEXT structure
128+
that is returned by flink:vkGetPastPresentationTimingEXT contains
129+
pname:timeDomainsCounter, which is incremented if the time
130+
domain enabled for the swapchain is not currently available.
131+
132+
An example of why display timing properties can change is if a surface
133+
changes from being a window that’s a subset of the display size, to
134+
becoming full-screen-exclusive.
135+
While the surface was a subset of the display, a compositor might
136+
enforce fixed timings on the surface (e.g. FRR of 60Hz), where the
137+
presentation engine might be free to allow VRR behavior of a
138+
full-screen-exclusive surface.
139+
140+
It is possible that a full-screen-exclusive window can become
141+
temporarily obscured (e.g. when a short-term dialog pops up).
142+
In this case, the surface might use FRR timings while the dialog is
143+
visible and VRR otherwise.
144+
145+
146+
4) One Query for all Timing info vs. an initial query to determine FRR vs. VRR,
147+
and then FRR-specific vs VRR-specific queries?
148+
149+
*RESOLVED*: Have one query, as described in issue 1, that can be
150+
called whenever the application needs to obtain the timing properties
151+
of the surface.
152+
153+
154+
5) Query to determine time domain?
155+
156+
*RESOLVED*: Have a query to determine the time domain.
157+
This extension defines a basic swapchain-local time domain.
158+
Other extensions can add other platform-specific time domains.
159+
160+
161+
6) What time to use for targetPresentTime for early images?
162+
163+
*RESOLVED*: Have no query for determining the current time in the PE’s time
164+
domain; and do allow the special value of zero for targetPresentTime,
165+
meaning that there is no target.
166+
167+
On some platforms, there is no way to determine the current time, nor
168+
to determine surface timing properties until after at least one image
169+
has been presented.
170+
171+
In such cases, the special value of zero allows the application to
172+
indicate that timing feedback is desired, but that no
173+
targetPresentTime is requested.
174+
Later, once the application has obtained feedback, it can specify
175+
targetPresentTime by using the result's actualPresentTime.
176+
177+
178+
7) How long before an application’s request for new image duration is honored?
179+
180+
*UNRESOLVED*: Apparently, changes to some vendors' display hardware settings do
181+
not take effect immediately.
182+
It is not clear what settings, and therefore, it is not clear how to
183+
address this issue.
184+
185+
186+
8) Do we have a query for the anticipated latency from present to feedback?
187+
188+
*RESOLVED*: Do not provide a query for the feedback latency.
189+
190+
There is some amount of latency from when an application calls
191+
vkQueuePresentKHR to when the image is displayed to the user, to when feedback
192+
is available to the application on when the image was actually displayed to the
193+
user.
194+
The first time (from the call till the image is presented) generally doesn’t
195+
matter, because the application will likely be providing a targetPresentTime
196+
(i.e. the application may have some indication for how long this will be).
197+
However, the latency between targetPresentTime until feedback is available may
198+
be much longer.
199+
For example, on Android on the 1st-generation Pixel phone (60Hz FRR display),
200+
the latency was approximately 5 refresh cycles (83.33ms).
201+
For higher-frequency displays, the latency may have a larger number of refresh
202+
cycles.
203+
204+
205+
9) Do we have a query(s) about the number of VkPastPresentationTimingEXT
206+
structs to keep?
207+
208+
*RESOLVED*: Do not provide a query for the number of results the swapchain
209+
is allowed to store before querying them with
210+
vkGetPastPresentationTimingEXT. Let the application specify that value with
211+
a dedicated API.
212+
213+
214+
10) How is the SWAPCHAIN_LOCAL and STAGE_LOCAL time domain used with the
215+
calibrated timestamps extension?
216+
217+
*RESOLVED*: Define a struct to chain into
218+
VkCalibratedTimestampInfoEXT::pNext that specifies a swapchain and present
219+
stage.
220+
221+
222+
11) Should VK_PRESENT_MODE_FIFO_LATEST_READY_EXT be part of this extension,
223+
or split out into its own extension?
224+
225+
*RESOLVED*: It is only tangentially related.
226+
Split it out into its own extension and define the interaction here.
227+
228+
=== Version History
229+
230+
* Revision 1, 2018-05-11 (Ian Elliott)
231+
** Internal revisions.
232+
233+
* Revision 2, 2022-11-30 (Lionel Duc)
234+
** Rebase for public discussions.
235+
236+
* Revision 3, 2024-10-09 (Lionel Duc)
237+
** Public revisions.

‎appendices/glossary.adoc‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -827,6 +827,11 @@ Framebuffer Region::
827827
A framebuffer region is a set of sample (x, y, layer, sample)
828828
coordinates that is a subset of the entire framebuffer.
829829

830+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
831+
Frame Rate::
832+
A non-Vulkan term for Image Present Rate (IPR).
833+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
834+
830835
Front-Facing::
831836
See Facingness.
832837
endif::VK_GRAPHICS_VERSION_1_0[]
@@ -924,6 +929,19 @@ Image::
924929
of device memory.
925930
Represented by a slink:VkImage object.
926931

932+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
933+
Image Present Duration::
934+
The amount of time the application intends for each
935+
newly-presented image to be visible to the user.
936+
This value should: be a multiple of the refresh cycle duration.
937+
938+
Image Present Rate::
939+
The number of newly-presented images the application intends to present
940+
each second (a.k.a. frame rate).
941+
On fixed refresh rate displays, this value should: be a multiple of
942+
the refresh rate.
943+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
944+
927945
Image Subresource::
928946
A specific mipmap level, layer, and set of aspects of an image.
929947

@@ -1142,6 +1160,11 @@ Invocation Repack Instruction::
11421160
implementation may: change the set of invocations that are executing.
11431161
endif::VK_KHR_ray_tracing_pipeline,VK_NV_ray_tracing[]
11441162

1163+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1164+
IPD::
1165+
Image Present Duration.
1166+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1167+
11451168
ifdef::VK_KHR_deferred_host_operations[]
11461169
Join (Deferred Host Operations)::
11471170
The act of instructing a thread to participate in the execution of a
@@ -1733,6 +1756,11 @@ ifdef::VK_KHR_ray_tracing_pipeline[flink:vkCmdTraceRaysKHR, and flink:vkCmdTrace
17331756
.
17341757
endif::VK_KHR_ray_tracing_pipeline,VK_NV_ray_tracing[]
17351758

1759+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1760+
RC::
1761+
Refresh Cycle.
1762+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1763+
17361764
ifdef::VK_KHR_video_queue[]
17371765
Reconstructed Picture::
17381766
A video picture resource reconstructed from a compressed bitstream using
@@ -1755,6 +1783,17 @@ Reference Picture Metadata::
17551783
Opaque state associated with a DPB slot, maintained by a video session.
17561784
endif::VK_KHR_video_queue[]
17571785

1786+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1787+
Refresh Cycle::
1788+
The periodic process for updating the contents of the Presentation Engine's display.
1789+
1790+
Refresh Cycle Duration::
1791+
The amount of time from the start of one refresh cycle to the next.
1792+
1793+
Refresh Rate::
1794+
The number of refresh cycles per second.
1795+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1796+
17581797
Release Operation (Resource)::
17591798
An operation that releases ownership of an image subresource or buffer
17601799
range.

0 commit comments

Comments
 (0)