Skip to content

Commit fb7f033

Browse files
Merge pull request #749 from Devsh-Graphics-Programming/intended_submit_info_changes
SIntendedSubmitInfo Changes
2 parents c1d5f34 + 2e49c04 commit fb7f033

File tree

1 file changed

+69
-32
lines changed

1 file changed

+69
-32
lines changed

include/nbl/video/utilities/SIntendedSubmitInfo.h

Lines changed: 69 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,10 @@ namespace nbl::video
1111
//! Struct meant to be used with any Utility (not just `IUtilities`) which exhibits "submit on overflow" behaviour.
1212
//! Such functions are non-blocking (unless overflow) and take `SIntendedSubmitInfo` by reference and patch it accordingly.
1313
//! for example: in the case the `waitSemaphores` were already waited upon, the struct will be modified to have it's `waitSemaphores` emptied.
14-
//! MAKE SURE to do a submit to `queue` by yourself with the result of `popSubmit(...)` implicitly converted to `std::span<const IQueue::SSubmitInfo>` !
14+
//! User is supposed to begin EXACTLY ONE of the `scratchCommandBuffer` before first ever submit.
15+
//! Never `begin` or `end` scratch commandBuffers ever again.
16+
//! Always assume the current recording commandBuffer is in RECORDING state (except between `submit` and `beginNextCommandBuffer` call intended for ADVANCED users)
17+
//! The "current recording commandBuffer" is returned by `getCurrentRecordingCommandBufferInfo` or updated via the pointer ref in beginNextCommandBuffer or overflowSubmit)
1518
struct SIntendedSubmitInfo final : core::Uncopyable
1619
{
1720
// All commandbuffers must be compatible with the queue we're about to submit to
@@ -72,36 +75,45 @@ struct SIntendedSubmitInfo final : core::Uncopyable
7275
}
7376

7477
public:
75-
// This parameter is required but may be unused if there is no need (no overflow) to do submit
78+
// This parameter is required but may be unused if there is no need (no overflow) to perform submit operations (no call to `overflowSubmit` or `submit`)
7679
IQueue* queue = nullptr;
77-
// Use this parameter to wait for previous operations to finish before whatever commands the Utility you're using records
80+
81+
// Use this parameter to wait for previous operations to finish before whatever commands the Utility you're using records.
7882
std::span<const IQueue::SSubmitInfo::SSemaphoreInfo> waitSemaphores = {};
79-
// Fill the commandbuffers you want to run before the first command the Utility records to run in the same submit,
80-
// for example baked command buffers with pipeline barrier commands.
83+
84+
// Fill the commandbuffers you want to run before the first command the Utility records to run in the same submit,
85+
// for example, baked command buffers with pipeline barrier commands.
8186
std::span<const IQueue::SSubmitInfo::SCommandBufferInfo> prevCommandBuffers = {};
82-
// A set of command buffers the Utility can round robin its transient commands. All must be individually resettable.
83-
// Command buffers are cycled through for use and submission in a simple modular arithmetic fashion.
84-
// EXACTLY ONE commandbuffer must be in recording state! This is the one the utility will use immediately to record commands.
85-
// But remember that even though its scratch, you can record some of your own preceeding commands into it as well.
87+
88+
// A set of command buffers the Utility can use in a round-robin manner for its transient commands. All command buffers must be individually resettable.
89+
// Command buffers are cycled through for use and submission using a simple modular arithmetic fashion.
90+
// EXACTLY ONE commandbuffer must be in recording state! This is the one the utilities will use immediately to record commands.
91+
// However, even though it's scratch, you can record some of your own preceding commands into it as well.
8692
std::span<IQueue::SSubmitInfo::SCommandBufferInfo> scratchCommandBuffers = {};
87-
// This parameter is required but may be unused if there is no need (no overflow) to do submit.
88-
// This semaphore is needed to "stitch together" additional submits if they occur so they occur before and after the original intended waits and signals.
89-
// The initial `scratchSemaphore.value` gets incremented and signalled on each submit, can start at 0 because an overflow will signal `value+1`.
90-
// NOTE: You should signal this semaphore as well when doing your last/tail submit! Why?
91-
// Utilities might use the `[Multi]TimelineEventHandler` to latch cleanups on `scratchSemaphore` where this value
92-
// is a value that they expect to actually get signalled in the future.
93-
// NOTE: Do not choose the values for waits and signals yourself, as the overflows may increment the counter by an arbitrary amount!
94-
// You can actually examine the change in `scratchSemaphore.value` to figure out how many overflows occurred.
93+
94+
// This semaphore is needed to indicate when each sub-submit is complete and resources can be reclaimed.
95+
// It also ensures safe usage of the scratch command buffers by blocking the CPU if the next scratch command buffer to use is in a PENDING state and not safe to begin.
96+
// The initial `scratchSemaphore.value` gets incremented and signaled on each submit. It can start at 0 because an overflow will signal `value+1`.
97+
// NOTE: You should signal this semaphore when doing your last/tail submit manually OR when not submitting the recorded scratch at all
98+
// (in which case, signal from Host or another submit). Why? The utilities that deal with `SIntendedSubmitInfo` might use the
99+
// `[Multi]TimelineEventHandler` to latch cleanups/deallocations on `scratchSemaphore`, and `getFutureScratchSemaphore()`
100+
// is the value they expect to get signaled in the future.
101+
// NOTE: Each overflow submit bumps the value to wait and signal by +1. A utility may overflow an arbitrary number of times.
102+
// Therefore, DO NOT choose the values for waits and signals manually, and do not modify the `scratchSemaphore` field after initialization or first use.
103+
// You can examine the change in `scratchSemaphore.value` to observe how many overflows or submits have occurred.
95104
IQueue::SSubmitInfo::SSemaphoreInfo scratchSemaphore = {};
96-
// Optional: If you had a semaphore whose highest pending signal is 45 but gave the scratch a value of 68 (causing 69 to get signalled in a popped submit),
97-
// but only used 4 scratch command buffers, we'd wait for the semaphore to reach 66 before resetting the next scratch command buffer.
98-
// That is obviously suboptimal if the next scratch command buffer wasn't pending with a signal of 67 at all (you'd wait until 70 gets signalled).
99-
// Therefore you would need to override this behaviour somehow and be able to tell to only wait for the semaphore at values higher than 68.
105+
106+
// Optional: If you had a semaphore whose highest pending signal is 45 but gave the scratch a value of 68 (causing 69 to get signaled in a popped submit),
107+
// but only used 4 scratch command buffers, we'd wait for the semaphore to reach 66 before resetting the next scratch command buffer.
108+
// This is obviously suboptimal if the next scratch command buffer wasn't pending with a signal of 67 at all (you'd be waiting until 70 gets signaled).
109+
// Therefore, you would need to override this behavior to only wait for semaphore values higher than 68.
100110
size_t initialScratchValue = 0;
101-
// Optional: Callback to perform some other CPU work while blocking for one of the submitted scratch command buffers to complete execution.
102-
// Can get called repeatedly! The argument is the scratch semaphore (so it can poll itself to know when to finish work - prevent priority inversion)
111+
112+
// Optional: Callback to perform some other CPU work while blocking for one of the submitted scratch command buffers to complete execution.
113+
// This callback may be called repeatedly! The argument provided is the scratch semaphore, allowing it to poll itself to determine when it can finish its work,
114+
// preventing priority inversion.
103115
std::function<void(const ISemaphore::SWaitInfo&)> overflowCallback = {};
104-
116+
105117
//
106118
inline ISemaphore::SWaitInfo getFutureScratchSemaphore() const {return {scratchSemaphore.semaphore,scratchSemaphore.value+1};}
107119

@@ -175,22 +187,47 @@ struct SIntendedSubmitInfo final : core::Uncopyable
175187
return retval;
176188
}
177189

178-
// recordingCmdBuf pointer may change so some other command buffer info in `scratchCommandBuffers` when applicable
190+
// ! [inout] recordingCmdBuf: current recording SCommandBufferInfo pointing to an element of `scratchCommandBuffers`.
191+
// To achieve multiple submits in flight, the pointer may change and point to another element of `scratchCommandBuffers` when applicable.
179192
inline IQueue::RESULT overflowSubmit(const IQueue::SSubmitInfo::SCommandBufferInfo* &recordingCmdBuf)
193+
{
194+
IQueue::RESULT res = submit(recordingCmdBuf, {});
195+
if (res != IQueue::RESULT::SUCCESS)
196+
return res;
197+
if (!beginNextCommandBuffer(recordingCmdBuf))
198+
return IQueue::RESULT::OTHER_ERROR;
199+
return IQueue::RESULT::SUCCESS;
200+
}
201+
202+
// Submits via the current recording command buffer
203+
// ! recordingCmdBuf: current recording SCommandBufferInfo pointing to an element of `scratchCommandBuffers`.
204+
// ! Optional: signalSemaphores: your signal semaphores that indicate the job is finished.
205+
// ! Don't attempt to use the `recordingCmdBuf` after calling this function and before calling `beginNextCommandBuffer` because it will be in PENDING state
206+
inline IQueue::RESULT submit(const IQueue::SSubmitInfo::SCommandBufferInfo* recordingCmdBuf, const std::span<const IQueue::SSubmitInfo::SSemaphoreInfo> signalSemaphores)
180207
{
181208
const auto pLast = &scratchCommandBuffers.back();
182209
if (recordingCmdBuf<scratchCommandBuffers.data() || recordingCmdBuf>pLast)
183210
return IQueue::RESULT::OTHER_ERROR;
211+
184212
// First, submit the already buffered up work
185213
recordingCmdBuf->cmdbuf->end();
186214

187215
// we only signal the scratch semaphore when overflowing
188-
const auto submit = popSubmit(recordingCmdBuf->cmdbuf,{});
216+
const auto submit = popSubmit(recordingCmdBuf->cmdbuf,signalSemaphores);
189217
IQueue::RESULT res = queue->submit(submit);
190-
if (res!=IQueue::RESULT::SUCCESS)
191-
return res;
218+
return res;
219+
}
192220

193-
// Get the next scratch buffer
221+
// If next command buffer is not PENDING and safe to use, it will reset and begin it, otherwise it will block on the scratch semaphore
222+
// ! recordingCmdBuf: previous recording SCommandBufferInfo pointing to an element of `scratchCommandBuffers`
223+
// To achieve multiple submits in flight, the pointer may change and point to another element of `scratchCommandBuffers` when applicable.
224+
inline bool beginNextCommandBuffer(const IQueue::SSubmitInfo::SCommandBufferInfo* &recordingCmdBuf)
225+
{
226+
const auto pLast = &scratchCommandBuffers.back();
227+
if (recordingCmdBuf<scratchCommandBuffers.data() || recordingCmdBuf>pLast)
228+
return false;
229+
230+
// Get the next command buffer to record
194231
if (recordingCmdBuf!=pLast)
195232
recordingCmdBuf++;
196233
else
@@ -209,17 +246,17 @@ struct SIntendedSubmitInfo final : core::Uncopyable
209246
if (waitResult!=ISemaphore::WAIT_RESULT::SUCCESS)
210247
{
211248
assert(false);
212-
return IQueue::RESULT::OTHER_ERROR;
249+
return false;
213250
}
214251
}
215252
// could have just called begin to reset but also need to reset with the release resources flag
216253
recordingCmdBuf->cmdbuf->reset(IGPUCommandBuffer::RESET_FLAGS::RELEASE_RESOURCES_BIT);
217254
recordingCmdBuf->cmdbuf->begin(IGPUCommandBuffer::USAGE::ONE_TIME_SUBMIT_BIT);
218255

219-
return res;
256+
return true;
220257
}
221258

222-
// useful overload if you forgot what was scratch
259+
// useful overload if you forgot what was command buffer to record.
223260
inline IQueue::RESULT overflowSubmit()
224261
{
225262
auto recordingCmdBuf = valid();

0 commit comments

Comments
 (0)