| # Queue Present Wait Semaphore Management |
| |
| The following shorthand notations are used throughout this document: |
| |
| - PE: Presentation Engine |
| - ANI: vkAcquireNextImageKHR |
| - QS: vkQueueSubmit |
| - QP: vkQueuePresentKHR |
| - W: Wait |
| - S: Signal |
| - R: Render |
| - P: Present |
| - SN: Semaphore N |
| - IN: Swapchain image N |
| - FN: Fence N |
| |
| --- |
| |
| ## Introduction |
| |
| Vulkan requires the application (ANGLE in this case) to acquire swapchain images and queue them for |
| presentation, synchronizing GPU submissions with semaphores. A single frame looks like the |
| following: |
| |
| CPU: ANI ... QS ... QP |
| S:S1 W:S1 W:S2 |
| S:S2 |
| GPU: <------------ R -----------> |
| PE: <-------- P ------> |
| |
| That is, the GPU starts rendering after submission, and the presentation is started when rendering is |
| finished. Note that Vulkan tries to abstract a large variety of PE architectures, some of which do |
| not behave in a straight-forward manner. As such, ANGLE cannot know what the PE is exactly doing |
| with the images or when the images are visible on the screen. The only signal out of the PE is |
| received through the semaphore that's used in ANI. |
| |
| With multiple frames, the pipeline looks different based on present mode. Let's focus on |
| FIFO (the arguments in this document translate to all modes) with 3 images: |
| |
| CPU: QS QP QS QP QS QP QS QP |
| I1 I1 I2 I2 I3 I3 I1 I1 |
| GPU: <---- R I1 ----><---- R I2 ----><---- R I3 ----><---- R I1 ----> |
| PE: <----- P I1 -----><----- P I2 -----><----- P I3 -----><----- P I1 -----> |
| |
| First, an issue is evident here. The CPU is submitting jobs and queuing images for presentation |
| faster than the GPU can render them or the PE can view them. This can cause the length of the |
| submit queue to grow indefinitely, resulting in larger and larger input lag. In FIFO mode, the PE |
| present queue also grows indefinitely. |
| |
| To address this issue, ANGLE paces the CPU such that the length of the submit queue is kept at a |
| maximum of 1 image (i.e. submission with one image is being processed, and another one is in queue): |
| |
| CPU: QS QS W:F1 QS W:F2 QS |
| I1 I2 I3 I1 |
| S:F1 S:F2 S:F3 S:F4 |
| GPU: <---- R I1 ----><---- R I2 ----><---- R I3 ----><---- R I1 ----> |
| |
| > Note: Ideally, the length of the PE present queue should also be kept at a maximum of 1 (i.e. one |
| > image being presented, and another in queue). However, the Vulkan WSI extension doesn't provide |
| > enough control to achieve this. In heavy application, the length of the PE present queue is |
| > probably 1 anyway (as the rendering time is almost as long as the frame (i.e. present time), in |
| > which case pacing the submissions similarly paces the presentation). In theory, in FIFO mode, the |
| > length of the PE present queue is below n+2 where n is the number of swapchain images. |
| > |
| > To understand why, imagine a FIFO swapchain with 1000 images and submissions that are |
| > infinitesimally short. In this case, the CPU pacing is effectively a no-op (as the GPU instantly |
| > finishes jobs) for the first 1002 submissions. The 1003rd submission waits for F1001 (which uses |
| > I1). However, the 1001st submission will not start until the PE switches to presenting I2 (at the |
| > next V-Sync). The CPU then waits for V-Sync before the 1003rd submission. The CPU waits for one |
| > V-Sync for every subsequent submission, keeping the length of the queue 1002. |
| > [`VK_GOOGLE_display_timing`][DisplayTimingGOOGLE] is likely a solution to this problem. |
| |
| Associated with each QP operation is a semaphore signaled by the preceding QS and waited on by the |
| PE before the image can be presented. Currently, there's no feedback from Vulkan (See [internal |
| Khronos issue][VulkanIssue1060]) regarding _when_ the PE has actually finished waiting on the |
| semaphore! This means that the application cannot generally know when to destroy the corresponding |
| semaphore. However, taking ANGLE's CPU pacing into account, we are able to destroy (or rather |
| reuse) semaphores when they are provably unused. |
| |
| This document describes an approach for destroying semaphores that should work with all valid PE |
| architectures, but will be described in terms of more common PE architectures (e.g. where the PE |
| only backs each VkImage and VkSemaphore handle with one actual memory object, and where the PE |
| cycles between the swapchain images in a straight-forward manner). |
| |
| The interested reader may follow the discussion in this abandoned [gerrit CL][CL1757018] for more |
| background and ideas. |
| |
| [DisplayTimingGOOGLE]: https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VK_GOOGLE_display_timing.html |
| [VulkanIssue1060]: https://gitlab.khronos.org/vulkan/vulkan/issues/1060 |
| [CL1757018]: https://chromium-review.googlesource.com/c/angle/angle/+/1757018 |
| |
| ## Determining When a QP Semaphore is Waited On |
| |
| Let's combine the above diagrams with all the details: |
| |
| CPU: ANI | QS | QP | ANI | QS | QP | ANI | W:F1 | QS | QP | ANI | W:F2 | QS | QP |
| I1 | I1 | I1 | I2 | I2 | I2 | I3 | | I3 | I3 | I1 | | I1 | I1 |
| S:SA1 | W:SA1 | | S:SA2 | W:SA2 | | S:SA3 | | W:SA3 | | S:SA4 | | W:SA4 | |
| | S:SP1 | W:SP1 | | S:SP2 | W:SP2 | | | S:SP3 | W:SP3 | | | S:SP4 | W:SP4 |
| | S:F1 | | | S:F2 | | | | S:F3 | | | | S:F4 | |
| |
| Let's focus only on sequences that return the same image: |
| |
| CPU: ANI | W:F(X-2) | QS | QP | ... | ANI | W:F(Y-2) | QS | QP |
| I1 | | I1 | I1 | | I1 | | I1 | I1 |
| S:SAX | | W:SAX | | | S:SAY | | W:SAY | |
| | | S:SPX | W:SPX | | | | S:SPY | W:SPY |
| | | S:FX | | | | | S:FY | |
| |
| Note that X and Y are arbitrarily distanced (including possibly being sequential). |
| |
| Say we are at frame Y+2. There's therefore a wait on FY. The following holds: |
| |
| FY is signaled |
| => SAY is signaled |
| => The PE has handed I1 back to the application |
| => The PE has already processed the *previous* QP of I1 |
| => SPX is waited on |
| |
| At this point, we can destroy SPX. In other words, in frame Y+2, we can destroy SPX (note that 2 is |
| the number of frames the CPU pacing code uses). If frame Y+1 is not using I1, this means the |
| history of present semaphores for I1 would be `{SPX, SPY}` and we can destroy the oldest semaphore |
| in this list. If frame Y+1 is also using I1, we should still destroy SPX in frame Y+2, but the |
| history of the present semaphores for I1 would be `{SPX, SPY, SP(Y+1)}`. |
| |
| In the Vulkan backend, we simplify destruction of semaphores by always keeping a history of 3 |
| present semaphores for each image (again, 3 is H+1 where H is the swap history size used in CPU |
| pacing) and always reuse (instead of destroy) the oldest semaphore of the image that is about to be |
| presented. |
| |
| To summarize, we use the completion of a submission using an image to prove when the semaphore used |
| for the *previous* presentation of that image is no longer in use (and can be safely destroyed or |
| reused). |
| |
| ## Swapchain recreation |
| |
| When recreating the swapchain, all images are eventually freed and new ones are created, possibly |
| with a different count and present mode. For the old swapchain, we can no longer rely on the |
| completion of a future submission to know when a previous presentation's semaphore can be destroyed, |
| as there won't be any more submissions using images from the old swapchain. |
| |
| > For example, imagine the old swapchain was created in FIFO mode, and one image is being presented |
| > until the next V-Sync. Furthermore, imagine the new swapchain is created in MAILBOX mode. Since |
| > the old swapchain's image will remain presented until V-Sync, the new MAILBOX swapchain can |
| > perform an arbitrarily large number of (throw-away) presentations. The old swapchain (and its |
| > associated present semaphores) cannot be destroyed until V-Sync; a signal that's not captured by |
| > Vulkan. |
| |
| ANGLE resolves this issue by deferring the destruction of the old swapchain and its remaining |
| present semaphores to the time when the semaphore corresponding to the first present of the new |
| swapchain can be destroyed. In the example in the previous section, if SPX is the present semaphore |
| of the first QP performed on the new swapchain, at frame Y+2, when we know SPX can be destroyed, we |
| know that the first image of the new swapchain has already been presented. This proves that all |
| previous QPs of the old swapchain have been processed. |
| |
| > Note: the swapchain can potentially be destroyed much earlier, but with no feedback from the |
| > presentation engine, we cannot know that. This delays means that the swapchain could be recreated |
| > while there are pending old swapchains to be destroyed. The destruction of both old swapchains |
| > must now be deferred to when the first QP of the new swapchain has been processed. If an |
| > application resizes the window constantly and at a high rate, ANGLE would keep accumulating old |
| > swapchains and not free them until it stops. While a user will likely not be able to do this (as |
| > the rate of window system events is lower than the framerate), this can be programmatically done |
| > (as indeed done in EGL dEQP tests). Nvidia for example fails creation of a new swapchain if there |
| > are already 20 allocated (on desktop, or less than ten on Quadro). If the backlog of old |
| > swapchains get larger than a threshold, ANGLE calls `vkQueueWaitIdle()` and destroys the |
| > swapchains. |
