Option for offscreen frame boundary conversion

Add `--use-ext-frame-boundary` option to convert all offscreen
frame boundaries to `VK_EXT_frame_boundary` frame boundaries.

Extensions used to specify an offscreen frame boundary can be
unsupported by the replay device. As there are many of such
extensions, it seems important to be able to "convert" these
frame boundaries to a more "standard" one.
The `VK_EXT_frame_boundary` seems to be the best choice as it
is specialy designed for this purpose, platform independent,
and supports all functionnalities supported by other extensions:
- Specifying resources attached to the frame (images and buffers)
- Depending on the execution of other commands
- Specifying frame number, label and description

Change-Id: I9fb932b219380c6f9098a16a83de2086f2b424ee

Author: marius-pelegrin-arm

USAGE_android.md

@@ -876,6 +876,10 @@ optional arguments:
                         in the original capture. This allows preserving frames
                         when capturing a replay that uses. offscreen swapchain.
                         (forwarded to replay tool)
+  --use-ext-frame-boundary
+                        Convert all offscreen frame boundaries to
+                        `VK_EXT_frame_boundary` frame boundaries.
+                        (forwarded to replay tool)
 ```
 
 The command will force-stop an active replay process before starting the replay

USAGE_desktop_Vulkan.md

@@ -549,6 +549,9 @@ Optional arguments:
               was called in the original capture.
               This allows preserving frames when capturing a replay that uses.
               offscreen swapchain.
+  --use-ext-frame-boundary
+              Convert all offscreen frame boundaries to `VK_EXT_frame_boundary`
+              frame boundaries.
 ```
 
 ### Key Controls

android/scripts/gfxrecon.py

@@ -86,6 +86,7 @@ def CreateReplayParser():
     parser.add_argument('--onhb', '--omit-null-hardware-buffers', action='store_true', default=False, help='Omit Vulkan calls that would pass a NULL AHardwareBuffer* (forwarded to replay tool)')
     parser.add_argument('--use-colorspace-fallback', action='store_true', default=False, help='Swap the swapchain color space if unsupported by replay device. Check if color space is not supported by replay device and swap to VK_COLOR_SPACE_SRGB_NONLINEAR_KHR. (forwarded to replay tool).')
     parser.add_argument('--offscreen-swapchain-frame-boundary', action='store_true', default=False, help='Should only be used with offscreen swapchain. Activates the extension VK_EXT_frame_boundary (always supported if trimming, checks for driver support otherwise) and inserts command buffer submission with VkFrameBoundaryEXT where vkQueuePresentKHR was called in the original capture. This allows preserving frames when capturing a replay that uses. offscreen swapchain. (forwarded to replay tool)')
+    parser.add_argument('--use-ext-frame-boundary', action='store_true', default=False, help='Convert all offscreen frame boundaries to `VK_EXT_frame_boundary` frame boundaries. (forwarded to replay tool)')
     parser.add_argument('--mfr', '--measurement-frame-range', metavar='START-END', help='Custom framerange to measure FPS for. This range will include the start frame but not the end frame. The measurement frame range defaults to all frames except the loading frame but can be configured for any range. If the end frame is past the last frame in the trace it will be clamped to the frame after the last (so in that case the results would include the last frame). (forwarded to replay tool)')
     parser.add_argument('--measurement-file', metavar='DEVICE_FILE', help='Write measurements to a file at the specified path. Default is: \'/sdcard/gfxrecon-measurements.json\' on android and \'./gfxrecon-measurements.json\' on desktop. (forwarded to replay tool)')
     parser.add_argument('--quit-after-measurement-range', action='store_true', default=False, help='If this is specified the replayer will abort when it reaches the <end_frame> specified in the --measurement-frame-range argument. (forwarded to replay tool)')
@@ -198,6 +199,9 @@ def MakeExtrasString(args):
 
     if args.offscreen_swapchain_frame_boundary:
         arg_list.append('--offscreen-swapchain-frame-boundary')
+    
+    if args.use_ext_frame_boundary:
+        arg_list.append('--use-ext-frame-boundary')
 
     if args.memory_translation:
         arg_list.append('-m')

framework/application/application.h

@@ -84,6 +84,8 @@ class Application final
 
     void StopRunning() { running_ = false; }
 
+    uint32_t GetCurrentFrameNumber() const { return file_processor_->GetCurrentFrameNumber(); }
+
   private:
     // clang-format off
     std::string                                                  name_;              ///< Application name to display in window title bar.

framework/decode/vulkan_object_info.h

@@ -460,6 +460,7 @@ struct ShaderEXTInfo : VulkanObjectInfo<VkShaderEXT>
 struct CommandBufferInfo : public VulkanPoolObjectInfo<VkCommandBuffer>
 {
     bool                                                is_frame_boundary{ false };
+    std::string                                         frame_boundary_label;
     std::vector<format::HandleId>                       frame_buffer_ids;
     std::unordered_map<format::HandleId, VkImageLayout> image_layout_barriers;
 };

framework/decode/vulkan_replay_consumer_base.cpp

@@ -2143,6 +2143,59 @@ void VulkanReplayConsumerBase::WriteScreenshots(const Decoded_VkPresentInfoKHR*
     }
 }
 
+void VulkanReplayConsumerBase::FillFrameBoundaryExtFromCommandBufferInfo(const CommandBufferInfo* command_buffer_info,
+                                                                         VkFrameBoundaryEXT*      frame_boundary,
+                                                                         std::vector<VkImage>&    frame_boundary_images)
+{
+    assert(command_buffer_info->is_frame_boundary);
+
+    frame_boundary_images.clear();
+
+    for (size_t i = 0; i < command_buffer_info->frame_buffer_ids.size(); ++i)
+    {
+        auto framebuffer_info = object_info_table_.GetFramebufferInfo(command_buffer_info->frame_buffer_ids[i]);
+
+        for (size_t j = 0; j < framebuffer_info->attachment_image_view_ids.size(); ++j)
+        {
+            auto image_view_id   = framebuffer_info->attachment_image_view_ids[j];
+            auto image_view_info = object_info_table_.GetImageViewInfo(image_view_id);
+            auto image_info      = object_info_table_.GetImageInfo(image_view_info->image_id);
+
+            frame_boundary_images.push_back(image_info->handle);
+        }
+    }
+
+    frame_boundary->sType       = VK_STRUCTURE_TYPE_FRAME_BOUNDARY_EXT;
+    frame_boundary->pNext       = nullptr;
+    frame_boundary->flags       = VK_FRAME_BOUNDARY_FRAME_END_BIT_EXT;
+    frame_boundary->frameID     = application_->GetCurrentFrameNumber();
+    frame_boundary->imageCount  = frame_boundary_images.size();
+    frame_boundary->pImages     = frame_boundary_images.data();
+    frame_boundary->bufferCount = 0;
+    frame_boundary->pBuffers    = nullptr;
+    frame_boundary->tagName     = application_->GetCurrentFrameNumber();
+    frame_boundary->tagSize     = command_buffer_info->frame_boundary_label.size();
+    frame_boundary->pTag        = command_buffer_info->frame_boundary_label.data();
+}
+
+void VulkanReplayConsumerBase::InsertFrameBoundaryExt(void* pnext_chain, const VkFrameBoundaryEXT* frame_boundary)
+{
+    VkBaseOutStructure* current = reinterpret_cast<VkBaseOutStructure*>(pnext_chain);
+    while (current->pNext != nullptr)
+    {
+        current = current->pNext;
+
+        if (current->sType == VK_STRUCTURE_TYPE_FRAME_BOUNDARY_EXT)
+        {
+            GFXRECON_LOG_WARNING(
+                "Trying to insert VkFrameBoundaryEXT but there already is one. The new one will be ignored.");
+            return;
+        }
+    }
+
+    current->pNext = reinterpret_cast<VkBaseOutStructure*>(&frame_boundary);
+}
+
 bool VulkanReplayConsumerBase::CheckCommandBufferInfoForFrameBoundary(const CommandBufferInfo* command_buffer_info)
 {
     GFXRECON_ASSERT(command_buffer_info != nullptr);
@@ -2503,6 +2556,24 @@ VulkanReplayConsumerBase::OverrideCreateInstance(VkResult original_result,
         }
     }
 
+    if (options_.use_ext_frame_boundary)
+    {
+        bool frame_boundary_extension_found = false;
+        for (const char* extension_name : filtered_extensions)
+        {
+            if (gfxrecon::util::platform::StringCompareNoCase(extension_name, VK_EXT_FRAME_BOUNDARY_EXTENSION_NAME))
+            {
+                frame_boundary_extension_found = true;
+                break;
+            }
+        }
+
+        if (!frame_boundary_extension_found)
+        {
+            filtered_extensions.push_back(VK_EXT_FRAME_BOUNDARY_EXTENSION_NAME);
+        }
+    }
+
     // Disable layers; any layers needed for replay should be enabled for the replay app with the VK_INSTANCE_LAYERS
     // environment variable or debug.vulkan.layers Android property.
     if (modified_create_info.enabledLayerCount > 0)
@@ -2788,11 +2859,19 @@ void VulkanReplayConsumerBase::OverrideDestroyDevice(
 
     if (device_info != nullptr)
     {
-        device = device_info->handle;
+        device            = device_info->handle;
+        auto device_table = GetDeviceTable(device);
+
+        auto it = fba_resources_.find(device);
+        if (it != fba_resources_.end())
+        {
+            device_table->DestroyCommandPool(device, it->second.first, nullptr);
+            fba_resources_.erase(device);
+        }
 
         if (screenshot_handler_ != nullptr)
         {
-            screenshot_handler_->DestroyDeviceResources(device, GetDeviceTable(device));
+            screenshot_handler_->DestroyDeviceResources(device, device_table);
         }
 
         device_info->allocator->Destroy();
@@ -3327,6 +3406,31 @@ VkResult VulkanReplayConsumerBase::OverrideQueueSubmit(PFN_vkQueueSubmit func,
         fence = fence_info->handle;
     }
 
+    std::vector<VkFrameBoundaryEXT>   inserted_frame_boundaries;
+    std::vector<std::vector<VkImage>> inserted_frame_boundaries_images;
+    if (options_.use_ext_frame_boundary)
+    {
+        for (uint32_t i = 0; i < submitCount; ++i)
+        {
+            size_t                  command_buffer_count = submit_info_data[i].pCommandBuffers.GetLength();
+            const format::HandleId* command_buffer_ids   = submit_info_data[i].pCommandBuffers.GetPointer();
+            for (uint32_t j = 0; j < command_buffer_count; ++j)
+            {
+                const CommandBufferInfo* command_buffer_info =
+                    GetObjectInfoTable().GetCommandBufferInfo(command_buffer_ids[j]);
+
+                if (command_buffer_info->is_frame_boundary)
+                {
+                    FillFrameBoundaryExtFromCommandBufferInfo(command_buffer_info,
+                                                              &inserted_frame_boundaries.emplace_back(),
+                                                              inserted_frame_boundaries_images.emplace_back());
+                    InsertFrameBoundaryExt(submit_info_data[i].decoded_value, &inserted_frame_boundaries.back());
+                    break;
+                }
+            }
+        }
+    }
+
     // Only attempt to filter imported semaphores if we know at least one has been imported.
     // If rendering is restricted to a specific surface, shadow semaphore and forward progress state will need to be
     // tracked.
@@ -3494,6 +3598,32 @@ VkResult VulkanReplayConsumerBase::OverrideQueueSubmit2(PFN_vkQueueSubmit2 func,
         fence = fence_info->handle;
     }
 
+    std::vector<VkFrameBoundaryEXT>   inserted_frame_boundaries;
+    std::vector<std::vector<VkImage>> inserted_frame_boundaries_images;
+    if (options_.use_ext_frame_boundary)
+    {
+        for (uint32_t i = 0; i < submitCount; ++i)
+        {
+            size_t     command_buffer_count = submit_info_data[i].pCommandBufferInfos->GetLength();
+            const auto command_buffer_infos = submit_info_data[i].pCommandBufferInfos->GetMetaStructPointer();
+
+            for (uint32_t j = 0; j < command_buffer_count; ++j)
+            {
+                const CommandBufferInfo* command_buffer_info =
+                    GetObjectInfoTable().GetCommandBufferInfo(command_buffer_infos[j].commandBuffer);
+
+                if (command_buffer_info->is_frame_boundary)
+                {
+                    FillFrameBoundaryExtFromCommandBufferInfo(command_buffer_info,
+                                                              &inserted_frame_boundaries.emplace_back(),
+                                                              inserted_frame_boundaries_images.emplace_back());
+                    InsertFrameBoundaryExt(submit_info_data[i].decoded_value, &inserted_frame_boundaries.back());
+                    break;
+                }
+            }
+        }
+    }
+
     // Only attempt to filter imported semaphores if we know at least one has been imported.
     // If rendering is restricted to a specific surface, shadow semaphore and forward progress state will need to be
     // tracked.
@@ -5152,6 +5282,33 @@ VkResult VulkanReplayConsumerBase::OverrideCreateDebugUtilsMessengerEXT(
                 pMessenger->GetHandlePointer());
 }
 
+void VulkanReplayConsumerBase::OverrideCmdInsertDebugUtilsLabelEXT(
+    PFN_vkCmdInsertDebugUtilsLabelEXT                   func,
+    CommandBufferInfo*                                  command_buffer_info,
+    StructPointerDecoder<Decoded_VkDebugUtilsLabelEXT>* label_info_decoder)
+{
+    const VkDebugUtilsLabelEXT* label_info = label_info_decoder->GetPointer();
+
+    bool call_next_layer = true;
+
+    // Look for the label that identifies this command buffer as a VR frame boundary.
+    if (util::platform::StringContains(label_info->pLabelName, graphics::kVulkanVrFrameDelimiterString))
+    {
+        command_buffer_info->is_frame_boundary    = true;
+        command_buffer_info->frame_boundary_label = label_info->pLabelName;
+
+        if (options_.use_ext_frame_boundary)
+        {
+            call_next_layer = false;
+        }
+    }
+
+    if (call_next_layer)
+    {
+        func(command_buffer_info->handle, label_info);
+    }
+}
+
 VkResult VulkanReplayConsumerBase::OverrideCreateSwapchainKHR(
     PFN_vkCreateSwapchainKHR                                      func,
     VkResult                                                      original_result,
@@ -6977,12 +7134,24 @@ void VulkanReplayConsumerBase::OverrideCmdDebugMarkerInsertEXT(
     StructPointerDecoder<Decoded_VkDebugMarkerMarkerInfoEXT>* marker_info_decoder)
 {
     const VkDebugMarkerMarkerInfoEXT* marker_info = marker_info_decoder->GetPointer();
-    func(command_buffer_info->handle, marker_info);
+
+    bool call_next_layer = true;
 
     // Look for the debug marker that identifies this command buffer as a VR frame boundary.
     if (util::platform::StringContains(marker_info->pMarkerName, graphics::kVulkanVrFrameDelimiterString))
     {
-        command_buffer_info->is_frame_boundary = true;
+        command_buffer_info->is_frame_boundary    = true;
+        command_buffer_info->frame_boundary_label = marker_info->pMarkerName;
+
+        if (options_.use_ext_frame_boundary)
+        {
+            call_next_layer = false;
+        }
+    }
+
+    if (call_next_layer)
+    {
+        func(command_buffer_info->handle, marker_info);
     }
 };
 
@@ -7074,6 +7243,103 @@ VkResult VulkanReplayConsumerBase::OverrideCreateFramebuffer(
     return result;
 }
 
+void VulkanReplayConsumerBase::OverrideFrameBoundaryANDROID(PFN_vkFrameBoundaryANDROID func,
+                                                            const DeviceInfo*          device_info,
+                                                            const SemaphoreInfo*       semaphore_info,
+                                                            const ImageInfo*           image_info)
+{
+    GFXRECON_ASSERT((device_info != nullptr));
+
+    VkDevice    device    = device_info->handle;
+    VkSemaphore semaphore = semaphore_info ? semaphore_info->handle : VK_NULL_HANDLE;
+    VkImage     image     = image_info ? image_info->handle : VK_NULL_HANDLE;
+
+    if (options_.use_ext_frame_boundary)
+    {
+        auto device_table = GetDeviceTable(device);
+
+        // Retrieve adequate queue family
+
+        uint32_t queueFamily = 0;
+
+        // Create command pool and command buffer if necessary
+
+        auto it = fba_resources_.find(device);
+        if (it == fba_resources_.end())
+        {
+            VkCommandPoolCreateInfo commandPoolCreateInfo;
+            commandPoolCreateInfo.sType            = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO;
+            commandPoolCreateInfo.pNext            = nullptr;
+            commandPoolCreateInfo.flags            = 0;
+            commandPoolCreateInfo.queueFamilyIndex = queueFamily;
+
+            VkCommandPool commandPool;
+            device_table->CreateCommandPool(device, &commandPoolCreateInfo, nullptr, &commandPool);
+
+            VkCommandBufferAllocateInfo commandBufferAllocateInfo;
+            commandBufferAllocateInfo.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
+            commandBufferAllocateInfo.pNext              = nullptr;
+            commandBufferAllocateInfo.commandPool        = commandPool;
+            commandBufferAllocateInfo.level              = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
+            commandBufferAllocateInfo.commandBufferCount = 1;
+
+            VkCommandBuffer commandBuffer;
+            device_table->AllocateCommandBuffers(device, &commandBufferAllocateInfo, &commandBuffer);
+
+            VkCommandBufferBeginInfo commandBufferBeginInfo;
+            commandBufferBeginInfo.sType            = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
+            commandBufferBeginInfo.pNext            = nullptr;
+            commandBufferBeginInfo.flags            = VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT;
+            commandBufferBeginInfo.pInheritanceInfo = nullptr;
+
+            device_table->BeginCommandBuffer(commandBuffer, &commandBufferBeginInfo);
+            device_table->EndCommandBuffer(commandBuffer);
+
+            it = fba_resources_.emplace(device, std::make_pair(commandPool, commandBuffer)).first;
+        }
+
+        // Queue submission with VkFrameBoundaryEXT
+
+        VkPipelineStageFlags dstStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
+
+        VkSubmitInfo submitInfo;
+        submitInfo.sType                = VK_STRUCTURE_TYPE_SUBMIT_INFO;
+        submitInfo.pNext                = nullptr;
+        submitInfo.waitSemaphoreCount   = 1;
+        submitInfo.pWaitSemaphores      = &semaphore;
+        submitInfo.pWaitDstStageMask    = &dstStageMask;
+        submitInfo.commandBufferCount   = 1;
+        submitInfo.pCommandBuffers      = &it->second.second;
+        submitInfo.signalSemaphoreCount = 0;
+        submitInfo.pSignalSemaphores    = nullptr;
+
+        VkFrameBoundaryEXT frameBoundaryExt;
+        frameBoundaryExt.sType       = VK_STRUCTURE_TYPE_FRAME_BOUNDARY_EXT;
+        frameBoundaryExt.pNext       = nullptr;
+        frameBoundaryExt.flags       = VK_FRAME_BOUNDARY_FRAME_END_BIT_EXT;
+        frameBoundaryExt.frameID     = application_->GetCurrentFrameNumber();
+        frameBoundaryExt.imageCount  = (image == VK_NULL_HANDLE ? 0 : 1);
+        frameBoundaryExt.pImages     = (image == VK_NULL_HANDLE ? nullptr : &image);
+        frameBoundaryExt.bufferCount = 0;
+        frameBoundaryExt.pBuffers    = nullptr;
+        frameBoundaryExt.tagName     = frameBoundaryExt.frameID;
+        frameBoundaryExt.tagSize     = 0;
+        frameBoundaryExt.pTag        = nullptr;
+
+        submitInfo.pNext = &frameBoundaryExt;
+
+        VkQueue queue;
+        device_table->GetDeviceQueue(device, queueFamily, 0, &queue);
+        device_table->QueueSubmit(queue, 1, &submitInfo, VK_NULL_HANDLE);
+
+        // Destruction of command pool and command buffer is done at destruction of the device
+    }
+    else
+    {
+        func(device, semaphore, image);
+    }
+}
+
 // We want to allow skipping the query for tool properties because the capture layer actually adds this extension
 // and the application may end up using the query.  However, this extension may not be present for replay, so
 // we stub it out in that case.  This will generate warnings in the GfxReconstruct output, but it shouldn't result