Memory leak with with viewports when using OpenGL3 backend

[parbo]

Dear ImGui 1.84 WIP (18307)
--------------------------------
sizeof(size_t): 4, sizeof(ImDrawIdx): 2, sizeof(ImDrawVert): 20
define: __cplusplus=199711
define: _WIN32
define: _MSC_VER=1929
define: _MSVC_LANG=201402
define: IMGUI_HAS_VIEWPORT
define: IMGUI_HAS_DOCK
--------------------------------
io.BackendPlatformName: imgui_impl_glfw
io.BackendRendererName: imgui_impl_opengl3
io.ConfigFlags: 0x00000441
 NavEnableKeyboard
 DockingEnable
 ViewportsEnable
io.ConfigViewportsNoDecoration
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x00001406
 HasMouseCursors
 HasSetMousePos
 PlatformHasViewports
 RendererHasViewports
--------------------------------
io.Fonts: 1 fonts, Flags: 0x00000000, TexSize: 512,64
io.DisplaySize: 1280.00,720.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 8.00,8.00
style.WindowBorderSize: 1.00
style.FramePadding: 4.00,3.00
style.FrameRounding: 0.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 8.00,4.00
style.ItemInnerSpacing: 4.00,4.00

My Issue/Question:

When using the opengl3 backend with viewports enabled there is a memory leak of a couple of MB per second after dragging a window outside the main viewport. It recovers if the windows is dragged back into the main viewport. I can reproduce with both example_sdl_opengl3 and example_glfw_opengl3. It does not happen with the opengl2 backend.

To reproduce: start the example, drag the "Hello, world!" window out of the viewport.

[AidanSun05]

I think this is what you're talking about:

This extra memory allocation is for supporting a second platform context for the window that was just dragged out, and I'm pretty sure there's no leak.
A memory leak is when a process doesn't deallocate its memory when it terminates. In the case of multi-viewports, the memory graph dipped down shortly after the window was dragged back into the viewport. This indicates that the extra memory was freed correctly, making it not leaked.

[parbo]

@AidanSun05 Keep it outside the main viewport. Memory usage grows indefinitely. While not a true memory leak in that the memory is returned, it is still a resource usage that makes extra viewports unusable.

[ocornut]

@AidanSun05 I presume @parbo is referring to an actual leak, as per the "a couple MB per second" comment.

It's probably highly dependent on the GPU driver and platform and there might be a possibility that updating drivers could fix it. OpenGL tends to be poorly supported. Even if the drivers are faulty, it would also really be worthwhile to investigate the cause.

For what it is worth, on my home laptop I do not see a memory increase in that window.

We get that sort of stuff mentioned from time to time (e.g. #3381 and #2981) but nobody with a repro could investigate further.

[parbo]

This is having the window outside the main viewport for 6 minutes, then putting it back. So the resources are eventually returned when the second viewport is no longer used, but the memory usage grows indefinitely until then.

[AidanSun05]

@ocornut - Thanks for clearing things up. (I couldn't reproduce this on my machine either)

@parbo - This is apparently caused by your OS, your GPU, or its driver. Updating your software might help, though as previously mentioned, we can't really investigate/solve this issue because there's no way to repro it.

[ocornut]

I tried with my Intel HD integrated card and Nvidia GTX 1060M and couldn't get things to grow here.
Confirmed GPU by printing

    #define GL_VENDOR    0x1F00
    #define GL_RENDERER  0x1F01
    const GLubyte* vendor = glGetString(GL_VENDOR);
    const GLubyte* renderer = glGetString(GL_RENDERER);
    printf("vendor = '%s'\n", vendor);
    printf("renderer = '%s'\n", renderer);

This is apparently caused by your OS, your GPU, or its driver.

We have to stay open-minded in the sense that OpenGL is a known pile of legacy mess, and there might be GL programming idioms that won't trigger driver issues the same way.

The "problem" if if you update your drivers now and it magically fixes it, it won't be investigated (again). So maybe best to take a chance and investigate.

Some ideas

• Apparently there's an OpenGL Idiom called "buffer orphaning" which might help the driver?
• I wonder if using RenderDoc could lead to any hint?

[parbo]

It only happens with my Intel GPU, not with the Nvidia. I have a Dell XPS 15 in case it helps. The Intel UHD graphics driver version is 27.20.100.9168.

[ocornut]

Here what I would do

• Comment all the code out in ImGui_ImplOpenGL3_RenderWindow() and verify if it still happen or not
• Comment the ImGui_ImplOpenGL3_RenderDrawData() call in ImGui_ImplOpenGL3_RenderWindow() and verify if it still happen or not

If it is indeed the call to ImGui_ImplOpenGL3_RenderDrawData() that does it,

• Comment out the whole loop for (int n = 0; n < draw_data->CmdListsCount; n++) in ImGui_ImplOpenGL3_RenderDrawData() and see if it still happens

If it is that loop, would you be able to look into OpenGL buffer orphaning idiom and see if modifying the code in that loop (presumably the glBufferData call) see if using orphaning idioms fixes it for you.

[parbo]

Kind of tricky doing anything when the main viewport isn't rendering, so I did bool is_main = ImGui::GetMainViewport() == draw_data->OwnerViewport; and used is_main as an additional condition in the two loops in ImGui_ImplOpenGL3_RenderDrawData(). It is definitely the glBufferData calls. I assume the calling pattern with multiple viewports makes the driver keep around old buffers for some reason.

I have very little experience with OpenGL, so I'm not sure it's anywhere near correct, but this hack seems to fix it:

    static GLsizeiptr v_max_size = 0;
    auto v_size = (GLsizeiptr)cmd_list->VtxBuffer.Size * (int)sizeof(ImDrawVert);
    if (v_size > v_max_size) {
      v_max_size = v_size;
      glBufferData(GL_ARRAY_BUFFER, v_max_size, NULL, GL_STREAM_DRAW);
    }
    static GLsizeiptr i_max_size = 0;
    auto i_size = (GLsizeiptr)cmd_list->IdxBuffer.Size * (int)sizeof(ImDrawIdx);
    if (i_size > i_max_size) {
      i_max_size = i_size;
      glBufferData(GL_ELEMENT_ARRAY_BUFFER, i_max_size, NULL, GL_STREAM_DRAW);
    }
    // Upload vertex/index buffers
    glBufferSubData(GL_ARRAY_BUFFER, 0, v_size, (const GLvoid *)cmd_list->VtxBuffer.Data);
    glBufferSubData(GL_ELEMENT_ARRAY_BUFFER, 0, i_size, (const GLvoid *)cmd_list->IdxBuffer.Data);

It should probably be done in some nicer way.

[ocornut]

Great to hear, that's one piece of the puzzle.

I think we would need to gather a maximum number of references (from other thread/issues) about what effectively works best with drivers. e.g. I worry that glBufferSubData() may carry its own source of bizarre issues.

Have you tried this?

        glBufferData(GL_ARRAY_BUFFER, (GLsizeiptr)cmd_list->VtxBuffer.Size * (int)sizeof(ImDrawVert), NULL, GL_STREAM_DRAW);
        glBufferData(GL_ARRAY_BUFFER, (GLsizeiptr)cmd_list->VtxBuffer.Size * (int)sizeof(ImDrawVert), (const GLvoid*)cmd_list->VtxBuffer.Data, GL_STREAM_DRAW);
        glBufferData(GL_ELEMENT_ARRAY_BUFFER, (GLsizeiptr)cmd_list->IdxBuffer.Size * (int)sizeof(ImDrawIdx), NULL, GL_STREAM_DRAW);
        glBufferData(GL_ELEMENT_ARRAY_BUFFER, (GLsizeiptr)cmd_list->IdxBuffer.Size * (int)sizeof(ImDrawIdx), (const GLvoid*)cmd_list->IdxBuffer.Data, GL_STREAM_DRAW);

[ocornut]

It would be worth trying to solution. We're going to have to have any of those change run on a maximum of platforms and measure performances (and possible "leaks"). I think @floooh had experience with that sort of thing?

[parbo]

Unfortunately, this does not fix it for me.

        glBufferData(GL_ARRAY_BUFFER, (GLsizeiptr)cmd_list->VtxBuffer.Size * (int)sizeof(ImDrawVert), NULL, GL_STREAM_DRAW);
        glBufferData(GL_ARRAY_BUFFER, (GLsizeiptr)cmd_list->VtxBuffer.Size * (int)sizeof(ImDrawVert), (const GLvoid*)cmd_list->VtxBuffer.Data, GL_STREAM_DRAW);
        glBufferData(GL_ELEMENT_ARRAY_BUFFER, (GLsizeiptr)cmd_list->IdxBuffer.Size * (int)sizeof(ImDrawIdx), NULL, GL_STREAM_DRAW);
        glBufferData(GL_ELEMENT_ARRAY_BUFFER, (GLsizeiptr)cmd_list->IdxBuffer.Size * (int)sizeof(ImDrawIdx), (const GLvoid*)cmd_list->IdxBuffer.Data, GL_STREAM_DRAW);

[floooh]

I think @floooh had experience with that sort of thing?

I'm seeing temporary memory spikes in my own swapchain wrapper code during window resizing, the exact nature depends on the operating system and 3D backend, but it goes from "memory usage goes up but remains stable during resize and recovers about a second after resize", to "memory keeps growing during resize and only recovers after the resize has finished" - but this is swapchain-surface related. But I haven't really found a solution to fix those memory spike issues (except of course not resizing the swapchain during window resizing).

I'm not as far yet with my ImGui-multiviewport-experiments to have a GL backend, so in this specific situation I can't help with an alternative implementation to check against unfortunately.