UI is extremely slow for traces with > 2000 spans

[shooshx]

Requirement - what kind of business use case are you trying to solve?

Tracing and debugging a complex microservices system

Problem - what in Jaeger blocks you from solving the requirement?

One of the key processes has traces from 7 different services. A single trace can get up to 5000 spans
When such a span is displayed in the UI the UI get extremely slow.

• Clicking a single span to see the tags takes about 5 seconds to open
• Loading the page and switching to the tab also takes a long time
  With such a span, the top canvas that shows a summary of all the spans is basically useless since everything is so tiny in it

Proposal - what do you suggest to solve the problem or improve the existing situation?

• Fix the issue that makes opening the tags of a span take so long
• Create DOM elements only for the spans that are in view so that the page DOM tree would not be so huge
• Allow resizing the top canvas so that it could be useful

[joe-farro]

The optimizations you've suggested were implemented years ago.im wondering if your running a very old version or if there's been a serious regression... What version of jaeger-query are you running?

…

On Thu, Oct 1, 2020, 8:47 AM Shy Shalom ***@***.***> wrote: Requirement - what kind of business use case are you trying to solve? Tracing and debugging a complex microservices system Problem - what in Jaeger blocks you from solving the requirement? One of the key processes has traces from 7 different services. A single trace can get up to 5000 spans When such a span is displayed in the UI the UI get extremely slow. - Clicking a single span to see the tags takes about 5 seconds to open - Loading the page and switching to the tab also takes a long time With such a span, the top canvas that shows a summary of all the spans is basically useless since everything is so tiny in it Proposal - what do you suggest to solve the problem or improve the existing situation? - Fix the issue that makes opening the tags of a span take so long - Create DOM elements only for the spans that are in view so that the page DOM tree would not be so huge - Allow resizing the top canvas so that it could be useful — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#645>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFW5POHIIWKN7NVALKWOADSIR26PANCNFSM4SAMHQ2Q> .

[shooshx]

How can know what version it is?
Clicking About->Documentation brings me to this page: https://www.jaegertracing.io/docs/1.17/
Looking at the code I'm seeing a link to ./static/js/main.2763c360.chunk.js
Running this:
document.getElementsByTagName("*").length
In a trace page with 2819 spans returns 8619, so yea, it's definitely creating a few DOM elements for every span

[shooshx]

Can I please get any reply...? is this project being actively maintained?

[rubenvp8510]

Hi, How are you running the query service? Are you using a docker images? all-in one? if this is the case, which tag are you using?

Other way to find the version is executing the following:

• for all-in-one : docker exec -it <docker-container-id> /go/bin/all-in-one-linux version
• not all-in-one : docker exec -it <query-docker-container-id> /go/bin/query-linux version
• or if you have access to the binaries in case you are not executing on containers `/query-linux version.

As Joe mentioned, the optimizations you suggested should be already implemented but this could be a regression. I'll do some profiling to see if that is the case.

[shooshx]

Running jaeger query 1.17.1
Is this something that got better since 1.17.1 ?

[rubenvp8510]

virtualized lists was introduced earlier, and another round of performance optimizations on 1.1.0 and 1.9.0. Nothing new for 1.17 and above, not related to performance.

[shooshx]

So is anybody going to confirm this main findings?

Clicking a single span to see the tags takes about 5 seconds to open

[yurishkuro]

I regularly open much larger traces, never had issues like that. With the very large traces (10,000+ spans), it takes a few seconds to load them, but once loaded the UI handles them fine.

Can you share the JSON of your traces so that we can try to reproduce?

[shooshx]

No, unfortunately I can't share that

[yurishkuro]

You can anonymized the data in the trace. Unfortunately, our built-in anonymizer is currently unfinished (jaegertracing/jaeger#2556).

[rubenvp8510]

Hi

I wasn't able to reproduce this issue with master or the latest release, then I realized that indeed there was some performance improvement work on that view on 1.9, not on the virtualized lists but some memoize optimizations. I was able to replicate performance issues with Jaeger UI version 1.8, which is the UI version that Jaeger 1.17 uses.

@shooshx Could you please try with Jaeger 1.20 and see if you still have performance issues?

Thanks

[RusDenS]

Hi
We are faced with the same problem. Jaeger UI opening 20 tracers with more than 10,000 spans.
Perhaps there are plans in the search does not transmit trace + Spans and transmit only the list of services. And when you go to the trace, load the full list of spans? This would reduce the search time.