vizviewer support showing .json.gz files

[yodahuang]

https://ui.perfetto.dev/ supports uploading .json.gz files while the vizview does not, with or without --use_external_processor. The error message is Do not support file type foo.json.gz.

By saving the tracing file as .gz files, the size of the trace may be 20 times smaller. As the .json.gz still need to be uncompressed before viewing, it kinda beats the benefit.

[gaogaotiantian]

Right, because vizviewer needs to read some data from the report first, before opening it - mainly the source code of the trace. If you only need the original Perfetto, you can always use their latest version. The viztracer report conforms to chrome trace format so the original Perfetto can always read it. However, if you want to customized features added by viztracer, the data needs to be read first - that will include a decompressing anyway, so gz does not help much.

[yodahuang]

Thanks. I see the explanation of the difference in #426. That said, there could still be benefit, as we are trading disk size with the time / memory of the viewer. I understand adding the functionality probably add more dependency and may not be what we want though.

---

Hijacking the issue for another quick question: for the torch profiler added recently, I found that depends on baseTimeNanoseconds, which is added to libkineto in pytorch/kineto#928, quite recent. It would be great to mention the minimum PyTorch version that viztracer support profiling. Or is it possible to let it support older PyTorch profiler format?

[gaogaotiantian]

You need to decompress the data on disk - it could be in a temp dir I guess, but it will take some of your disk space. I can do that, but gz decompressing is actually not super fast when the file is large. I don't think it requires any new dependency as python stdlib supports gz.

And yes, viztracer won't work if there's no baseTimeNanoseconds because it won't align. I'll find the version PyTorch has the support and update in the documentation (and probably a warning in the code).

[gaogaotiantian]

The baseTimeNanoseconds issue is fixed in #541.

[gaogaotiantian]

Okay turns out I overthought about this. Supporting gz is quite straightforward and it's added.