Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make console logs in browser configuration details easier to read #2659

Closed
miketaylr opened this issue Oct 16, 2018 · 20 comments
Closed

Make console logs in browser configuration details easier to read #2659

miketaylr opened this issue Oct 16, 2018 · 20 comments
Assignees

Comments

@miketaylr
Copy link
Member

Now that we have console logs coming in, we can think of ways to make them prettier.

screen shot 2018-10-16 at 4 27 47 pm

screen shot 2018-10-16 at 4 28 14 pm

cf. #2658

@karlcow
Copy link
Member

karlcow commented Oct 25, 2018

Almost there. #2670 is here to fix a couple of things. And then we can prettify.

@karlcow
Copy link
Member

karlcow commented Nov 22, 2018

This is an example of a log as kept right now. I'm trying to understand a couple of things.

Where the console.log( is coming from

@wisniewskit As you worked on it, could you tell me?

[u'[console.log(==== uploader chunkcheck: true ===) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:214:759]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Instantiating FileInput...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.info(SoundManager V2.97a.20150601 (HTML5-only mode)) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:184:11563]',
 u'[console.info(SoundManager 2 HTML5 support: mp3 = true, mp4 = true, ogg = true, opus = true, wav = true) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:184:11563]',
 u'[console.info(SoundManager 2: Ready. ) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:184:11563]',
 u'[JavaScript Error: "downloadable font: OS/2: Bad sTypoLineGap, setting it to 0: -32 (font-family: "filemail" style:normal weight:400 stretch:100 src index:1) source: https://www.filemail.com/fonts/filemail.woff?-x1gbnc" {file: "unknown" line: 2 column: 11 source: "@font-face {\n  font-family: "filemail";\n  src: url("/fonts/filemail.eot?#iefix-x1gbnc") format("embedded-opentype"), url("/fonts/filemail.woff?-x1gbnc") format("woff"), url("/fonts/filemail.ttf?-x1gbnc") format("truetype"), url("/fonts/filemail.svg?-x1gbnc#filemail") format("svg");\n  font-style: normal;\n  font-weight: normal;\n}"}]',
 u'[JavaScript Warning: "Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://127.0.0.1:56666/?_=1542831690530. (Reason: CORS request did not succeed)."]']

Adding a bit of info here.
This comes from webcompat/web-bugs#21650

if I check the log in my browser I get.

as a screenshot

capture d ecran 2018-11-22 a 09 18 39

as text

09:15:28.627 Navigated to https://www.filemail.com/
09:15:34.521 Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help http://xhr.spec.whatwg.org/ default:16:31368
09:15:34.870 ==== uploader chunkcheck: true === default:214:759
09:15:34.877 	default mode: browser default:211:12887
09:15:34.881 Instantiating FileInput... default:211:12887
09:15:34.883 Trying runtime: html5 default:211:12887
09:15:34.883
Object { accept: (1) […], multiple: true, required_caps: {…}, container: div#filemailuploader.fm-taf
, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", name: "file", browse_button: a#addBtn.button.large
 }
default:211:12887
09:15:34.892 	default mode: browser default:211:12887
09:15:34.894 	selected mode: browser default:211:12887
09:15:34.894 Runtime 'html5' initialized default:211:12887
09:15:34.895 Instantiating FileDrop... default:211:12887
09:15:34.898 Trying runtime: html5 default:211:12887
09:15:34.898
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: div#dropArea.dropFilesHere
, container: div#dropArea.dropFilesHere
 }
default:211:12887
09:15:34.900 	default mode: browser default:211:12887
09:15:34.901 	selected mode: browser default:211:12887
09:15:34.901 Runtime 'html5' initialized default:211:12887
09:15:34.902 Instantiating FileDrop... default:211:12887
09:15:34.902 Trying runtime: html5 default:211:12887
09:15:34.902
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: div.attachment-section
, container: div.attachment-section
 }
default:211:12887
09:15:34.904 	default mode: browser default:211:12887
09:15:34.904 	selected mode: browser default:211:12887
09:15:34.904 Runtime 'html5' initialized default:211:12887
09:15:34.905 Instantiating FileDrop... default:211:12887
09:15:34.905 Trying runtime: html5 default:211:12887
09:15:34.905
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: textarea#FilemailMessageToURL.inner-textarea.required
, container: textarea#FilemailMessageToURL.inner-textarea.required
 }
default:211:12887
09:15:34.907 	default mode: browser default:211:12887
09:15:34.908 	selected mode: browser default:211:12887
09:15:34.908 Runtime 'html5' initialized default:211:12887
09:15:34.908 Instantiating FileDrop... default:211:12887
09:15:34.909 Trying runtime: html5 default:211:12887
09:15:34.909
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: textarea#FilemailMessage.inner-textarea.required
, container: textarea#FilemailMessage.inner-textarea.required
 }
default:211:12887
09:15:34.911 	default mode: browser default:211:12887
09:15:34.911 	selected mode: browser default:211:12887
09:15:34.911 Runtime 'html5' initialized default:211:12887
09:15:34.928 SoundManager 2: No Flash detected. Trying HTML5-only mode. default:184:11563
09:15:34.931 SoundManager V2.97a.20150601 (HTML5-only mode) default:184:11563
09:15:34.931 SoundManager 2 HTML5 support: mp3 = true, mp4 = true, ogg = true, opus = true, wav = true default:184:11563
09:15:34.932 SoundManager 2: Ready. ✓ default:184:11563
09:15:34.940 Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://127.0.0.1:56666/?_=1542845734415. (Reason: CORS request did not succeed).[Learn More]
09:15:35.359 Request to access cookie or storage on “https://www.facebook.com/tr/?id=496570193805925&ev=PixelInitialized&dl=https%3A%2F%2Fwww.filemail.com%2F&rl=&if=false&ts=1542845735214” was blocked because it came from a tracker and content blocking is enabled. www.filemail.com
09:15:36.265 Request to access cookie or storage on “https://bam.nr-data.net/1/57bd6ab2db?a=56318757&v=1099.d27c17c&to=MlNWMERRDBcDUUQPWQsZdTdmHwYBBFNFCkJLV0cUTg%3D%3D&rst=7249&ref=https://www.filemail.com/&ap=75&be=1315&fe=7147&dc=5951&perf=%7B%22timing%22:%7B%22of%22:1542845728553,%22n%22:0,%22f%22:2,%22dn%22:2,%22dne%22:50,%22c%22:50,%22s%22:326,%22ce%22:932,%22rq%22:932,%22rp%22:1285,%22rpe%22:1285,%22dl%22:1297,%22di%22:5948,%22ds%22:5951,%22de%22:6379,%22dc%22:7147,%22l%22:7147,%22le%22:7150%7D,%22navigation%22:%7B%7D%7D&jsonp=NREUM.setToken” was blocked because it came from a tracker and content blocking is enabled. www.filemail.com
09:15:39.934 XML Parsing Error: no root element found
Location: https://www.filemail.com/api/internal/languageusage/report
Line Number 1, Column 1: report:1:1

​```

@karlcow
Copy link
Member

karlcow commented Nov 22, 2018

Also another big problem 😨
in webcompat/web-bugs#21657
We can see that the string is truncated. We need to figure out if the truncation happens on webcompat.com side or on the reporter side.

The truncation happens at 7947 chars

@karlcow
Copy link
Member

karlcow commented Nov 22, 2018

@wisniewskit ah got an answer to my question:

Where the console.log( is coming from

https://searchfox.org/mozilla-central/rev/8f89901f2d69d9783f946a7458a6d7ee70635a94/browser/extensions/webcompat-reporter/experimentalAPIs/tabExtras.js#40-55

  function getConsoleMessages(windowIds) {
    const ConsoleAPIStorage = Cc["@mozilla.org/consoleAPI-storage;1"]
                              .getService(Ci.nsIConsoleAPIStorage);
    let messages = [];
    for (const id of windowIds) {
      messages = messages.concat(ConsoleAPIStorage.getEvents(id) || []);
    }
    return messages.map(evt => {
      const {columnNumber, filename, level, lineNumber, timeStamp} = evt;
      const args = evt.arguments.map(arg => {
        return "" + arg;
      }).join(", ");
      const message = `[console.${level}(${args}) ${filename}:${lineNumber}:${columnNumber}]`;
      return {timeStamp, message};
    });
  }

We should probably make it more structured.

[
  {
    "console": [
      {
        "level": "${level}",
        "log": "${args}",
        "uri": "${filename}",
        "pos": "${lineNumber}:${columNumber}"
      },
      {
        "level": "${level}",
        "log": "${args}",
        "uri": "${filename}",
        "pos": "${lineNumber}:${columNumber}"
      }
    ]
  }
]

@wisniewskit
Copy link
Member

Ah, you beat me to it, sorry. Yes, I don't mind structuring these things rather than using strings. This was my first attempt. I think your suggestion is fine. When we've decided, let's open a Bugzilla bug to fix the desktop and Fennec reporters.

I don't know where the truncation happens, though. I tried to avoid intentionally truncating anything (that was of course the whole reason to use POST requests in the first place). I will try to investigate this ASAP.

@karlcow
Copy link
Member

karlcow commented Nov 22, 2018

@wisniewskit thanks. I just reformatted a bit. Trying to make sense myself. :)

@karlcow
Copy link
Member

karlcow commented Nov 22, 2018

Also @wisniewskit did you see all the logs which are object returns [object Object] and not the content. Probably not useful in the current form.

@wisniewskit
Copy link
Member

No, I haven't? Please link me to one or two issues where that happens, and I'll gladly investigate that as well.

@karlcow
Copy link
Member

karlcow commented Nov 22, 2018

@wisniewskit webcompat/web-bugs#21650 This one see the details.
Now try in your own browser.

This is comment #2659 (comment) see the log and the screenshot.

@karlcow
Copy link
Member

karlcow commented Nov 27, 2018

SeeAlso #2717 and #2716

@karlcow
Copy link
Member

karlcow commented Nov 27, 2018

@ksy36
Copy link
Contributor

ksy36 commented Jun 29, 2019

Regarding the object format from this bug , I've looked at the previewers that Nicolas mentioned and looks like it may be something that could work for us.

They limit a number of properties down to 10 and fold the rest:

Screen Shot 2019-06-28 at 10 16 17 PM

We, perhaps, could do something similar - for all objects and arrays in console.log send maximum 10 properties/elements from the extension, so we have a "preview"? That could help with really big or nested objects.

So webcompat.com will just receive a part of it, and to see the whole object we'd need to go to the actual website and look in the console.

@karlcow @wisniewskit @denschub do you think that would be sufficient enough for diagnosis?

@karlcow
Copy link
Member

karlcow commented Jul 1, 2019

Good lurking. Thanks @ksy36 Agreed with you.

@wisniewskit
Copy link
Member

So webcompat.com will just receive a part of it, and to see the whole object we'd need to go to the actual website and look in the console.

Why not only show it that way in the UI, but still send the data to webcompat.com? That way we could still (for example) access the full result as a comment in the HTML or something like that, without having to reproduce the problem ourselves.

@ksy36
Copy link
Contributor

ksy36 commented Jul 3, 2019

So this proposal is only for the objects in console log, e.g. when a site has console.log(window) we would just send 10 properties of the window object. The rest of the data will still come in full.

The main reason for that is that objects can be really huge or have circular references in them. And probably they're not as useful for diagnosis as errors, for example. Though it's hard to tell now :) We can always make it better, if preview is not enough.

@wisniewskit
Copy link
Member

Right, I suppose that's fine now that I've thought it through again.

@miketaylr
Copy link
Member Author

There's a few possible approaches to this, I think there's some very low hanging fruit here. :)

One idea (I think from @karlcow) is to just upload the console messages as a text blob and store them on disk (eventually in the cloud, once that's sorted for images), and just hyperlink to them in the issue. This could work. And if we served them as application/json, I think the built-in Firefox JSON viewer would kick in and be nice -- but only if we structured everything as a JSON object.

Maybe something like:

{
  "line 1": "logged message: blah",
  "line 2": Object()
}

Or we could save the text blob as another format and beautify the text, etc. Just an idea.

@wisniewskit
Copy link
Member

@miketaylr, I'd rather we keep them as JSON (which they are already)... too many nightmares about having to parsing text blobs that were incorrectly formatted. Unless we have zero control over the server containing the blobs, it shouldn't be too tricky to create a programmatic to check if the user wants a plain text or JSON formatted message (or we could just do a simple fetch-as-JSON and then make a table/list/text blob on the client side).

@ksy36
Copy link
Contributor

ksy36 commented Nov 21, 2019

I've tried looking at the logs in the built-in Firefox JSON viewer. It's better than what we have right now, but seems hard to navigate and catch obvious errors..

Screen Shot 2019-11-21 at 2 49 20 PM

Also tried building a simple colored list based on JSON (used devtools colors).

Screen Shot 2019-11-21 at 2 48 06 PM

This looks better, though messages about cookies blocking could be grouped together. Tried this on another site and got 74 of them :)

So perhaps we could do:

  1. Once the form is submitted, save logs as json file on webcompat (e.g. 45169.json)
  2. When building a body of a github issue instead of having console logs in <pre> tag, provide a link to webcompat.com, something like https://webcompat.com/logs/45169 ("View console logs")
  3. On this new route we parse the previously saved json and display this colored list

thoughts? @karlcow @miketaylr @wisniewskit

@miketaylr
Copy link
Member Author

This seems like a cool idea to me. And if we save the raw JSON to the server, we can change the way we display it in the future.

though messages about cookies blocking could be grouped together

That could be good -- or it might be interesting to have it reflect the way that devtools current displays it, which I guess is in the order its displayed here. Seems like a good "v2" improvement if you want to do it. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants