Make console logs in browser configuration details easier to read

[miketaylr]

Now that we have console logs coming in, we can think of ways to make them prettier.

cf. #2658

[karlcow]

Almost there. #2670 is here to fix a couple of things. And then we can prettify.

[karlcow]

This is an example of a log as kept right now. I'm trying to understand a couple of things.

Where the console.log( is coming from

@wisniewskit As you worked on it, could you tell me?

[u'[console.log(==== uploader chunkcheck: true ===) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:214:759]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Instantiating FileInput...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.log(Instantiating FileDrop...) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(Trying runtime: html5) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log([object Object]) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tdefault mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u'[console.log(\tselected mode: browser) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]',
 u"[console.log(Runtime 'html5' initialized) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:211:12887]",
 u'[console.info(SoundManager V2.97a.20150601 (HTML5-only mode)) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:184:11563]',
 u'[console.info(SoundManager 2 HTML5 support: mp3 = true, mp4 = true, ogg = true, opus = true, wav = true) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:184:11563]',
 u'[console.info(SoundManager 2: Ready. ) https://www.filemail.com/bundles/js/default?v=R1iSYgzt8XCQHg-EMwry4hVKE5504Mw8JlbomBZDl7o1:184:11563]',
 u'[JavaScript Error: "downloadable font: OS/2: Bad sTypoLineGap, setting it to 0: -32 (font-family: "filemail" style:normal weight:400 stretch:100 src index:1) source: https://www.filemail.com/fonts/filemail.woff?-x1gbnc" {file: "unknown" line: 2 column: 11 source: "@font-face {\n  font-family: "filemail";\n  src: url("/fonts/filemail.eot?#iefix-x1gbnc") format("embedded-opentype"), url("/fonts/filemail.woff?-x1gbnc") format("woff"), url("/fonts/filemail.ttf?-x1gbnc") format("truetype"), url("/fonts/filemail.svg?-x1gbnc#filemail") format("svg");\n  font-style: normal;\n  font-weight: normal;\n}"}]',
 u'[JavaScript Warning: "Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://127.0.0.1:56666/?_=1542831690530. (Reason: CORS request did not succeed)."]']

Adding a bit of info here.
This comes from webcompat/web-bugs#21650

if I check the log in my browser I get.

as a screenshot

as text

09:15:28.627 Navigated to https://www.filemail.com/
09:15:34.521 Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help http://xhr.spec.whatwg.org/ default:16:31368
09:15:34.870 ==== uploader chunkcheck: true === default:214:759
09:15:34.877 	default mode: browser default:211:12887
09:15:34.881 Instantiating FileInput... default:211:12887
09:15:34.883 Trying runtime: html5 default:211:12887
09:15:34.883
Object { accept: (1) […], multiple: true, required_caps: {…}, container: div#filemailuploader.fm-taf
, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", name: "file", browse_button: a#addBtn.button.large
 }
default:211:12887
09:15:34.892 	default mode: browser default:211:12887
09:15:34.894 	selected mode: browser default:211:12887
09:15:34.894 Runtime 'html5' initialized default:211:12887
09:15:34.895 Instantiating FileDrop... default:211:12887
09:15:34.898 Trying runtime: html5 default:211:12887
09:15:34.898
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: div#dropArea.dropFilesHere
, container: div#dropArea.dropFilesHere
 }
default:211:12887
09:15:34.900 	default mode: browser default:211:12887
09:15:34.901 	selected mode: browser default:211:12887
09:15:34.901 Runtime 'html5' initialized default:211:12887
09:15:34.902 Instantiating FileDrop... default:211:12887
09:15:34.902 Trying runtime: html5 default:211:12887
09:15:34.902
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: div.attachment-section
, container: div.attachment-section
 }
default:211:12887
09:15:34.904 	default mode: browser default:211:12887
09:15:34.904 	selected mode: browser default:211:12887
09:15:34.904 Runtime 'html5' initialized default:211:12887
09:15:34.905 Instantiating FileDrop... default:211:12887
09:15:34.905 Trying runtime: html5 default:211:12887
09:15:34.905
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: textarea#FilemailMessageToURL.inner-textarea.required
, container: textarea#FilemailMessageToURL.inner-textarea.required
 }
default:211:12887
09:15:34.907 	default mode: browser default:211:12887
09:15:34.908 	selected mode: browser default:211:12887
09:15:34.908 Runtime 'html5' initialized default:211:12887
09:15:34.908 Instantiating FileDrop... default:211:12887
09:15:34.909 Trying runtime: html5 default:211:12887
09:15:34.909
Object { accept: (1) […], required_caps: {…}, runtime_order: "html5,silverlight,flash", preferred_caps: {…}, swf_url: "/js/plupload3/Moxie.swf", xap_url: "/js/plupload3/Moxie.xap", drop_zone: textarea#FilemailMessage.inner-textarea.required
, container: textarea#FilemailMessage.inner-textarea.required
 }
default:211:12887
09:15:34.911 	default mode: browser default:211:12887
09:15:34.911 	selected mode: browser default:211:12887
09:15:34.911 Runtime 'html5' initialized default:211:12887
09:15:34.928 SoundManager 2: No Flash detected. Trying HTML5-only mode. default:184:11563
09:15:34.931 SoundManager V2.97a.20150601 (HTML5-only mode) default:184:11563
09:15:34.931 SoundManager 2 HTML5 support: mp3 = true, mp4 = true, ogg = true, opus = true, wav = true default:184:11563
09:15:34.932 SoundManager 2: Ready. ✓ default:184:11563
09:15:34.940 Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://127.0.0.1:56666/?_=1542845734415. (Reason: CORS request did not succeed).[Learn More]
09:15:35.359 Request to access cookie or storage on “https://www.facebook.com/tr/?id=496570193805925&ev=PixelInitialized&dl=https%3A%2F%2Fwww.filemail.com%2F&rl=&if=false&ts=1542845735214” was blocked because it came from a tracker and content blocking is enabled. www.filemail.com
09:15:36.265 Request to access cookie or storage on “https://bam.nr-data.net/1/57bd6ab2db?a=56318757&v=1099.d27c17c&to=MlNWMERRDBcDUUQPWQsZdTdmHwYBBFNFCkJLV0cUTg%3D%3D&rst=7249&ref=https://www.filemail.com/&ap=75&be=1315&fe=7147&dc=5951&perf=%7B%22timing%22:%7B%22of%22:1542845728553,%22n%22:0,%22f%22:2,%22dn%22:2,%22dne%22:50,%22c%22:50,%22s%22:326,%22ce%22:932,%22rq%22:932,%22rp%22:1285,%22rpe%22:1285,%22dl%22:1297,%22di%22:5948,%22ds%22:5951,%22de%22:6379,%22dc%22:7147,%22l%22:7147,%22le%22:7150%7D,%22navigation%22:%7B%7D%7D&jsonp=NREUM.setToken” was blocked because it came from a tracker and content blocking is enabled. www.filemail.com
09:15:39.934 XML Parsing Error: no root element found
Location: https://www.filemail.com/api/internal/languageusage/report
Line Number 1, Column 1: report:1:1

​```

[karlcow]

Also another big problem 😨
in webcompat/web-bugs#21657
We can see that the string is truncated. We need to figure out if the truncation happens on webcompat.com side or on the reporter side.

The truncation happens at 7947 chars

[karlcow]

@wisniewskit ah got an answer to my question:

Where the console.log( is coming from

https://searchfox.org/mozilla-central/rev/8f89901f2d69d9783f946a7458a6d7ee70635a94/browser/extensions/webcompat-reporter/experimentalAPIs/tabExtras.js#40-55

  function getConsoleMessages(windowIds) {
    const ConsoleAPIStorage = Cc["@mozilla.org/consoleAPI-storage;1"]
                              .getService(Ci.nsIConsoleAPIStorage);
    let messages = [];
    for (const id of windowIds) {
      messages = messages.concat(ConsoleAPIStorage.getEvents(id) || []);
    }
    return messages.map(evt => {
      const {columnNumber, filename, level, lineNumber, timeStamp} = evt;
      const args = evt.arguments.map(arg => {
        return "" + arg;
      }).join(", ");
      const message = `[console.${level}(${args}) ${filename}:${lineNumber}:${columnNumber}]`;
      return {timeStamp, message};
    });
  }

We should probably make it more structured.

[
  {
    "console": [
      {
        "level": "${level}",
        "log": "${args}",
        "uri": "${filename}",
        "pos": "${lineNumber}:${columNumber}"
      },
      {
        "level": "${level}",
        "log": "${args}",
        "uri": "${filename}",
        "pos": "${lineNumber}:${columNumber}"
      }
    ]
  }
]

[wisniewskit]

Ah, you beat me to it, sorry. Yes, I don't mind structuring these things rather than using strings. This was my first attempt. I think your suggestion is fine. When we've decided, let's open a Bugzilla bug to fix the desktop and Fennec reporters.

I don't know where the truncation happens, though. I tried to avoid intentionally truncating anything (that was of course the whole reason to use POST requests in the first place). I will try to investigate this ASAP.

[karlcow]

@wisniewskit thanks. I just reformatted a bit. Trying to make sense myself. :)

[karlcow]

Also @wisniewskit did you see all the logs which are object returns [object Object] and not the content. Probably not useful in the current form.

[wisniewskit]

No, I haven't? Please link me to one or two issues where that happens, and I'll gladly investigate that as well.

[karlcow]

@wisniewskit webcompat/web-bugs#21650 This one see the details.
Now try in your own browser.

This is comment #2659 (comment) see the log and the screenshot.

[karlcow]

SeeAlso #2717 and #2716

[karlcow]

• Paper trail https://bugzilla.mozilla.org/show_bug.cgi?id=1510073 for the structure of the log.
• Paper trail https://bugzilla.mozilla.org/show_bug.cgi?id=1510067 for the object format

[ksy36]

Regarding the object format from this bug , I've looked at the previewers that Nicolas mentioned and looks like it may be something that could work for us.

They limit a number of properties down to 10 and fold the rest:

We, perhaps, could do something similar - for all objects and arrays in console.log send maximum 10 properties/elements from the extension, so we have a "preview"? That could help with really big or nested objects.

So webcompat.com will just receive a part of it, and to see the whole object we'd need to go to the actual website and look in the console.

@karlcow @wisniewskit @denschub do you think that would be sufficient enough for diagnosis?