Take into account that 'null' as a value for details is a valid JSON

[karlcow]

@miketaylr Current staging is crashing on reporting issue.

[2018-10-24 06:14:13,421] INFO in views: 210.194.43.135 http://14.3.0.example.com/
Traceback (most recent call last):
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/webcompat/staging.webcompat.com/env/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "./webcompat/views.py", line 242, in create_issue
    json_response = report_issue(form, proxy=True)
  File "./webcompat/issues.py", line 31, in report_issue
    data=json.dumps(build_formdata(form)))
  File "./webcompat/form.py", line 357, in build_formdata
    body += build_details(details)
  File "./webcompat/form.py", line 161, in build_details
    console_logs = content.pop('consoleLog', None)
AttributeError: 'NoneType' object has no attribute 'pop'

This is what is breaking it.

webcompat.com/webcompat/form.py

Lines 157 to 165 in 1f59479

	console_logs = None
	content = details
	try:
	content = json.loads(details)
	console_logs = content.pop('consoleLog', None)
	except ValueError:
	# if we got a ValueError, details was a string, so just pass it
	# into get_details below
	pass

This was written for solving issue #2653 with the request #2658

So basically we get an issue when the content is None

json.loads(details) might return None. We need to find out why.

One way to have a more robust way of doing things here.

[karlcow]

if I print locally details, the value is indeed null

form_object

MultiDict([('problem_category', u'unknown_bug'), ('username', u''), ('csrf_token', u'*****'), ('description', u"let's try"), ('reported_with', u'web'), ('url', u'https://test.example.com/'), ('image', u''), ('submit_type', u'github-proxy-report'), ('details', u'null'), ('steps_reproduce', u'what'), ('ua_header', u'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:64.0) Gecko/20100101 Firefox/64.0'), ('os', u'Mac OS X 10.14'), ('browser', u'Firefox 64.0')])
null

ahah indeed.

>>> import json
>>> json.loads('null')
>>> # doesn't return anything. Aka None.
>>> json.loads('{"foo": "bar"}')
{u'foo': u'bar'} # this is working
>>> json.loads('{"foo"}') # This will fail indeed
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting : delimiter: line 1 column 7 (char 6)

So it's why the try is not doing what we are expecting.

[karlcow]

We need an additional test here for strings.
And probably a different way to test this.

[miketaylr]

Thank you for finding this bug! I will spend time on it this afternoon and see what we can come up with.

[karlcow]

I will fix this. :)

[karlcow]

so we have two issues:

1. json.loads('blah') doesn't generate any errors
2. content.pop() will throw when content is a string as it should.

[karlcow]

ahaha. Just understood what was really happening.

>>> json.loads(u'cool')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
>>> json.loads('cool')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
>>> json.loads(u'null')
>>> 

null is a special case and "valid" JSON value.

[karlcow]

Here is the list of values with a special meaning.
https://docs.python.org/2/library/json.html#json.JSONDecoder

JSON         | Python
--------------------
object       | dict
array        | list
string       | unicode
number (int) | int, long
number (real)| float
true         | True
false        | False
null         | None