[bug] 502 Bad Gateway on entering a utf-8 string in URL field.

[karlcow]

@softvision-oana-arbuzov found an interesting issue.

1. Go to https://staging.webcompat.com/
2. Enter "價格" in the URL field… and boom.

[karlcow]

Additional tests.

• é fails too. So anything which is not ascii.
• http://愛.com/ fails too. (which is a problem, it should be converted to http://xn--vgu.com/) aka IDN
• http://example.com/愛 works.

[karlcow]

Ah! good. I was afraid it would be an issue with url parsing and libraries not handling idn, but it seems to be a lot simpler. We are sometimes a bit loose on strings. We should straighten that or switch to python 3.

http://™.com/
127.0.0.1 - - [04/Jan/2018 09:49:19] "POST /issues/new HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/karl/.virtualenvs/webcompatcom/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/karl/code/webcompat.com/webcompat/views.py", line 227, in create_issue
    response = report_issue(form, proxy=True).json()
  File "/Users/karl/code/webcompat.com/webcompat/issues.py", line 24, in report_issue
    data=json.dumps(build_formdata(form)))
  File "/Users/karl/code/webcompat.com/webcompat/form.py", line 231, in build_formdata
    summary = '{0} - {1}'.format(domain, problem_summary)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2122' in position 0: ordinal not in range(128)

[karlcow]

webcompat.com/webcompat/form.py

Lines 223 to 225 in 58d7215

	# Do domain extraction for adding to the summary/title
	url = form_object.get('url')
	normalized_url = normalize_url(url)

form_object returns a unicode string

[karlcow]

And just realized we extract twice the url

webcompat.com/webcompat/form.py

Line 241 in 58d7215

'url': form_object.get('url'),