Re-land "Handle two sizes images" feature

[miketaylr]

• make sure bug form uploads work
• make sure tests pass: Fix failing image upload tests #1268
• make sure removing upload from preview removes link from description (see here)

[miketaylr]

cf. #1266

[miketaylr]

Ah, here's the culprit: https://github.com/karlcow/webcompat.com/blob/144bf43c449ba9ebcbb28f2284a1251e80866110/webcompat/static/js/lib/bugform.js#L405

We need to pass in the entire response on upload success, not just the URL.

[karlcow]

Found the culprit for the bugform

ah and you pointed it too in
#1271 (comment)

[karlcow]

nosetests done.

[karlcow]

hmmm functional tests…

11:01:03.176 INFO - Selenium Server is up and running
11:01:06.933 INFO - Executing: [new session: Capabilities [{idle-timeout=60, browserName=firefox, name=tests/intern}]])
11:01:06.954 INFO - Creating a new session for Capabilities [{idle-timeout=60, browserName=firefox, name=tests/intern}]
org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:
ult","description":"The default theme.","creator":"Mozilla","homepageURL":null,"contributors":["Mozilla Contributors"]},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"descriptor":"/Applications/Firefox.app/Contents/Resources/browser/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}.xpi","installDate":1483926679000,"updateDate":1483926679000,"applyBackgroundUpdates":1,"skinnable":true,"size":6856,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"hasBinaryComponents":false,"strictCompatibility":true,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"50.1.0","maxVersion":"50.1.0"}],"targetPlatforms":[],"seen":true}

grmph.

[karlcow]

With Firefox 50
with → java -jar /Users/karl/bin/selenium-server-standalone-2.53.1.jar

Let's try with a newer version of Selenium

→ wget http://selenium-release.storage.googleapis.com/3.0/selenium-server-standalone-3.0.1.jar

ahaha

→ java -jar selenium-server-standalone-3.0.1.jar 
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/openqa/grid/selenium/GridLauncherV3 : Unsupported major.minor version 52.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)

let's see

→ java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

Let's continue https://www.java.com/en/download/mac_download.jsp

And the path has changed…

→ /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

Let's try

→ /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java -jar selenium-server-standalone-3.0.1.jar 
11:44:45.956 INFO - Selenium build info: version: '3.0.1', revision: '1969d75'
11:44:45.957 INFO - Launching a standalone Selenium Server

yeah! ok back to functional tests.

11:45:39.104 WARN - Exception: The path to the driver executable must be set by the webdriver.gecko.driver system property; for more information, see https://github.com/mozilla/geckodriver. The latest version can be downloaded from https://github.com/mozilla/geckodriver/releases

Still no luck

[karlcow]

I wonder if @andreastt knows how to get it work. I'm at loss.
I have window with:

/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java -jar selenium-server-standalone-3.0.1.jar

one with

./geckodriver -b /Applications/Firefox.app/Contents/MacOS/firefox-bin -p 4444

one with

python run.py #aka the webcompat app.

one with

node_modules/.bin/intern-runner config=tests/intern user=testusername pw=testpassword

And I can't make the tests execute.
It fails with

! [POST http://localhost:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"firefox","name":"tests/intern","idle-timeout":60}}] POST /wd/hub/session did not match a known command

tests/intern
Tunnel: Ready
 
Total: Pending
Passed: 0  Failed: 0  Skipped: 0

----------|-----------|-----------|-----------|-----------|
File      |   % Stmts |% Branches |   % Funcs |   % Lines |
----------|-----------|-----------|-----------|-----------|
----------|-----------|-----------|-----------|-----------|
All files |       100 |       100 |       100 |       100 |
----------|-----------|-----------|-----------|-----------|

[andreastt]

@karlcow

! [POST http://localhost:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"firefox","name":"tests/intern","idle-timeout":60}}] POST /wd/hub/session did not match a known command

You don’t say what error you are seeing and didn’t provide a trace-level log from geckodriver, but an educated guess is that the client binding/intermediary server you are using is sending HTTP requests to the the wrong endpoint. /wd/hub/session is not a registered endpoint on the geckodriver HTTPD, as it will only respond to /session.

Support in Selenium for W3C WebDriver conforming driver implementations isn’t perfect, so it’s quite likely you are running into a specification integration bug with Selenium. Unfortunately this is all I can tell from the above.

It might be useful to try to run your tests directly against geckodriver, without going via the Selenium remote server.

[miketaylr]

IIRC, with Selenium 3, you need to have geckodriver: https://github.com/mozilla/geckodriver/releases

@karlcow can you install that and see if that fixes the issue? If so, can you file a bug to update docs as well?

[andreastt]

With Firefox 47.0.1 or newer you have to use geckodriver, but it only works with Selenium 3. You don’t have to use Selenium, but if you do Selenium has integration bugs with the W3C WebDriver specification. WebDriver conformance in geckodriver/Marionette is not complete, but better with the more recent Firefox you are using.

tl;dr: Many moving pieces trying to adapt to each other.

[karlcow]

@miketaylr @andreastt

IIRC, with Selenium 3, you need to have geckodriver
#1271 (comment)

~/bin/geckodriver -V
geckodriver 0.13.0

 /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

Firefox 50.1.0

[miketaylr]

FWIW, this is how I run tests locally (I don't fire up geckodriver manually):

testw () {
  echo "Hold on to your butts."
  wd="/Users/miket/dev/compat/webcompat.com"
  osascript -e "tell application \"Terminal\" to do script \"cd $wd; . env/bin/activate && python run.py\"" > /dev/null
  osascript -e "tell application \"Terminal\"" \
            -e "tell application \"System Events\" to keystroke \"t\" using {command down}" \
            -e "do script \"cd $wd;java -jar selenium-server-standalone-3.0.0-beta4.jar\" in front window" \
            -e "end tell" > /dev/null
  osascript -e "tell application \"Terminal\" to do script \"cd $wd; sleep 2; intern\"" > /dev/null
}

[karlcow]

Thanks @miketaylr and @andreastt

(I don't fire up geckodriver manually)

yes me too before. :)
And I do basically the same thing than you. Just manually:

(webcompatcom)08:38:58 ~/code/webcompat.com
→ python run.py
('secrets', '/Users/karl/code/webcompat.com')
 * Running on http://localhost:5000/ (Press CTRL+C to quit)
 * Restarting with stat
('secrets', '/Users/karl/code/webcompat.com')

then another window

08:40:31 ~/code/webcompat.com
→ /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java -jar ~/bin/selenium-server-standalone-3.0.1.jar
08:41:24.041 INFO - Selenium build info: version: '3.0.1', revision: '1969d75'
08:41:24.042 INFO - Launching a standalone Selenium Server
2017-01-11 08:41:24.084:INFO::main: Logging initialized @650ms
08:41:24.189 INFO - Driver provider org.openqa.selenium.ie.InternetExplorerDriver registration is skipped:
 registration capabilities Capabilities [{ensureCleanSession=true, browserName=internet explorer, version=, platform=WINDOWS}] does not match the current platform MAC
08:41:24.190 INFO - Driver provider org.openqa.selenium.edge.EdgeDriver registration is skipped:
 registration capabilities Capabilities [{browserName=MicrosoftEdge, version=, platform=WINDOWS}] does not match the current platform MAC
08:41:24.190 INFO - Driver class not found: com.opera.core.systems.OperaDriver
08:41:24.190 INFO - Driver provider com.opera.core.systems.OperaDriver registration is skipped:
Unable to create new instances on this machine.
08:41:24.190 INFO - Driver class not found: com.opera.core.systems.OperaDriver
08:41:24.190 INFO - Driver provider com.opera.core.systems.OperaDriver is not registered
2017-01-11 08:41:24.267:INFO:osjs.Server:main: jetty-9.2.15.v20160210
2017-01-11 08:41:24.298:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler@735b5592{/,null,AVAILABLE}
2017-01-11 08:41:24.367:INFO:osjs.ServerConnector:main: Started ServerConnector@cb5822{HTTP/1.1}{0.0.0.0:4444}
2017-01-11 08:41:24.367:INFO:osjs.Server:main: Started @933ms
08:41:24.367 INFO - Selenium Server is up and running

then another window.

08:44:12 ~/code/webcompat.com
node_modules/.bin/intern-runner config=tests/intern user=******* pw=********
tests/intern
Tunnel: Ready

Total: Pending
Passed: 0  Failed: 0  Skipped: 0

if I scroll up I can see that all the tests are failing.

For example

× firefox 50.1.0 on MAC 16.3.0 - User Activity (non-auth) - Trying to view someone else's activity fails (logged out)
Error: [POST http://localhost:4444/wd/hub/session/8654af62-5d7c-42f8-8819-9b29dd726c40/timeouts] connect ECONNREFUSED
Error: connect ECONNREFUSED
  at errnoException  <net.js:904:11>
  at Object.afterConnect [as oncomplete]  <net.js:895:19>
  at new CompatCommand  <node_modules/intern/runner.js:213:14>
  at <CompatCommand.target.(anonymous function) [as setFindTimeout] (/Users/karl/code/webcompat.com/node_modules/intern/node_modules/leadfoot/Command.js:592:11)>
  at Test.registerSuite.Trying to view someone else's activity fails  <logged out>
  at Test.run  <node_modules/intern/lib/Test.js:169:19>
  at <node_modules/intern/lib/Suite.js:237:13>
  at signalListener  <node_modules/intern/node_modules/dojo/Deferred.js:37:21>
  at Promise.then.promise.then  <node_modules/intern/node_modules/dojo/Deferred.js:258:5>
  at runTest  <node_modules/intern/lib/Suite.js:236:46>
  at <node_modules/intern/lib/Suite.js:249:7>
  at process._tickCallback  <node.js:419:13>
  at new CompatCommand  <node_modules/intern/runner.js:213:14>
  at <CompatCommand.target.(anonymous function) [as get] (/Users/karl/code/webcompat.com/node_modules/intern/node_modules/leadfoot/Command.js:592:11)>
  at Test.registerSuite.Trying to view someone else's activity fails  <logged out>
  at Test.run  <node_modules/intern/lib/Test.js:169:19>
  at <node_modules/intern/lib/Suite.js:237:13>
  at signalListener  <node_modules/intern/node_modules/dojo/Deferred.js:37:21>
  at Promise.then.promise.then  <node_modules/intern/node_modules/dojo/Deferred.js:258:5>
  at runTest  <node_modules/intern/lib/Suite.js:236:46>
  at <node_modules/intern/lib/Suite.js:249:7>
  at process._tickCallback  <node.js:419:13>
  at new CompatCommand  <node_modules/intern/runner.js:213:14>
  at CompatCommand.find  <node_modules/intern/node_modules/leadfoot/Command.js:22:10>
  at <CompatCommand.prototype.(anonymous function) [as findByCssSelector] (/Users/karl/code/webcompat.com/node_modules/intern/node_modules/leadfoot/lib/strategies.js:24:16)>
  at Test.registerSuite.Trying to view someone else's activity fails  <logged out>
  at Test.run  <node_modules/intern/lib/Test.js:169:19>
  at <node_modules/intern/lib/Suite.js:237:13>
  at signalListener  <node_modules/intern/node_modules/dojo/Deferred.js:37:21>
  at Promise.then.promise.then  <node_modules/intern/node_modules/dojo/Deferred.js:258:5>
  at runTest  <node_modules/intern/lib/Suite.js:236:46>
  at <node_modules/intern/lib/Suite.js:249:7>
  at new CompatCommand  <node_modules/intern/runner.js:213:14>
  at <CompatCommand.target.(anonymous function) [as getVisibleText] (/Users/karl/code/webcompat.com/node_modules/intern/node_modules/leadfoot/Command.js:656:11)>
  at Test.registerSuite.Trying to view someone else's activity fails  <logged out>
  at Test.run  <node_modules/intern/lib/Test.js:169:19>
  at <node_modules/intern/lib/Suite.js:237:13>
  at signalListener  <node_modules/intern/node_modules/dojo/Deferred.js:37:21>
  at Promise.then.promise.then  <node_modules/intern/node_modules/dojo/Deferred.js:258:5>
  at runTest  <node_modules/intern/lib/Suite.js:236:46>
  at <node_modules/intern/lib/Suite.js:249:7>
  at process._tickCallback  <node.js:419:13>
  at new CompatCommand  <node_modules/intern/runner.js:213:14>

So now I need to figure out why :)
I can see on the app side that no request has been made which is logical because it seems the issue is higher up.

[miketaylr]

@karlcow ah, I can reproduce your error. If I downgrade Firefox to 49.0.1 and use geckodriver 0.11, tests run for me locally. But do not with 50 and geckodriver 0.13. So, I'd suggest downgrading so you can make progress on this, and I'll file bugs to upgrade Intern and get things fixed for upgraded everything.

[karlcow]

I'll wait for #1277

[miketaylr]

I'll wait for #1277

This is done now. 👍

[andreastt]

Let me know if I can be of any help making the transition to geckodriver.

[miketaylr]

Thanks @andreastt! The Intern 3 upgrade has landed, which uses geckodriver under the covers. Seems to be mostly working for us. :)

https://github.com/webcompat/webcompat.com/blob/master/tests/intern.js#L39-L40

[miketaylr]

@karlcow it would be cool to try to re-land your work here. 👍

[karlcow]

OK let's put a hardline on this. It will be solved by the end of next week.