Fixes #712: Mock (some of the) communication with GitHub API

[miketaylr]

r? @karlcow

Here's the general strategy:

We have a @mockable_repsponse decorator to add to an API endpoint.

IF we're in test mode (python run.py -t), instead of making the request to GitHub, we serve a file from test/fixtures/ instead.

Limitations:

• Creating fixtures files is an annoying manual process. I plan on writing a script to generate/record these in the future. Write script to generate fixture files #777.
• We're not yet mocking login/logout w/ GitHub. Figure out how to mock logging in and out of the site #776.
• We're only mocking READs from the API and not WRITEs. Figure out how to mock POST/PUT/PATCH  #778.
• I need to write docs on adding new fixture files until we have a script to generate them. Write documentation for mocking API responses #775.

I needed to remove a few tests from our label tests, but will open an issue to figure that out. Solving the WRITE mocking will help, I think.

[miketaylr]

One failure on the travis push (and same failure on PR build)

FAIL: main - Search (non-auth) - Results are loaded from the query params (306ms)
AssertionError: Loading image is visible: expected 'wc-Loader js-loader' to include 'is-active'

I was having trouble with this locally as well. The mock response is too fast (like 3 - 5ms), I think it's hidden before Intern can find the loader image. This might mean we need to re-think how to test this.

[karlcow]

Ah interesting! You decided to mock at the python level. I thought that you would mock at the intern level of the functional test suite, which are the one mostly generating requests to the github API through webcompat.

https://www.sitepen.com/blog/2014/07/14/mocking-data-with-intern/

Ok let me look at this one. At first sight it looks good. Let me try to import your branch.

git checkout master
git fetch upstream;git merge upstream/master
# Importing your branch in a local branch
git checkout -b 712/1 upstream/issues/712/1
# Branch 712/1 set up to track remote branch issues/712/1 from upstream.
# Switched to a new branch '712/1'
python run.py
# * Running on http://localhost:5000/ (Press CTRL+C to quit)
# * Restarting with stat
# OK this part is working

In a separate shell window

java -jar /Users/karl/bin/selenium-server-standalone-2.46.0.jar
# 11:22:19.468 INFO - Launching a standalone Selenium Server
# 11:22:19.545 INFO - Java: Oracle Corporation 24.51-b03
# 11:22:19.545 INFO - OS: Mac OS X 10.11 x86_64
# 11:22:19.578 INFO - v2.46.0, with Core v2.46.0. Built from revision 87c69e2
# 11:22:19.706 INFO - Driver provider org.openqa.selenium.ie.InternetExplorerDriver registration is skipped:
# registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match the current platform MAC
# 11:22:19.707 INFO - Driver class not found: com.opera.core.systems.OperaDriver
# 11:22:19.707 INFO - Driver provider com.opera.core.systems.OperaDriver is not registered
# 11:22:20.134 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
# 11:22:20.134 INFO - Selenium Server is up and running

In yet another shell window:

workon webcompatcom
# (webcompatcom)11:24:13 ~/code/webcompat.com
node_modules/.bin/intern-runner config=tests/intern

I wonder if I'm missing a couple of things. Probably CSS/JS. Some tests are failing.

Interesting if I type

→ grunt

module.js:340
    throw err;
          ^
Error: Cannot find module 'grunt-legacy-log'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/Users/karl/code/webcompat.com/node_modules/grunt/lib/grunt.js:30:11)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)

[karlcow]

Another source for doing it on the intern side. I do not know if it's easier.
http://davidwalsh.name/nock
https://github.com/pgte/nock

[miketaylr]

The way I'm doing mocking this branch gets the job done, but I think it's kind of weird (and intrusive) to put in the mocking decorators right in the endpoint routes. :/

I wonder if I'm missing a couple of things.

Ah yeah, start the server in test mode: python run.py -t. There should only be 1 failure related to a loader image, which I need to fix.

I did look at Nock (and about 8 similar things -- but all my notes are local, oops). It seems like similar things work really well if your app is written in Node, or are geared towards unit testing (rather than in-browser functional testing). It might be possible to glue this and Intern together though? I'll look around.

Another option I thought about was running the site behind a proxy that intern runs func tests on, and intercepts requests to port 5000 that start with /api/ returning mock data, passing everything else through transparently. But then I chased after the current approach.

[miketaylr]

• remove unused import
• fix failing test
• consider doings something else entirely :p

[miketaylr]

(Hmm, I'm not sure about that grunt error though. O_o)

[karlcow]

(Hmm, I'm not sure about that grunt error though. O_o)

yes. Frustrating because the UI of the different branches I'm trying is not working so far. I checked the permissions in node, some were given to root and fixed them with

sudo chown -R karl:staff node_modules

Tried make update but it failed too.

npm ERR! Additional logging details can be found in:
npm ERR!     /Users/karl/code/webcompat.com/npm-debug.log
npm ERR! not ok code 0
make: *** [update] Error 3

Ah maybe more locking in

4734 error Error: EACCES, mkdir '/Users/karl/.npm/lodash/2.4.2'

Let's see

sudo chown -R karl:staff /Users/karl/.npm

Gotcha!

make update worked.
Let's try grunt.

yeeha!
Done, without errors.

Maybe I can debug @magsout issue now 432ba9e#commitcomment-13916997

[miketaylr]

Heh, computers are the worst.

[miketaylr]

Oops, I forgot to run the server in test mode before pushing my "re-write" of that failing test. :/

[miketaylr]

Needed to rebase against master to fix a conflict, so incoming destructive git push -f.

[miketaylr]

OK, tests should be passing now. I filed #791 to come back and make sure we add/re-write the test that I ripped out to get this to pass.

@karlcow, what do you think. We merge now and keep improving API mocking, including login/logout?

Or not worth the effort and re-write everything w/ a proxy? That's fine (honestly, no hard feelings), but I probably won't revisit this work for a few months if so--have to move on to other Moz goals stuff and screenshot uploading.

[karlcow]

@miketaylr in the back of my head I have the feeling there's a way that would make it easier, but I haven't found it too. So :)

To move forward and to breathe a bit, let's merge this one and we can still improve in the future. It's a step in the direction we desire. :)

💯

[miketaylr]

Cool, thanks. I might play with the proxy approach on some stray Saturday -- I think that will be simpler.