Initial docstring linting.

- Apply one docstring lint to the global lint list (D302 - unicode docstrings should start with u""").
- Create an inclusion list with many more docstring checks (public modules need a docstring, white space and quoting issue, check imperative mood and capitializaton). Add a couple modules to this inclusion list.
- Add tox and Travis targets for this linting.

This gives a person who wants to work on improving Galaxy's in-code docstring documentation three ways to "raise the bar" for the project. One could:
- Add more modules to the inclusion list (.ci/flake8_docstrings_include_list.txt). Many more modules should contain docstrings - especially things in galaxy-lib and web controllers.
- Add more checks to the existing inclusion list checks - I'd say D101 and/or D103 first or any of the missing D2XX or D4XX options. D102 seems a bit too ambitious and I worry it'd be too much of a burden.
- Pick some DXXX check that is ignored globally and fix it for the whole project and then remove it from the exclude list in setup.cfg.

Author: jmchilton

.ci/flake8_docstrings_include_list.txt

@@ -0,0 +1,2 @@
+lib/galaxy/jobs/metrics
+lib/galaxy/exceptions

.ci/flake8_wrapper.sh

@@ -5,4 +5,4 @@ set -e
 flake8 --exclude `paste -sd, .ci/flake8_blacklist.txt` .
 
 # Apply stricter rules for the directories shared with Pulsar
-flake8 --ignore= --max-line-length=150 lib/galaxy/jobs/runners/util/
+flake8 --ignore=D --max-line-length=150 lib/galaxy/jobs/runners/util/

.ci/flake8_wrapper_docstrings.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+set -e
+
+# D100 - Missing docstring in public module.
+# D2XX - Whitespace issues.
+# D3XX - Quoting issues.
+# D401 - First line should be in imperative mood 
+# D403 - First word of the first line should be properly capitalized
+args="--ignore=D --select=D100,D201,D202,D206,D207,D208,D209,D211,D3,D401,D403"
+
+# If the first argument is --include, lint the modules expected to pass. If
+# the first argument is --exclude, lint all modules the full Galaxy linter lints
+# (this will fail).
+
+if [ "$1" = "--include" ];
+then
+    flake8 $args `paste .ci/flake8_docstrings_include_list.txt`
+else
+    flake8 $args --exclude `paste -sd, .ci/flake8_blacklist.txt` .
+fi

.travis.yml

@@ -11,6 +11,7 @@ env:
   - TOX_ENV=py27-lint-imports
   - TOX_ENV=py27-lint-imports-include-list
   - TOX_ENV=validate-test-tools
+  - TOX_ENV=py27-lint-docstring-include-list
 
 matrix:
   include:

lib/galaxy/util/__init__.py

@@ -570,7 +570,7 @@ def mask_password_from_url( url ):
 
 
 def ready_name_for_url( raw_name ):
-    """ General method to convert a string (i.e. object name) to a URL-ready
+    u""" General method to convert a string (i.e. object name) to a URL-ready
     slug.
 
     >>> ready_name_for_url( "My Cool Object" )

setup.cfg

@@ -1,12 +1,13 @@
 [flake8]
-# These are exceptions allowed (encouraged?) by Galaxy style guidelines.
+# These are exceptions allowed by Galaxy style guidelines.
 # 128 continuation line under-indented for visual indent
 # 201 and 202 are spaces after ( and before )
 # 203 whitespace before ':'
 # 402 module level import not at top of file # TODO, we would like to improve this.
 # 501 is line length
 # W503 is line breaks before binary operators, which has been reversed in PEP 8.
-ignore = E128,E201,E202,E203,E501,E402,W503
+# D** are docstring linting - which we mostly ignore except D302. (Hopefully we will solve more over time).
+ignore = E128,E201,E202,E203,E501,E402,W503,D100,D101,D102,D103,D104,D105,D200,D201,D202,D204,D205,D206,D207,D208,D209,D210,D211,D300,D301,D400,D401,D402,D403
 exclude = lib/galaxy/util/jstree.py
 # For flake8-import-order
 # https://github.com/PyCQA/flake8-import-order/blob/master/tests/test_cases/complete_smarkets.py

tox.ini

@@ -1,11 +1,13 @@
 [tox]
-envlist = py27-lint, py27-lint-imports, py27-lint-imports-include-list, py27-unit, qunit, mako-count, web-controller-line-count, py33-lint, py34-lint, py35-lint, validate-test-tools
+envlist = py27-lint, py27-lint-imports, py27-lint-imports-include-list, py27-unit, qunit, mako-count, web-controller-line-count, py33-lint, py34-lint, py35-lint, validate-test-tools, py27-lint-docstring-include-list, py27-lint-docstring
 skipsdist = True
 
 [testenv:py27-lint]
 commands = bash .ci/flake8_wrapper.sh
 whitelist_externals = bash
-deps = flake8
+deps =
+    flake8
+    flake8_docstrings==1.0.2
 
 [testenv:py33-lint]
 commands = bash .ci/flake8_py3_wrapper.sh
@@ -70,3 +72,19 @@ whitelist_externals = bash
 [testenv:validate-test-tools]
 commands = bash .ci/validate_test_tools.sh
 whitelist_externals = bash
+
+[testenv:py27-lint-docstring]
+commands = bash .ci/flake8_wrapper_docstrings.sh --exclude
+whitelist_externals = bash
+skip_install = True
+deps =
+    flake8
+    flake8-docstrings==1.0.2
+
+[testenv:py27-lint-docstring-include-list]
+commands = bash .ci/flake8_wrapper_docstrings.sh --include
+whitelist_externals = bash
+skip_install = True
+deps =
+    flake8
+    flake8-docstrings==1.0.2