Test npm, etc. installation (#527)

* More labman to labcontrol renaming

Renames most internal instances of 'labman' to a suitable variant of
'LabControl'. For the most part, only database schema-related instances remain.

* Additional modifications to support name change

* Change labman -> labcontrol in text

Mostly this is to avoid confusion while running things like nosetests.

* Add Chrome as a dependency of Travis

Test adding Chrome as a dependency of Travis builds before requiring it
(in headless mode) for unit-testing.

* Test npm, qunit, headless-chrome additions

Test that npm exists with language: node_js specified.
node-qunit-puppeteer is a runner.js like interface for headless-chrome and qunit.
node-qunit-puppeteer installs its own chromium as a dependency.
test forgoing of chrome as an addon.
test javascript unit-testing using qunit and chrom{e|ium} using the node-qunit-puppeteer script and a trivial qunit-based test in a new directory called ctests.

* Test forcing of Node version

* Change Node version

Change version of Node from 7 to 12, to match local development.
node-qunit-puppeteer failing may be related to version differences
between local (success) and Travis (failure).

* Test qunit & unittest test cohabitation

* Arbitrary Retest

It appears that unittest failure may have been arbitrary and due to
timeout.

* Renaming test_config.cfg to qiita_test_config.cfg

Renaming test_config.cfg to qiita_test_config.cfg; at face value
test_config.cfg implies test configuration for LabControl.

* node-qunit-puppeteer wrapped in tester.py

* Exclude tester.py from unittests

nosetests assumes any .py file is a unittest, and fails when trying to
execute functions within gui/tests/tester.py.

* tester.py conflicts with nosetests

the new wrapper .py file (tester.py) conflicts with nosetests, as it
appears to expect all python files it finds in test/ to be unittests.
Moving js tests to js_tests.

* Added an exclusion for js_tests to nosetests

* fix nose at 1.3.7

fix nosetests at 1.3.7 to gain options that are present locally during
development.

* Add nose-exclude module

Adding nose-exclude module to enable --exclude-dir switch for nosetests.

* Minor changes

* Minor changes

* Remove vestigial statements

* Minor changes

Changed name of tester.py to js_tester.py to avoid confusion with
db/tests/tester.py.

Changed URL reference to qunit css/js files with file reference.

* Modified qunit reference

Modified qunit reference to remove file:/// prefix.

* Minor changes

Skeleton labcontrol config file renamed to be more appropriate to its
function.
Example Qunit tests have been removed.
Documentation cleanup.

* Flake8 setup.py

.travis.yml

@@ -1,10 +1,13 @@
 dist: trusty
 language: python
+language: node_js
+node_js:
+  - "12"
 # qiita travis has set sudo to false
 sudo: required
-addons:
+#addons:
   # install chrome for testing purposes
-  chrome: stable
+  #  chrome: stable
 group: deprecated-2017Q4
 env:
   matrix:
@@ -20,6 +23,11 @@ before_install:
   - export PATH=/home/travis/miniconda3/bin:$PATH
   # Update conda itself
   - conda update --yes conda
+  # node-qunit-puppeteer is a runner.js like interface for
+  # headless-chrome and qunit.
+  # node-qunit-puppeteer installs its own chromium as a dependency.
+  # hence, a separate chrome installation is not required.
+  - npm install -g node-qunit-puppeteer
 install:
   # Hard limit for transaction log, to prevent VHD from filling during tests
   - psql -c "ALTER SYSTEM SET max_wal_size=2"
@@ -56,15 +64,17 @@ install:
   # Deactivate the Qiita environment
   - conda deactivate
   # Set the labcontrol config file
-  - sed "s#CERTIFICATE_FILEPATH=#CERTIFICATE_FILEPATH=$PWD/support_files/server.crt#g" labcontrol/db/support_files/test_config.cfg > ~/.labcontrol.cfg.1
+  - sed "s#CERTIFICATE_FILEPATH=#CERTIFICATE_FILEPATH=$PWD/support_files/server.crt#g" labcontrol/db/support_files/skeleton_labcontrol.cfg > ~/.labcontrol.cfg.1
   - sed "s#KEY_FILEPATH=#KEY_FILEPATH=$PWD/support_files/server.key#g" ~/.labcontrol.cfg.1 > ~/.labcontrol.cfg
   # Install labcontrol
   - travis_retry conda create --yes -n labcontrol python=$PYTHON_VERSION pip nose flake8 mock
   - source activate labcontrol
   - python --version
-  - pip install -U pip
-  - pip install sphinx sphinx-bootstrap-theme coveralls
-  - pip install https://github.com/qiita-spots/qiita_client/archive/master.zip
+  - travis_retry pip install -U pip
+  - nosetests -V
+  - travis_retry pip install nose-exclude
+  - travis_retry pip install sphinx sphinx-bootstrap-theme coveralls
+  - travis_retry pip install https://github.com/qiita-spots/qiita_client/archive/master.zip
   - travis_retry pip install .
   - mkdir -p support_files
   - openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout support_files/server.key -out support_files/server.crt -subj "/C=US/ST=CA/L=LaJolla/O=/CN=localhost"
@@ -76,7 +86,8 @@ install:
   - psql -d qiita_test -c 'select * from qiita.study;'
   - labcontrol patch
 script:
-  - nosetests --with-doctest --with-coverage -v --cover-package=labcontrol
+  - python labcontrol/gui/js_tests/js_tester.py all-tests
+  - nosetests --exclude-dir=labcontrol/gui/js_tests --with-doctest --with-coverage -v --cover-package=labcontrol
   - flake8 labman setup.py scripts/*
   - labcontrol/db/tests/tester.py integration_tests
   - labcontrol/db/tests/tester.py stress_tests --num_plates_amplicon 2 --num_plates_shotgun 2

labcontrol/gui/js_tests/js_tester.py

@@ -0,0 +1,121 @@
+import click
+from os.path import join, abspath, dirname
+from subprocess import Popen, PIPE
+import os
+
+
+'''
+Notes: It's generally not a good idea to have Click commands call other click
+commands, although it can be done using context objects.
+
+This means that it's better to have an umbrella command where users can specify
+to turn off one or more tests, rather than have individual tests as separate
+commands, with an umbrella command to call them all.
+'''
+
+
+# used to establish the 'cli' group of commands.
+# all commands are specified on the command line as the first parameter.
+@click.group()
+def cli():
+    pass
+
+
+def run_js_test(timeout_in_milliseconds, html_file):
+    '''Runs all Qunit js tests found within given html_file.'''
+
+    # params to node-qunit-puppeteer need to be encapsulated into one set of
+    # quotes
+    params = []
+    params.append('--allow-file-access-from-files')
+    params.append('--no-sandbox')
+    params = ' '.join(params)
+
+    cmd = []
+    cmd.append('node-qunit-puppeteer')
+    cmd.append(html_file)
+    cmd.append(str(timeout_in_milliseconds))
+
+    cmd.append('"%s"' % params)
+    cmd = ' '.join(cmd)
+
+    p = Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE)
+    o, e = p.communicate()
+    o = o.decode("utf-8")
+    e = e.decode("utf-8")
+
+    # returncodes appear to be limited to 0 (success) and 1 (error).
+    # all information appears to write out stdout; stderr is empty
+    # on success or error.
+    if p.returncode == 0:
+        return(True, o)
+    else:
+        return(False, o)
+
+
+# Currently, there is only one command. Add more commands here.
+# note, the default has been explicitly defined here as 30,000ms. If none is
+# supplied, the node_qunit_puppeteer script itself will default to 30,000ms.
+@cli.command()
+@click.option('--timeout_in_milliseconds', required=False,
+              type=click.IntRange(0, None),
+              default=30000, show_default=True,
+              help='timeout for a unittest, in milliseconds')
+def all_tests(timeout_in_milliseconds):
+    '''Run all tests found in the same location as this script.'''
+
+    click.echo('Run all tests')
+
+    # assume all javascript qunit-based tests are referenced in one or more
+    # html files that are located within the same directory as this file
+    test_dir = abspath(dirname(__file__))
+
+    # if a single test fails, this value will become False
+    all_tests_successful = True
+
+    # build a list of html files to process. The contents of this list will
+    # then be processed in turn. The reason for this is to discover the number
+    # of files to be processed, and set up the progress bar.
+    html_files = []
+
+    for path, sub_directories, files in os.walk(test_dir, topdown=False):
+        for file in files:
+            if file.endswith('.html'):
+                html_files.append(join(path, file))
+
+    count = len(html_files)
+
+    # note that if no html files (tests) are found, this script will return
+    # success (0).
+    click.echo("%d html files found" % count)
+
+    # similarly, store the stdout of each test in results, instead of
+    # displaying them as they are returned to us. This allows the progress bar
+    # to properly stay in a single location and reach 100% when all tests have
+    # completed.
+    results = []
+
+    with click.progressbar(length=count) as bar:
+        for html_file in html_files:
+            success, test_output = run_js_test(timeout_in_milliseconds,
+                                               html_file)
+            bar.update(1)
+            results.append(test_output)
+            if not success:
+                all_tests_successful = False
+
+    for result in results:
+        click.echo(result)
+
+    # returning 0 or 1 doesn't appear to percolate up as the return code for
+    # this running script. Instead force an exit here with either 0 or 1,
+    # depending on whether one or more tests errored out. (This is also like
+    # Emperor.)
+    if all_tests_successful:
+        exit(0)
+
+    exit(1)
+
+
+if __name__ == '__main__':
+    cli()