Writing Tests

How To Write Tests for Breedbase

Types of test

Unit tests

• are stored in the sgn repo under t/unit/<class_name>.t
• unit tests are tests that use and test Perl modules directly
• unit tests can be only used for Perl modules that do not access the database
• unit tests can be run with prove directly, or with t/test_fixture.pl

Unit Fixture tests

• are stored under t/unit_fixture/<test_name>.t
• test Perl classes that interact with the database
• are run using t/test_fixture.pl.
• Unit fixture tests are based on the SGN::Test::Fixture class, which instantiates a fixture database and has accessors for different types of database handles and schemas.
• each unit_fixture test should remove all the elements it has added and change all the elements back to what they were before after the test is completed. This can be achieved by calling the clean_up_db function in the fixture object (SGN::Test::Fixture). The tests should also change anything back to the initial state if anything has been renamed etc. in the test.
• Deletion of objects from the fixture is not recommended. To test deletion, add and object, then remove it.

Integration tests

• Integration tests test how different units cooperate
• In the Breedbase testing suite, there are two types of integration tests, mech tests and selenium tests.
• Integration tests are run using t/test_fixture.pl

Mech tests

• mech tests are based on Test::WWW::Mechanize
• are stored under t/unit_mech/<test_name>.t
• they can be used to test html or json generated by the server
• mech tests do not run javascript code, only raw html/json can be tested
• are run using t/test_fixture.pl
• the function clean_up_db in SGN::Test::Fixture can be helpful to bring the state of the fixture back to its initial state after a test has added objects to the database.

Selenium tests

• based on Selenium2
• are stored under t/selenium2/<test_name>.t
• Selenium will run a browser, and the tests will send requests to the browser.
• Tests can access the Javascript interpolated structure of a web page
• are run using t/test_fixture.pl
• again, the function clean_up_db in SGN::Test::Fixture can be helpful to bring the state of the fixture back to its initial state after a test has added objects to the database.

Old tests, no longer used

Validation tests

• the old validation tests have been moved to t/legacy/validate. They should still be run, but require an intact SGN database (such as cxgn-test).
• Tests that verify that web requests don't blow up (return a 500/etc) and that HTML/JSON/etc is valid ####Selenium version 1 tests
• the old selenium tests have been moved to t/legacy/selenium

General info about writing tests in Perl

An introduction to the philosophy of testing in Perl can be found here.

General guidelines:

• The test script lives in a folder t/ in the root directory of the component (perllib/t, sgn/t, etc). Inside the t/ directory, the first level of directories defines the type of test (unit, unit_fixture, unit_mech, or selenium2). In general, the subdirectories inside the different test type directories mirror the directory structure of the components.
• The test script should be based on Test::More or Test::Most. Use Test::Warn if you have warnings. Use Test::Exception to test exceptions and die messages.
• The test script does not take any parameters.
• If the test needs file-based test data, include them in the t/data/
• Use the diag function of Test::More to produce diagnostic output for when a test fails. However, do not produce this output for normal successful test runs!
• The last argument to most testing functions is what to print when a test runs. Make this output useful for debugging failing tests.
• Important A test should not leave changed fixture database behind. Either clean up by deleting items that the test created, or wrap the test in a $dbh->start_work() ... $dbh->rollback() block (or use other ways to execute transactions).

Using the fixture

• The test should be based on the database fixture, not a version of the production database (which is too large and unwieldy, and whose contents could change and the tests fail).
• The fixture can be cloned from GitHub, https://github.com/solgenomics/fixture.git. It should be placed in the cxgn/ directory, where the t/test_fixture.pl script will look for it.
• If you write a test using the fixture, but the fixture contains no corresponding data, the data needs to be added to the fixture (talk to Lukas).
• test using the fixture have to be run using the t/test_fixture.pl script.

More on t/test_fixture.pl

The main script to run tests is t/test_fixture.pl in the sgn repo. It creates a fixture database from the fixture dump, runs missing db patches on the fixture, starts a test server, and runs the specified test, or runs all the tests in a subdirectory if a subdirectory is specified. It bases the test configuration on sgn_test.conf. Some popular options include:

• --noserver Don't run the server
• --nocleanup Don't clean up the fixture database or logfile
• --noparallel Don't run in parallel mode

The environment variable $SGN_CONF_FILE can be used to specify a different testing configuration file than sgn_test.conf. The environment variable $TEST_DB_NAME can be used to specify an already existing test database.

How To Write Unit Tests

Unit tests should use Perl modules or other programs directly. If you need to do web requests, then you are writing an integration test, not a unit test.

You can create a new unit test and start editing it with the new_sgn_file command

new_sgn_file perl_unit_test.t mytest.t

where mytest.t will be the name of the newly created test. If you are testing SGN::Foo::Bar, then you should probably call the test t/SGN/Foo/Bar.t, so it is easy to find.

Here is an example:

#!perl
use strict;
use warnings;
use Test::More;

BEGIN {
  use_ok(  'The::Module::Being::Tested'  )
    or BAIL_OUT('could not include the module being tested');
}

# add your tests here

done_testing;

How to Write Unit tests using the Database Fixture

These tests should be in t/unit_fixture/. Some unit tests will require a database backend. It is preferable to use the database fixture as the test database. To use the fixture, include the following code at the beginning of your test script:

use lib 't/lib';
use SGN::Test::Fixture;

The SGN::Test::Fixture class makes available accessors for a database handle and different schema objects (Bio::Chado::Schema, SGN::Schema, and CXGN::Phenome::Schema), that will access the fixture. Note that you have to run the test script using t/test_fixture.pl to use the fixture features.

Perldoc of SGN::Test::Fixture has more details on its usage.

How To Write Integration Tests

All integration tests should be written based on the Selenium2 framework. The should be in t/selenium2/.

The Selenium2 framework has two components: The testing scripts and libraries, and the Selenium2 !WebDriver, which connects to the browser and needs to be running for the tests to work.

Download the WebDriver from [http://docs.seleniumhq.org/download/].

The recommended location for the web driver jar files are in ~/cxgn/selenium2/

Start the WebDriver using java -jar selenium-server-standalone-3.141.59.jar before starting the tests.

To write tests, first read the POD of SGN::Test::WWW::WebDriver int he sgn repo with

perldoc t/lib/SGN/Test/WWW/WebDriver.pm

Note that the preferred location for the actual test files is in ~/sgn/t/selenium2/

An example test may look like this:

use strict;
use lib 't/lib';
use Test::More 'tests'=>8;
use SGN::Test::WWW::WebDriver;

my $t = SGN::Test::WWW::WebDriver->new();

$t->while_logged_in_as("submitter", sub { 
    $t->get_ok('/breeders/manage_programs');
    
    my $new_bp_link = $t->find_element_ok('new_breeding_program_link', 'id', 'new breeding program link');

    $new_bp_link->click();

    my $breeding_program_name_input = $t->find_element_ok('new_breeding_program_name', 'id', 'find add breeding program name input');

    $breeding_program_name_input->send_keys('WEBTEST');

    #### and so forth...
}

Running selenium tests using docker

The selenium tests, including all the other test suites, can be run using docker-compose using the docker-compose.test.yml file in the breedbase_dockerfile repo:

docker-compose -f docker-compose.test.yml