openQA is an automated test tool that makes it possible to test the whole installation process of an operating system. It’s free software released under the GPLv2 license. The source code and documentation are hosted in the os-autoinst organization on GitHub.
This document provides the information needed to start contributing to the openQA development improving the tool, fixing bugs and implementing new features. For information about writing or improving openQA tests, refer to the Tests Developer Guide. In both documents it’s assumed that the reader is already familiar with openQA and has already read the Starter Guide. All those documents are available at the official repository.
As mentioned, the central point of development is the os-autoinst organization on GitHub where several repositories can be found:
-
openQA containing documentation, server, worker and other support scripts.
-
os-autoinst with the standalone test tool.
-
os-autoinst-distri-opensuse containing the tests used in http://openqa.opensuse.org
-
os-autoinst-needles-opensuse with the needles associated to the tests in the former repository.
-
os-autoinst-distri-example with an almost empty set of tests meant to be used to start writing tests (and creating the corresponding needles) from scratch for a new operating system.
As in most projects hosted on GitHub, pull request are always welcome and are the right way to contribute improvements and fixes.
-
Every commit is checked by Travis CI as soon as you create a pull request but you should run the openQA tests locally, i.e. before every commit call
./script/tidy
to ensure your perl code changes are consistent with the style rules
-
You may also run local tests on your machine or in your own development environment to verify everything works as expected. Call
make test
for unit and integration tests.
-
For git commit messages use the rules stated on How to Write a Git Commit Message as a reference
-
Every pull request is reviewed in a peer review to give feedback on possible implications and how we can help each other to improve
If this is too much hassle for you feel free to provide incomplete pull requests for consideration or create an issue with a code change proposal.
But developers willing to get really involved into the development of openQA or people interested in following the always-changing roadmap should take a look at the openQAv3 project in openSUSE’s project management tool. This Redmine instance is used to coordinate the main development effort organizing the existing issues (bugs and desired features) into ‘target versions’.
Currently developers meet in IRC channel #opensuse-factory and in a daily jangouts call of the core developer team.
In addition to the ones representing development sprints, two other versions are always open. Easy hacks lists issues that are not specially urgent and that are considered to be easy to implement by newcomers. Developers looking for a place to start contributing are encouraged to simply go to that list and assign any open issue to themselves. Future improvements groups features that are in the developers' and users' wish list but that have little chances to be addressed in the short term, either because the return of investment is not worth it or because they are out of the current scope of the development.
openQA and os-autoinst repositories also include test suites aimed at preventing bugs and regressions in the software. Coveralls is configured in the repositories to encourage contributors to raise the tests coverage with every commit and pull request. New features and bug fixes are expected to be backed with the corresponding tests.
Everything in openQA, from os-autoinst to the web frontend and from the tests
to the support scripts is written in Perl. So having some basic knowledge
about that language is really desirable in order to understand and develop
openQA. Of course, in addition to bare Perl, several libraries and additional
tools are required. The easiest way to install all needed dependencies is
using the available os-autoinst and openQA packages, as described in the
Installation Guide.
In the case of os-autoinst, only a few CPAN modules are
required. Basically Carp::Always, Data::Dump. JSON and YAML. On the other
hand, several external tools are needed including
QEMU,
Tesseract and
OptiPNG. Last but not least, the
OpenCV library is the core of the openQA image matching
mechanism, so it must be available on the system.
The openQA package is built on top of Mojolicious, an excellent Perl framework for web development that will be extremely familiar to developers coming from other modern web frameworks like Sinatra and that have nice and comprehensive documentation available at its home page.
In addition to Mojolicious and its dependencies, several other CPAN modules are required by the openQA package. For a full list of hard dependencies, see the file DEPENDENCIES.txt at the root of the openQA repository. Some additional modules could be required when using a database engine other than the default SQLite.
As already mentioned, openQA relies on a database engine to store the information. PostgreSQL, MySQL and SQLite3 are supported, with the latter being the default option.
Lastly, png2theora (part of the Theora project) is
used to create a video from the screenshots generated by os-autoinst.
As stated in the previous section, every feature implemented in both packages
should be backed by proper tests.
Test::More is used to implement those
tests. As usual, tests are located under the /t/ directory. In the openQA
package, one of the tests consists of a call to
Perltidy to ensure that the contributed code
follows the most common Perl style conventions.
During the development process there are cases in which the database schema
needs to be changed. After modifying files in lib/OpenQA/Schema/Result
there are some steps that have to be followed so that new database instances
and upgrades include those changes.
-
First, you need to increase the database version number in the $VERSION variable at the first lines of the
lib/OpenQA/Schema/Schema.pmfile. Note that it’s recommended to notify the other developers before doing so, to synchronize in case there are more developers wanting to increase the version number at the same time. -
Then you need to generate the deployment files for new installations, this is done by running
./script/initdb --prepare_init. -
Afterwards you need to generate the deployment files for new installations, this is done by running
./script/upgradedb --prepare_upgrade. After doing so, the directoriesdbicdh/$ENGINE/deploy/<new version>anddbicdh/$ENGINE/upgrade/<prev version>-<new version>for SQLite, PosgreSQL and MySQL should have been created with some SQL files inside containing the statements to initialize the schema and to upgrade from one version to the next in the corresponding database engine. -
And finally, you need to create the fixtures files. Under
dbicdh/_common/deploy, rename the directory of the (previous) latest version to the new version and do the necessary changes (if any). Then, underdbicdh/_common/upgradecreate a<prev_version>-<new_version>directory and put some files there with SQL statements that upgrade the fixtures. Usually a diff from the previous version to the new one helps to see what has to be in the upgrade file.
The above steps are executed in the developer’s system. Once openQA is
installed in a production server, you should run either
./script/initdb --init_database or ./script/upgradedb --upgrade_database to actually
create or upgrade a database.
Fixtures (initial data stored in tables at installation time) are stored
in files into the dbicdh/_common/deploy/_any/<version> and
dbicdh/_common/upgrade/<prev_version>-<next_version> directories.
You can create as many files as you want in each directory. These files contain SQL statements that will be executed when initializing or upgrading a database. Note that those files (and directories) have to be created manually and they shouldn’t create a transaction, since each file is already automatically executed in its own transaction (so that changes are rolled back if there’s any problem) and sqlite doesn’t support nested transactions.
Executed SQL statements can be traced by setting the DBIC_TRACE environment
variable.
export DBIC_TRACE=1
It can be necessary during development to change the config files in etc/.
For example you have to edit etc/openqa/database.ini to use another database.
Or to increase the log level it’s useful to set the loglevel to debug in
etc/openqa/openqa.ini.
To avoid these changes getting in your git workflow, copy them to a new directory and set OPENQA_CONFIG in your shell setup files.
cp -ar etc/openqa etc/mine export OPENQA_CONFIG=$PWD/etc/mine
Note that OPENQA_CONFIG points to the directory containing openqa.ini, database.ini, client.conf and workers.ini.
OpenQA comes with three authentication modules providing authentication methods: OpenID, iChain and Fake (see Installation - User authentication).
All authentication modules reside in lib/OpenQA/Auth directory. During
OpenQA start, [auth]/method section of /etc/openqa/openqa.ini is read and according
to its value (or default OpenID) OpenQA tries to require OpenQA::WebAPI::Auth::$method.
If successful, module for given method is imported or the OpenQA ends with error.
Each authentication module is expected to export auth_config,
auth_login and auth_logout functions. In case of request-response mechanism (as in
OpenID), auth_response is imported on demand.
Currently there is no login page because all implemented methods use either 3rd party page or none.
Authentication module is expected to return HASH:
%res = (
# error = 1 signals auth error
error => 0|1
# where to redirect the user
redirect => ''
);Authentication module is expected to create or update user entry in OpenQA database after user validation user. See included modules for inspiration.