A library for executing keyword driven tests with Protractor.
testx is a library for end-to-end keyword driven tests. testx IS NOT a framework - you can use it in your protractor project to make test automation a breeze. testx is meant for testers. It requires a very limited set of technical skills. It is even suitable for the business people in your project.
testx aims to be simple, extensible and easy to integrate.
Simplicity is the defining quality of testx. It has very few concepts and avoids variations whenever possible. The goal is to have non-technical people feel right at home when automating their tests. This is why testx tries to think about everything from the user point of view.
In order to make testx as simple as possible we need to make certain sacrifices. The main of those is the fact that testx is logic-less. This means that in a testx script there are no branches and no loops. A testx script is just a list of steps.
However, sometimes you need something more complex, like a loop, in your tests. Because of situations like this testx makes it very easy to extend it, while keeping the core principle of simplicity intact.
testx relies on protractor to do the heavy lifting - anything you can do with Protractor is still possible. A very important part is the ability to integrate your test execution into your continuous deployment pipeline. testx tries really hard to not make this any more difficult.
A testx project is just a protractor project. Just install testx as a dependency and you can start running your testx scripts in (a part of) your test run.
In line with the core concept of simplicity, testx provides you with a CLI. It helps you create a testx project containing useful examples. Easy!
The main thing that testx does is to execute (run) a testx script. This happens in your protractor specification and looks like this:
describe("My first", () =>
it("testx test!", () =>
testx.run("tests/scripts/my-first-test.testx")));All the rest of it is in the testx script file.
Each testx script is just a list of steps. These steps are synchronous and are executed in sequence. The script (usually) lives in a .testx file. By default this is a YAML file. Every step consists of two things - the keyword to be executed at this step, and the (zero or more) parameters of the execution. In general the steps are just actions or checks performed on objects (elements). Simple!
Here is an example:
- go to: /
- set:
searchBox: "testxio\n"
- check matches:
orgName: 'testxio'There are 3 steps in this script - go to, set and check matches.
- The go to keyword is responsible for taking us to a URL - "/" is a URL relative to whatever baseUrl (test run parameter) is.
- The set keyword types "testxio" in the "searchBox" object (it happens to be an input box, we'll talk about objects later) and presses the Enter key ("\n").
- The check matches checks that the content of the "orgName" object (an H1 html element in this case) RegEx match the text "testxio".
Keywords tell testx what action you want to perform and the parameters of the keyword tell it which element (for example) you want to target. The combination of these two concepts represents a step. The type of action can be a click on a button or a link, typing something into an input box, etc. Keywords usually target (do something with) objects - the elements in the browser - or the test context (more on that later). The core keywords, the ones that come with testx, are generally about actions, that a user performs in the browser. A list of all core action is available further in this documentation.
Objects in testx are an abstraction of the HTML elements in the browser. You can think of an object as the name of a particular, actionable element. They are discovered by testx with a locator.
You need this sort of abstraction to shield you from changes in the exact HTML structure of the application under test. With an object, whenever a "logical" element on the screen changes (e.g. you need a different xpath/css to locate the element) changes you'll just change its locator and that's it, you don't need to change any of your scripts.
Locators are organized into what we refer to as the object map, but this is just a fancy name for a JSON object, where keys are the name of the object and the values are the locators.
The names of the objects (in the object map) are how you refer to these objects in your testx scripts.
Key characteristics of testx are its simplicity and its ease of use. When it comes down to dealing with objects, this means that testx knows what to do with them by default. In other words, when you tell testx to set a particular object it knows how to do it - it will type into input boxes, select from dropdowns or click buttons.
Sometimes you want to do something that is not common with a particular object on the screen or maybe you have a complex object that behaves in a very specific way (think of a wysiwyg text editor, for example). In such a case testx lets you define the behaviour of this element. This means that when you set or get (used in checks, for example) this object, custom code will be executed.
To do this, you need to provide a behaviour property to the object. It can have any or all of 3 polymorphic properties - get, set and wait. Get is used to retrieve the value of the element and is used by testx when it performs checks. Set is used by the set keyword and specifies the way testx manipulates this object. Wait describes how testx will know if the objects is on the screen and ready for manipulation.
Let's say you have a single link on the page and you want to check if the href attribute of that link (and not the default value of it, which is the text of the link) equals a certain value. There are a few ways to do that, but an example that uses the custom behaviour would look something like this:
In your objects definition:
const objects = {
myLink: {
locator: "xpath",
value: "//a",
behaviour: {
get: function() { return this.getAttribute("href") }
}
}
}And then in your script:
- go to: /
- check equals:
myLink: https://testx.ioThe test context includes all variables (and functions) you may need during the execution. It allows you to reuse your scripts. For example in this (contrived) script:
- go to: ${customUrl}you can pass customUrl in the test context and testx will navigate to that URL. This let's you run the same tests against different deployments, for example.
It is possible to not use YAML format for your scripts, but instead implement a custom parser for the format you prefer. TBA.
One of the most important things in every test execution is reporting the results. There are 3 types of default reporters, but you can implement your own (TBA).
The default ones are console, junit and HTML. They are found in the testx-jasmine-reporters project - check it out to see how to use the reporters.
The testx API is pretty simple. The only thing you absolutely need to know is how to run a testx script. Once you have require-ed testx (you have to do this in the onPrepare function of your Protractor configuration) you can run a testx script in you specs like:
const context = {
someVar: 1,
anotherVar: "text",
someFunc: () => "more text"
}
testx.run("path/to/the/testx-script.yml");
testx.run("path/to/the/testx-script.yml", context);testx.objects.add(myObjectsDict);Usually you'll have the objects defined somewhere else and a lot more common is:
testx.objects.add(require("path/to/myObjects.js"));It is possible to use a CSV file to define your objects. In this case you'd do:
testx.objects.add("path/to/myObjects.csv");Adding variables and functions to the context is also very simple:
testx.context.add({"myVar": "some text"});It is of course possible to add custom keywords to the execution. This is very common for non-trivial tests.
You'll add keywords like so:
const keywords = {
myKwd: (kwdArguments, context) => //do something here
}
testx.keywords.add(keywords);And then use them like so:
- myKwd:
firstArg: 1
secondArg: this is another argumentThese are the default keywords that come with testx.
Set performs an action on an element. It is polymorphic, meaning that the action depends on the type of the element. For example a set on an input will fill it in, while a set on a button will click it.
- set:
myInput: this text goes in
myButton: nullIn this example myInput is the reference (the name) of an input element and "this text goes in" will be "typed" in the input element. On the other hand myButton is a button that will be clicked. Actions on the elements are ordered, in this example the text will be typed in the input box firs and only then the button gets clicked.
Passing null as the action means click. So
- set:
myInput: nullwill cause testx to only click in the input box myInput.
Default actions per element type are: TBA
Check text, attribute value, existence, enabled or readonly properties of an object.
- check equals:
searchBox: testxio
resLink('testxio'): testxio · GitHub
- check equals:
resLink('testxio'):
href: https://github.com/testxio
host: github.com
isConnected: 'true'
nodeType: '1'
- check matches:
resLink('testxio'):
href: github\.com/\w{4}xio
localName: \w
- check matches:
resLink('testxio'):
href: https://github\..om/testxio
host: github\.com
isConnected: 't..e'
nodeType: '[1|2]'
- check matches:
searchBox: test[x|z].o
resLink('testxio'): tx\w{2} · GitHub
- check not equals:
searchBox: testx1111.io
resLink('testxio'): something else
- check not matches:
searchBox: testx1111\.io
resLink('testxio'): test11o
- check exists:
resLink('testxio'): true
resLink('no such thing'): false
- check enabled:
resLink('testxio'): true
searchBox: true
- check readonly:
resLink('testxio'): false
searchBox: falseWait for the (dis)appearance of an object.
- wait: googleSearchForm
- wait to appear: googleSearchForm
- wait:
- resLink('{{match}}')
- googleSearchForm
- wait:
to: appear
objects:
- resLink('{{match}}')
- wait to appear:
- resLink('{{match}}')
- wait to appear:
timeout: 2s
objects:
- resLink('{{match}}')
- wait:
to: disappear
objects:
- resLink('no such thing')
- wait to disappear: resLink('no such thing, really')
- wait to disappear:
- resLink('no such thing, really')- go to:
url: / # go to a path relative to the baseUrl
- go to: /test/index.html # shortcut to the version above
- go to: http://testx.io
- go back
- go forward
- refresh pageAssertions. expect result checks the result of the keyword executed before it. It can be used as a keyword, but it can be passed as a parameter to any other keyword as well. In the example below the id is a custom keyword that just returns its parameters, i.e. they are the result of the keyword.
These are useful when you want to do some processing of a text, that you get from the screen, and only then do assert. Use the check keywords when wanting to directly assert values of objects.
- go to:
url: /
expect result:
to equal: /
not to equal: f
to match: .
not to match: d
- custom kwd that just returns params:
test: test123
expect result:
to equal:
test: test123
not to equal:
test: sdaf
- expect:
${$result.test}:
to equal: test123
not to equal: something else
to match: \w{4}\d{3}
not to match: completely different
- put:
someVar: 1
- expect:
${someVar}:
to equal: 1
- id:
test: test123
- expect result:
to equal:
test: test123Runs another testx script, optionally passing it a context:
- run:
script: tests/scripts/sample.testx # the script to run
context: # this goes in the testx context
myVar: myVal
- run: tests/scripts/no-context.testx # no context shortcutPuts stuff in the test context of testx.
The following example puts 2 values in the test context - an object bound to the myFirstVar variable and an array bound to the secondVar variable. These values can be used in subsequent steps with ${myFirstVar} and ${secondVar}
- put:
myFirstVar:
one: two
three:
- four
- and five more
six: 'seven nine,ten'
secondVar:
- nine
- tenSaves the current value of an object into a context variable. In the example below the value (say text) of myElement is saved to myContextVar; you'll be able to retrieve it in subsequent steps with ${myContextVar}
- save #
myContextVar: myElement- clear local storage
- delete cookies
- ignore synchronization: true
- ignore synchronization: yes
- ignore synchronization: falseThis keyword will switch the test execution to different window or frame.
- switch to:
title: "My new window"
- switch to:
frame: "My new frame ID"- respond to dialog: OK
- respond to dialog:
response: ok
- respond to dialog:
response: CancelThe sleep keyword will pause the execution of the test. It is strongly recommended that you only use it only while developing your tests.
- sleep: 5s # the execution will stop here for 5 seconds
- sleep: 500ms # the execution will stop here for half a secondTBA
A core characteristic of testx is its extensibility. This means it is very easy to develop keywords packages, for example. It is of course possible to extend testx in other areas of functionality, but 99% of the time you'll extend with a keyword package.
Here is what we have so far:
| Npm package | Description |
|---|---|
| testx-http-keywords | Keywords to send simple http requests using the testx library. |
| testx-file-keywords | Keywords to test file (text and pdf) content using the testx library. |
| testx-pop3-keywords | Keywords to test mailserver using pop3. |
| testx-postgres-keywords | Keywords to test postgres db. |
| testx-soap-keywords | Extension for testx to test soap services. |
To make developing tests easier we've developed an Atom syntax plugin.