BrowserStack Examples TestNG

Introduction

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful.

This BrowserStack Example repository demonstrates a Selenium test framework written in TestNG with parallel testing capabilities. The Selenium test scripts are written for the open source BrowserStack Demo web application (Github). This BrowserStack Demo App is an e-commerce web application which showcases multiple real-world user scenarios. The app is bundled with offers data, orders data and products data that contains everything you need to start using the app and run tests out-of-the-box.

The Selenium tests are run on different platforms like on-prem, docker and BrowserStack using various run configurations and test capabilities.

---

Repository setup

• Clone the repository

• Ensure you have the following dependencies installed on the machine

  • Java >= 8
  • Maven >= 3.1+

  Maven:

   mvn install -DskipTests

  Gradle:

   gradle clean build

About the tests in this repository

This repository contains the following Selenium tests:

Module	Test name	Description
E2E	OrderTest	This test scenario verifies successful product purchase lifecycle end-to-end. It demonstrates the Page Object Model design pattern and is also the default test executed in all the single test run profiles.
Login	LoginTest	This test verifies the login workflow with different types of valid login users.
Login	LoginFailTest	This test verifies the login workflow error.
Login	LoginDataDrivenTest	This test verifies the login for all error cases in a datadriven way
Login	LoginDataDrivenReadFromCSVTest	This test verifies the login for all error cases in a datadriven way with CSV-file
Login	LoginRequestedTest	This test verifies that the login page is shown when you access the favourites page with being logged in
Offers	OfferTest	This test mocks the GPS location for Singapore and verifies that the product offers applicable for the Singapore location are shown.
Product	FilterTest	This test verifies that both filter options are working
User	UserTest	The first test verifies that the product images load for user: "image_not_loading_user" on the e-commerce application. Since the images do not load, the test case assertion fails. The second test verifies that existing orders are shown for user: "existing_orders_user"

---

Test infrastructure environments

• On-premise/self-hosted
• BrowserStack

Test Reporting

• Allure reports

---

On Premise / Self Hosted

This infrastructure points to running the tests on your own machine using a browser (e.g. Chrome) using the browser's driver executables (e.g. ChromeDriver for Chrome). Selenium enables this functionality using WebDriver for many popular browsers.

Prerequisites

• For this infrastructure configuration (i.e on-premise), ensure that the ChromeDriver executable is placed in the /src/test/resources/drivers folder.

Note: The ChromeDriver version must match the Chrome browser version on your machine.

Running Your Tests

Run a specific test on your own machine

• How to run the test?

  To run the default test scenario (e.g. End to End Scenario) on your own machine, use the following command:

  Maven:

  mvn clean test -P on-prem

  Gradle:

  gradle clean on-prem

  To run a specific test scenario, use the following command with the additional 'test' argument:

  Maven:

  mvn clean test -P on-prem -Dtest=LoginDataDrivenTest
  

  Gradle:

  gradle clean on-prem -Ptest-name=LoginDataDrivenTest

  where, the argument test or test-name can be any testclass implemented this repository.

• Output

  This run profile executes a specific test scenario on a single browser instance on your own machine.

Run the entire test suite on your own machine

• How to run the test?

  To run the entire test suite on your own machine, use the following command:

  Maven:

  mvn clean test -P on-prem-suite

  Gradle:

  gradle clean on-prem-suite

• Output

  This run profile executes the entire test suite sequentially on a single browser, on your own machine.

---

BrowserStack

BrowserStack provides instant access to 2,000+ real mobile devices and browsers on a highly reliable cloud infrastructure that effortlessly scales as testing needs grow.

Prerequisites

• Create a new BrowserStack account or use an existing one.

• Identify your BrowserStack username and access key from the BrowserStack Automate Dashboard and export them as environment variables using the below commands.

  • For *nix based and Mac machines:

  export BROWSERSTACK_USERNAME=<browserstack-username> &&
  export BROWSERSTACK_ACCESS_KEY=<browserstack-access-key>

  • For Windows:

  set BROWSERSTACK_USERNAME=<browserstack-username>
  set BROWSERSTACK_ACCESS_KEY=<browserstack-access-key>

  Alternatively, you can also hardcode username and access_key objects in the browserstack.yml file.

Note:

• We have configured a list of test capabilities in the browserstack.yml file. You can certainly update them based on your device / browser test requirements.
• The exact test capability values can be easily identified using the Browserstack Capability Generator

Running Your Tests

Run a specific test on BrowserStack

In this section, we will run a single test on Chrome browser on Browserstack. To change test capabilities for this configuration, please refer to the bstack-single.yml file.

• How to run the test?

  • Set Environment Variable to pick desired YML file:

    • For *nix based and Mac machines:

   
   export BROWSERSTACK_CONFIG_FILE="src/test/resources/conf/capabilities/bstack-single.yml"

  • For Windows:

  set BROWSERSTACK_CONFIG_FILE="src\test\resources\conf\capabilities\bstack-single.yml"

  To run the default test scenario (e.g. End to End Scenario) on your own machine, use the following command:

  Maven:

  mvn clean test -P bstack-single

  Gradle:

  gradle clean bstack-single

  To run a specific test scenario, use the following command with the additional 'test-name' argument:

  Maven:

  mvn clean test -P bstack-single -Dtest=LoginDataDrivenTest

  Gradle:

  gradle clean bstack-single -Ptest-name=LoginDataDrivenTest

  where, the argument test or test-name can be any testclass implemented in this repository.

• Output

  This run profile executes a single test on a single browser on BrowserStack. Please refer to your BrowserStack dashboard for test results.

Run the entire test suite in parallel on a single BrowserStack browser

In this section, we will run the tests in parallel on a single browser on Browserstack. Refer to bstack-parallel.yml file to change test capabilities for this configuration.

• How to run the test?

  • Set Environment Variable to pick desired YML file:

    • For *nix based and Mac machines:

   
   export BROWSERSTACK_CONFIG_FILE="src/test/resources/conf/capabilities/bstack-parallel.yml"

  • For Windows:

  set BROWSERSTACK_CONFIG_FILE="src\test\resources\conf\capabilities\bstack-parallel.yml"

  To run the entire test suite in parallel on a single BrowserStack browser, use the following command:

  Maven:

  mvn clean test -P bstack-parallel

  Gradle:

  gradle clean bstack-parallel

• Output

  This run profile executes the entire test suite in parallel on a single BrowserStack browser. Please refer to your BrowserStack dashboard for test results.

Run the entire test suite in parallel on multiple BrowserStack browsers

In this section, we will run the tests in parallel on multiple browsers on Browserstack. Refer to the bstack-parallel-browsers.yml file to change test capabilities for this configuration.

• Set Environment Variable to pick desired YML file:

  • For *nix based and Mac machines:

   
   export BROWSERSTACK_CONFIG_FILE="src/test/resources/conf/capabilities/bstack-parallel-browsers.yml"

  • For Windows:

  set BROWSERSTACK_CONFIG_FILE="src\test\resources\conf\capabilities\bstack-parallel-browsers.yml"

• How to run the test?

  To run the entire test suite in parallel on multiple BrowserStack browsers, use the following command:

  Maven:

  mvn clean test -P bstack-parallel-browsers

  Gradle:

  gradle clean bstack-parallel-browsers

[Web application hosted on internal environment] Running your tests on BrowserStack using BrowserStackLocal

Prerequisites

• Clone the BrowserStack demo application repository.

  git clone https://github.com/browserstack/browserstack-demo-app

• Please follow the README.md on the BrowserStack demo application repository to install and start the dev server on localhost.

• In this section, we will run a single test case to test the BrowserStack Demo app hosted on your local machine i.e. localhost. Refer to the bstack-local.yml file to change test capabilities for this configuration.

• Note: You may need to provide additional BrowserStackLocal arguments to successfully connect your localhost environment with BrowserStack infrastructure. (e.g if you are behind firewalls, proxy or VPN).

• Further details for successfully creating a BrowserStackLocal connection can be found here:

  • Local Testing with Automate
  • BrowserStackLocal Java GitHub

[Web application hosted on internal environment] Run a specific test on BrowserStack using BrowserStackLocal

• How to run the test?

  • Set Environment Variable to pick desired YML file:

    • For *nix based and Mac machines:

   
   export BROWSERSTACK_CONFIG_FILE="src/test/resources/conf/capabilities/bstack-local.yml"

  • For Windows:

  set BROWSERSTACK_CONFIG_FILE="src\test\resources\conf\capabilities\bstack-local.yml"

  • To run the default test scenario (e.g. End to End Scenario) on a single BrowserStack browser using BrowserStackLocal, use the following command:

  Maven:

  mvn clean test -P bstack-local

  Gradle:

  gradle clean bstack-local

  To run a specific test scenario, use the following command with the additional test-name argument: Maven:

  mvn clean test -P bstack-local -Dtest=LoginDataDrivenTest

  Gradle:

  gradle clean bstack-local -Ptest-name=LoginDataDrivenTest

  where, the argument test or test-name can be any testclass implemented in this repository.

• Output

  This run profile executes a single test on an internally hosted web application on a single browser on BrowserStack. Please refer to your BrowserStack dashboard(https://automate.browserstack.com/) for test results.

[Web application hosted on internal environment] Run the entire test suite in parallel on a single BrowserStack browser using BrowserStackLocal

In this section, we will run the test cases to test the internally hosted website in parallel on a single browser on Browserstack. Refer to the bstack-local-parallel.yml file to change test capabilities for this configuration.

• How to run the test?

• Set Environment Variable to pick desired YML file:

  • For *nix based and Mac machines:

   
   export BROWSERSTACK_CONFIG_FILE="src/test/resources/conf/capabilities/bstack-local-parallel.yml"

  • For Windows:

  set BROWSERSTACK_CONFIG_FILE="src\test\resources\conf\capabilities\browserstack-local-parallel.yml"

To run the entire test suite in parallel on a single BrowserStack browser using BrowserStackLocal, use the following command: Maven:

mvn clean test -P bstack-local-parallel

Gradle:

gradle clean bstack-local-parallel

• Output

  This run profile executes the entire test suite on an internally hosted web application on a single browser on BrowserStack. Please refer to your BrowserStack dashboard for test results.

[Web application hosted on internal environment] Run the entire test suite in parallel on multiple BrowserStack browser using BrowserStackLocal

In this section, we will run the test cases to test the internally hosted website in parallel on multiple browsers on Browserstack. Refer to the bstack-local-parallel-browsers.yml file to change test capabilities for this configuration.

• How to run the test?

  • Set Environment Variable to pick desired YML file:

    • For *nix based and Mac machines:

   
   export BROWSERSTACK_CONFIG_FILE="src/test/resources/conf/capabilities/bstack-local-parallel-browsers.yml"

  • For Windows:

  set BROWSERSTACK_CONFIG_FILE="src\test\resources\conf\capabilities\browserstack-local-parallel-browsers.yml"

  To run the entire test suite in parallel on a single BrowserStack browser using BrowserStackLocal, use the following command:

  Maven:

  mvn clean test -P bstack-local-parallel-browsers

  Gradle:

  gradle clean bstack-local-parallel-browsers

• Output

  This run profile executes the entire test suite on an internally hosted web application on multiple browsers on BrowserStack. Please refer to your BrowserStack dashboard for test results.

Generating Allure Reports

• Generate Report using the following command:

  Maven:

  mvn allure:report

  Gradle:

  gradle allureReport

• Serve the Allure report on a server:

  Maven:

  mvn allure:serve

  Gradle:

  gradle allureServe

Percy

Percy provides a visual review platform.

Prerequisites

• Sign up with your BrowserStack account, or create a new BrowserStack account.
• Create a new Percy project.
• Go to the Project's Settings page, identify your Percy Token, and export them as environment variables using the following commands.

Installation

1. Install the Percy CLI.

npm install @percy/cli

Export the Percy Token

• For *nix based and Mac machines:

  export PERCY_TOKEN=[your-project-token]

• For Windows:

  set PERCY_TOKEN=[your-project-token]

Run your first Visual Test

In this section, we will run the test cases to detect the visual differences. We run the test first, then toggle the property that instructs our test to change some CSS.

1. Run the test. Maven:

npx percy exec -- mvn clean test -P percy

Gradle:

npx percy exec -- gradle clean percy

2. Set the changeCSS property in the src/test/java/com/browserstack/test/login/LoginVisuaalTest directory to true.

3. Run the test again.

4. View your Project in the Percy dashboard and verify the differences.

Run the Ignore Region Test

In this section, we will run the test that ignores a specific area of a webpage. In a real-world situation, you can think of dynamic content and advertisements where this use case applies.

1. Run the test. Maven:

npx percy exec -- mvn clean test -P percy-ignore

Gradle:

npx percy exec -- gradle clean percy-ignore

2. View your Project in the Percy dashboard and verify the differences.

Additional Resources

• View your test results on the BrowserStack Automate dashboard
• Documentation for writing Automate test scripts in Java
• Customizing your tests capabilities on BrowserStack using our test capability generator
• List of Browsers & mobile devices for automation testing on BrowserStack
• Using Automate REST API to access information about your tests via the command-line interface
• Understand how many parallel sessions you need by using our Parallel Test Calculator
• For testing public web applications behind IP restriction, Inbound IP Whitelisting can be enabled with the BrowserStack Enterprise offering