Performance test customization in the Hydra Lab

Performance configurations in a test task

Performance strategy type

Name	Desc
TEST_SCHEDULE	Inspect the performance data every fixed time
TEST_LIFECYCLE	Inspect the performance data at the life cycle of the test run (Such as test run start, test run finish...)

Performance inspector type

The performance inspector type defines what performance data is going to be inspected.

Name	Desc
INSPECTOR_ANDROID_MEMORY_INFO	Inspect the memory usage for target Android app
INSPECTOR_ANDROID_BATTERY_INFO	Inspect the battery usage for target Android app
INSPECTOR_WIN_MEMORY	Inspect the memory usage for target Windows app
INSPECTOR_WIN_BATTERY	Inspect the battery usage for target Windows app
INSPECTOR_IOS_ENERGY	Inspect the energy usage for target iOS app
INSPECTOR_IOS_MEMORY	Inspect the memory usage for target iOS app

Trigger a test task with performance test from ADO pipeline

For how to trigger a test task run from ADO pipeline, please refer to Trigger a test task from ADO pipeline

1. Create a new YAML file based on testSpec.yml and add the performance configuration to the file.

For example:

testSpec:
  inspectionStrategies:
    # Inspect the battery usage of the Android app every 5 seconds
    - strategyType: TEST_SCHEDULE
      interval: 5000
      intervalUnit: MILLISECONDS
      inspection:
        inspectorType: INSPECTOR_ANDROID_BATTERY_INFO
        # App package name to be tested
        appId: com.contoso.exampleapp
        description: "Inspect battery on Android"
    # Inspect the memory usage of the Android app at test started, test failed, test success, test run started and test run finished
    - strategyType: TEST_LIFECYCLE
      when:
        - TEST_STARTED
        - TEST_FAILURE
        - TEST_SUCCESS
        - TEST_RUN_STARTED
        - TEST_RUN_FINISHED
      inspection:
        inspectorType: INSPECTOR_ANDROID_MEMORY_INFO
        appId: com.contoso.exampleapp
        description: "Inspect memory on Android"
    # Inspect the memory usage of the Windows app
    - strategyType: TEST_SCHEDULE
      interval: 5000
      intervalUnit: MILLISECONDS
      inspection:
        inspectorType: INSPECTOR_WIN_MEMORY
        appId: com.contoso.exampleapp
        description: "Inspect memory on Windows"

2. Run Gradle task requestHydraLabTest implemented in the Gradle plugin with performance test configurations

- task: Gradle@2
   displayName: Trigger Hydra Lab UI test
   inputs:
     gradleWrapperFile: 'UIAutomation/gradlew'
     workingDirectory: 'UIAutomation'
     # Performance configurations can be found in testSpec.yml
     tasks: 'requestHydraLabTest -PymlConfigFile=testSpec.yml -PappPath=$(build.artifactstagingdirectory) -PtestAppPath=$(build.artifactstagingdirectory)\testApk -PtestSuiteName=$(testSuiteName) -PqueueTimeOutSeconds=$(testQueueTimeOut) -PrunTimeOutSeconds=$(testRunTimeOut) -PdeviceIdentifier=$(Android.Device.Identifier) -PgroupTestType=$(groupTestType) -PextraArgs="type=$(Build.Reason),buildId=$(Build.BuildId)"  -PauthToken=$(authToken) -PpkgName=$(pkgName) -PtestPkgName=$(testPkgName)'
     publishJUnitResults: false

Performance test results

View performance test report

View performance test history report

Performance raw result files

The raw result files can be download via the artificats of the ADO pipeline

FAQ

Q: How to resolve error "Out-File: Could not find a part of the path 'xxx'" with no performance results files generated?

A: This error occurs when the length of the perforamce file path exceeds MAX_PATH, which is defined as 260 characters in Windows API. To resolve this, please follow the Maximum Path Length Limitation to enable the long paths in Windows