Skip to content

Commit

Permalink
docs(split): Cloud documentation was moved to a separate tab. (#856)
Browse files Browse the repository at this point in the history
* Cloud moved to a separate tab. Added Enterprise button

Co-authored-by: Anton Malinskiy <[email protected]>
  • Loading branch information
austyh and Malinskiy authored Nov 14, 2023
1 parent 6cff8e2 commit 723bd40
Show file tree
Hide file tree
Showing 133 changed files with 2,315 additions and 1,650 deletions.
28 changes: 28 additions & 0 deletions docs/cloud/cli/cicd.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: "CI/CD"
---

###
Marathon Cloud offers several ways to run tests within your CI/CD platform:
1. Platform-Specific Steps
2. Docker Image
3. Marathon CLI

### Platform steps
We provide shared steps for the following platforms:
- [Github Action](https://github.com/MarathonLabs/action-test)
- [CircleCI Orb](https://circleci.com/developer/orbs/orb/marathonlabs/marathon-cloud-orb)
- [Bitrise](https://bitrise.io/integrations/steps/run-tests-using-marathon-cloud)

### Docker image
Some platforms allow usage of [Marathon Docker Image](https://hub.docker.com/r/marathonlabs/marathon-cloud)
for each step. We recommend using it for Gitlab CI and Jenkins environments.

### Marathon CLI
If the previously mentioned solutions are not applicable,
you have the alternative of installing the [Marathon CLI](./installation) and executing tests using it.

### Other
If you have problems using all of the previous solutions feel free to [contact us](email:[email protected]).


19 changes: 19 additions & 0 deletions docs/cloud/cli/installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Installation"
---

###
This section of the documentation explains how to install the Marathon CLI on your local machine.
For running tests within your Continuous Integration and Delivery (CI/CD) system, please refer to the [CI/CD section](./cicd).
The installation can be performed using [Homebrew](https://brew.sh/). Here’s how to add the MarathonLabs repository:
```shell
brew tap malinskiy/tap
```
Next, install the Marathon Cloud CLI:

```shell
brew install malinskiy/tap/marathon-cloud
```
Alternatively, you can download prebuilt binaries for Windows, Linux,
or MacOS from [the Release page](https://github.com/MarathonLabs/marathon-cloud-cli/releases).

36 changes: 36 additions & 0 deletions docs/cloud/cli/parameters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
title: "Parameters"
---

###
You can find all the available marathon-cli parameters by running the command `marathon-cloud --help.`
Below, you'll find a list of the parameters you can set.

```bash
marathon-cloud --help
-app string
application filepath. Required
android example: /home/user/workspace/sample.apk
ios example: /home/user/workspace/sample.zip
-testapp string
test apk file path. Required
android example: /home/user/workspace/testSample.apk
ios example: /home/user/workspace/sampleUITests-Runner.zip
-platform string
testing platform. Required
possible values: "Android" or "iOS"
-api-key string
api-key for client. Required
-os-version string
Android or iOS OS version
-link string
link to commit
-name string
name for run, for example it could be description of commit
-o string
allure raw results output folder
-system-image string
OS-specific system image. For Android one of [default,google_apis]. For iOS only [default]
-isolated bool
Run each test using isolated execution. Default is false.
```
File renamed without changes.
File renamed without changes.
79 changes: 79 additions & 0 deletions docs/cloud/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Quick start

### Install
The installation can be performed using [Homebrew](https://brew.sh/). Here’s how to add the MarathonLabs repository:
```shell
brew tap malinskiy/tap
```
Next, install the Marathon Cloud CLI:

```shell
brew install malinskiy/tap/marathon-cloud
```
Alternatively, you can download prebuild binaries for Windows, Linux, or MacOS from [the Release page](https://github.com/MarathonLabs/marathon-cloud-cli/releases).


### API Key

Token creation and management are available at the [Tokens page](https://cloud.marathonlabs.io/tokens). Generate a token and save it somewhere safe for the next step.

### Samples (optional)

To showcase the advantages of Marathon Cloud compared to other solutions, we’ve prepared a sample app with 300 tests, out of which 15% are flaky. During the initial run, our platform will gather information about the tests. During the second run, it will optimize it to fit within 15 minutes.

<Tabs groupId="operating-systems" >
<TabItem value="iOS" label="iOS" className="tab-content-with-text">
Download prebuild iOS Application

```shell
curl https://cloud.marathonlabs.io/samples/ios/sample.zip -o sample.zip
```
Download prebuild iOS Test Application

```shell
curl https://cloud.marathonlabs.io/samples/ios/sampleUITests-Runner.zip -o sampleUITests-Runner.zip
```

</TabItem>
<TabItem value="Android" label="Android" className="tab-content-with-text" >
Download prebuild Android Application

```shell
curl https://cloud.marathonlabs.io/samples/android/app.apk -o app.apk
```

Download prebuild Android Test Application
```shell
curl https://cloud.marathonlabs.io/samples/android/appTest.apk -o appTest.apk
```

</TabItem>
</Tabs>

To use your own applications please read Documentation.


### Execution

Now you can start running your tests. Use the following command to execute the CLI with the necessary parameters:

<Tabs groupId="operating-systems">
<TabItem value="iOS" label="iOS">

```shell
marathon-cloud -api_key generated_api_key -apk sample.zip -testapk sampleUITests-Runner.zip -platform iOS
```
</TabItem>
<TabItem value="Android" label="Android">

```shell
marathon-cloud -api_key api_key -apk app.apk -testapk appTest.apk -platform Android
```
</TabItem>
</Tabs>

For additional parameters, refer to the [marathon-cloud-cli README](https://github.com/MarathonLabs/marathon-cloud-cli/#installation)

38 changes: 38 additions & 0 deletions docs/cloud/intro/android.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
title: "Android"
---

### Supported frameworks
Marathon Cloud supports tests written with:
- UIAutomator
- Cucumber
- Espresso
- [Kaspresso](https://github.com/KasperskyLab/Kaspresso)
- Flutter (with [Patrol](https://patrol.leancode.co/))

### Application and Test Application

Before initiating the testing process for your application, you’ll require two APK files:
one for the application that’s being tested, and another for the tests themselves.
Typically, debug variants are utilized for this purpose.

If the primary application resides under the app/ subproject,
you can execute the following command to build both the app and test APK:

```shell
# file structure
# |
# |--home
# |--john
# |--project <== you are here
# |--app <== it's your primary application
# ...
# |--build.gragle
# |--settings.gradle
./gradlew :app:assembleDebug :app:assembleDebugAndroidTest
```
Be sure to note the relative paths of the test APK and the app APK, as they will be required for running the tests.
In the context of our example, involving the `app` project and the `debug` build, these files can be located at the following paths:

- Application APK: `/home/john/project/app/build/outputs/apk/debug/app-debug.apk`
- Test Application APK: `/home/john/project/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk`
58 changes: 58 additions & 0 deletions docs/cloud/intro/ios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
title: "iOS"
---

### Supported frameworks
Marathon Cloud supports tests written with **XCTest and XCUITest frameworks**.
Both the application and the tests must be built for the **ARM architecture**.

### Application and Test Application

Before initiating the testing process for your iOS application, you’ll need to create two `.app` bundles: one for the application that's being tested, and another for the tests themselves. Typically, `debug` variants are utilized for this purpose.
Let's say our project is called "Sample". The code snippet below shows how to build the .app bundle:

```shell
# file structure
# |
# |--home
# |--john
# |--sample <== you are here
# |--sample <== it's your application
# ...
# |--sample.xcodeproj

xcodebuild build-for-testing \
-project sample.xcodeproj \
-scheme sample \
-destination 'platform=iOS Simulator,name=iPhone 14,OS=16.1' \
-derivedDataPath ./build
```

Note the relative paths of applications, as they will be required for running the tests. In the context of our example and `debug` build, these files can be located at the following paths:

- Application: `/home/john/sample/build/Build/Products/Debug-iphonesimulator/sample.app`
- Test APK: `/home/john/sample/build/Build/Products/Debug-iphonesimulator/sampleUITests-Runner.app`

One important thing to note is that `*.app` files are actually folders in disguise. To transfer them, it's necessary to convert these bundles into `.ipa` format or standard `zip` archives:

```shell
# file structure
# |
# |--home
# |--john
# |--sample <== you are here
# |--build <== derivedData folder
# |--sample <== it's your application
# ...
# |--sample.xcodeproj
cd build/Build/Products/Debug-iphonesimulator
# convert to zip archive in this example
zip -r sample.zip sample.app
zip -r sampleUITests-Runner.zip sampleUITests-runner.app
```

Further, we will use these files:

- Application: `/home/john/sample/build/Build/Products/Debug-iphonesimulator/sample.zip`
- Test APK: `/home/john/sample/build/Build/Products/Debug-iphonesimulator/sampleUITests-Runner.zip`

45 changes: 45 additions & 0 deletions docs/cloud/intro/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: "Overview"
---

###
[Marathon Cloud](https://cloud.marathonlabs.io) is a cloud testing infrastructure built on top of the Marathon test runner.
It automatically provisions virtual devices to accommodate your tests within 15 minutes.
The test execution is then delegated to Marathon test runner, which handles tasks such as batching, sorting, preventive retries, and post-factum retries.
This ensures an even distribution of tests across the provisioned devices.

### How it works
Whenever you submit an application and test application to Marathon Cloud, the following steps are taken:
- Calculation of the necessary number of devices
- Provisioning of the devices
- Distribution and execution of the tests
- Control of the tests and real-time load balancing
- Generation of reports
- Uploading of artifacts and the generated report


### Device provisioning

When you run tests with Marathon Cloud for the first time, we begin storing the test history in our database.
The next time you run these tests, we already have information on the average time and the probability of a successful execution for each test.
Using this data, we calculate the necessary number of devices to ensure that your tests will complete within 15 minutes.
We also monitor the progress of test executions and adjust the distribution of tests across devices as needed.
While the tests are running, our service can dynamically increase the number of devices to expedite the execution process.

### Batching

Balancing speed and stability is one of the primary challenges for Marathon Cloud.
In order to maintain fast test execution, we employ a batching strategy where we group 5 tests together in a single batch.
This approach involves executing 5 tests consecutively, and afterward, we reset the device to a clean state.
If you prefer, you can enable the "isolated" parameter to manage device cleaning yourself,
but please note that this may lead to an increase in the number of devices required and the overall time of devices taken for testing.
However, the total execution time will still be 15 minutes.









Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
title: "Pricing"
---

###
We operate on a Pay-As-You-Go model, meaning you only pay for the actual time spent running tests on our virtual devices. The pricing is set at a straightforward rate of $2 per hour for each virtual device.

## Billing
### Billing
To better understand this, picture having a test suite that requires 2 hours and 15 minutes (or 135 minutes) to run on one device. With Marathon Cloud, we automatically deploy an infrastructure of 9 virtual devices for you (135 minutes / 9 devices = 15 minutes). We then evenly distribute the tests among these devices, run them, and deliver the results to you within just 15 minutes. However, your tests will have used up 2 hours and 15 minutes (or 2.25 hours) of our device time in total. This means the cost of this test run for you would be $4.5.

# Calculating Time per Device
### Calculating Time per Device

The computation of time for each device during a test run is based on the time span from the beginning of the first test on that device to the end of the last test on the same device. This means we don't factor in the device booting process or the application installation stage. We do, however, take into account the brief pauses between tests and the periodic device clean-ups that occur when using a batching strategy. Typically, pauses between tests take only fractions of a second, but clean-ups can require up to 6-8 seconds.
51 changes: 51 additions & 0 deletions docs/cloud/sidebars.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/**
* Creating a sidebar enables you to:
- create an ordered group of docs
- render a sidebar for each doc of that group
- provide next/previous navigation
The sidebars can be generated from the filesystem, or explicitly defined here.
Create as many sidebars as you want.
*/

// @ts-check

/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
const sidebars = {
docs: [
{
type: 'category',
label: 'Getting started',
collapsed: false,
items: [
'index',
'intro/overview',
'intro/android',
'intro/ios',
]
},
{
type: 'category',
label: 'CLI',
collapsed: false,
items: [
'cli/installation',
'cli/parameters',
'cli/cicd'
]

},
{
type: 'category',
label: 'Misc',
collapsed: false,
items: [
'misc/pricing'
]
},

],
};

module.exports = sidebars;
Loading

0 comments on commit 723bd40

Please sign in to comment.