Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Best Practices to Kaspresso Wiki #162

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions wiki/00_Home.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ If you are looking forward of a wiki for old versions then follow links: <br>
[05. Device](./05_Device.md) <br>
[06. AdbServer](./06_AdbServer.md) <br>
[07. DocLoc](./07_DocLoc.md) <br>
[08. Best Practices](./08_BestPractices.md) <br>

## References

Expand Down
233 changes: 233 additions & 0 deletions wiki/08_BestPractices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
# Best Practices

## File structure

Here is shown an example structure of packages. It's not strong rule but we higly recommend to organize your files.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An example of the package structure is shown here. It's not a very strict rule but we strongly recommend organizing your files in such an order.


```
androidTest
└── java
└── com
├── actions
├── app_providers
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Использование нижних подчеркиваний в названиях пакетов противоречит
https://developer.android.com/kotlin/style-guide#package_names

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

да, соглашусь

├── common
├── functional
├── scenarios
├── screens
├── screenshot_tests
├── tests
└── views
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Нет описания, что это за пакет и какие там views хранятся

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

да, как-то упустили

```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Много элементарных грамматических ошибок. Я не буду проходиться по всем. Просто установи себе Grammarly.

1. **App_providers**
Put here all helper classes wich contains some business logic of main application. For example, if you need to use some Dagger component and check state of your interactor. **Don't use this interactors directly,** put accessing to interactors into method-wrappers witch returns some primitive or void for some simple actions and then call this methods in your test cases.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state of your interactor

О каком состоянии интерактора речь? Интерактор - это функция, состояние хранится в репозитории.

**Don't use this interactors directly

Не понял, что подразумевается по "прямым" и не "прямым" использованием. Давай добавим примеры.

**Don't use this interactors directly

Почему? К каким последствиям это может привести?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Да, стоит чуть подраскрыть. Человеку, который не в контексте, будет тяжело понять, что к чему.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wich

that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

contains

contain

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

of main application

of your application


2. **Common**
Here is collected all helper classes wich you can reuse in multiple test cases. We highly recommend to make it stateless (for example as kotlin Object). For example you can enable Accessibility with one row.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wich

that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We highly recommend to make it stateless (for example as kotlin Object)

абсолютно не понятна связь одного с другим


Some hint - if you want to call methods directly (without method `.run {}`) you can override operator `invoke`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

данная строчка и пример нужно выносить в отдельный пункт
в противном случае ты в начале как бы пишешь про одно, а раскрываешь другое. очень запутывает


```kotlin
object Accessibility {
fun BaseTestContext.enable() =
device.accessibility.enable(
YOUR_PACKAGE_NAME,
ACCESSIBILITY_CLASS_NAME
)

// this is optional
operator fun invoke(function: Accessibility.() -> Unit) {
return function.invoke(this)
}
}
```

```kotlin
...
step("some step") {
MainScreen {
Accessibility.run { enable() } // without invoke override
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tab vs space

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

да, поехало шо то

Accessibility { enable() } // with invoke override
}
}
....
```

3. **Functional**
This package needed for converting end-to-end tests into functional, for example it's may be working with DI, stub's and Advice. (Here is more about [Advice](#Advices))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Такие тесты обычно называют интеграционными. Почему functional?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Я согласен, что нужно переименовать, ибо функциональные тесты сами по себе немного про другое
@RuslanMingaliev как назовем?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Надо пообщаться с нашим QA, чтобы не было недопониманий

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This package needed for converting end-to-end tests into functional, for example it's may be working with DI, stub's and Advice.

This package is needed to convert end-to-end tests into functional tests. The package contains special classes to substitute real classes with stubs via DI. Also, you can notice using such classes as Advice. More detailed information is available below.

А вообще тут тоже стоит подраскрыть, я думаю.

4. **Scenarios**
There you can hold your mostly repeated steps wrapped into scenarios. In common it created with kotlin stateless objects. But you can hold some parameters for some logic and in this case create classic class with constructor parameters. We highly recommend to separate scenarios if it contains so many steps and logic. (Here is some more about [`scenario`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#test-structure-1))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it contains so many steps and logic

лучше давать количественную характеристику, а не качественную. Не понятно, 100 шагов норм или уже нет

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

английский тоже нужно переработать

5. **Screens**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

подраскрыть, привести пример хороший и плохой
и так везде бы

Here is located decriptions of screens. This classes must not contain business logic or states, only description of screen base operations or view checks. Current package also uses for keeping description of external classes described with uiAutomator dsl.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

decriptions

В моем понимании description - это строка или текст, который что-то описывает.

6. **Tests**
This package needs no introduction. Contains only test cases. All other classes moved to packages listed before.

## Screens

Here is some hints and info about screens in autotests.

### KScreen

You can extend base class KScreen<> with method `waitForScreen()`. This method used when screen may appear with delay. We recommend use [`flakySafely`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#flakysafely),with custom delay and intervals. Inside you can use `isVisible` method or even [`compose`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#compose) with several checks for difficult screens.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

без кода очень тяжело понять

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

английский


`layoutId` and `viewClass` used only for connection with source code. New developer (or maybe even you after some time) who will read your code can easily find real screen in your project by this fields.

For describing views use childs of KView (kakao views).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

не понял предложение


### KExternalScreen
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Чем не подходит UiScreen для таких случаев?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ну мы же специально разделяли UiScreen и немного проапгреденный KExternalScreen

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Первый раз услышал про KExternalScreen) Для чего разделяли и почему не подходит UiScreen? Давайте добавим в UiScreen недостающую функциональность


If you want write autotests with external screens (when your app use another app, like Chrome or Gmail) you can create class for external screens, for example
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want write autotests with external screens (when your app use another app, like Chrome or Gmail) you can create class for external screens, for example

Нужно написать, что бывают случаи, когда из тестов мы юзаем другие приложения, но и в этих случаях лучше придерживать паттерна PO и описывать каждый экран. Для описания экранов внешних приложений можно использовать KExternalScreen, который по своей сути очень походит на KScreen.
И вот в таком духе описывай.


```kotlin
abstract class KExternalScreen<out T : KExternalScreen<T>> {

abstract val packageName: String

open fun BaseTestContext.waitForScreen() = Unit

operator fun invoke(function: T.() -> Unit) {
function.invoke(this as T)
}

fun pressBack() {
val instrumentation: Instrumentation = InstrumentationRegistry.getInstrumentation()
UiDevice.getInstance(instrumentation).pressBack()
}
}
```

Or if you don't want to override `invoke` you may not use generics.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

какие дженерики??


Field `packageName` (like `layoutId` and `viewClass` from `KScreen`) uses for linking screen object to real application. Also it needed to remove code duplication (UiView needs packageName for describing views)

For describing views use childs of UiView ([uiAutomator DSL](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/02_Wrapper_over_UiAutomator.md)).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uiAutomator DSL

Kautomator


## Advices

If you want to separate end-to-end and functional test cases in single code of test we recommend to use interfaces and we named it `Advice` (but you can name it as you want). For example we want to pass registration on web-portal in our application. In e2e we must perform real registration but on functional we can stub it. So let's write some code:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Нужно вообще начинать с проблемы.
Прям бери с нашего с Мовчаном выступления.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Почему "advice"?)))


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ниже не стал проходиться сильно, так как основные моменты описаны уже выше.
Каждый пункт нужно начинать с истории, проблемы, продолжать примерами и четким разъяснением.
PR нужно еще серьезно дорабатывать, но начало положено.

```kotlin
interface RegistrationAdvice {
fun register()
fun passCaptcha() = Unit
}
```

This advice contains methods that will have different implementation for each type of test. Also we used stub for `passCaptcha` because in functional we don't need to pass captcha.

```kotlin
class RegistrationAdviceReal : RegistrationAdvice {
override fun register() {
// some real logic for register
}
override fun passCaptcha() {
// some real logic for captcha
}
}

class RegistrationAdviceFake : RegistrationAdvice {
override fun register() {
// some fake logic for register
}
// note that we don't needed to override passCaptcha()
}
```

So in test case we can write wrapper-methods for our test cases and run needed via marathon and filter by annotations
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Такой фильтр работает не только для Marathon-a. Думаю, здесь стоит написать про стандартный Instrumentation Runner


```kotlin
class RegistrationTestCase : TestCase() {
@Test
@E2e // marker annotation for marathon runner
fun registrationTestCase_e2e() =
registrationTestCase(RegistrationAdviceReal())

@Test
@Functional // marker annotation for marathon runner
fun registrationTestCase_functional() =
registrationTestCase(RegistrationAdviceFake())

private fun registrationTestCase(testAdvice: RegistrationAdvice) {
run {
...
step("pass registration") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Форматирование?

...
testAdvice {
register()
passCaptcha()
}
...
```

Sometimes your e2e test must do some work and functional test must not do it at all. In that case you create `interface A` with empty default implementations of all methods, manually override all methods in real advice `class B : A` and for functional advice create empty `class C : A`. (examples in kotlin syntax)

## Test cases

1. First rule of writing best test cases: inheritors of `TestCase` class must be exactly test cases or class wich overrides **must be applied to all test cases**. If you create multiple parent classes you may get pain with modifying and ordering classes or your new cases may do some work wich them must not to do.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

best test cases

Тут ни к чему качественная метрика.

First rule of writing best test cases: inheritors of TestCase class must be exactly test cases or class wich overrides must be applied to all test cases.

Не понял, что написано в этом предложении


But if you still needed to add some actions to group of cases you can create lamda-function with `Kaspresso.Builder` (here is more about [`Kaspresso.Bulder`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/03_Kaspresso_configurator.md)) receiver and pass it as class parameter. You still can use `before` and `after` sections in test case but also you will have `beforeEachTest` and `afterEachTest`. Example:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add some actions to group of cases

Если речь только про beforeEachTest и afterEachTest, то это не some actions, а конкретные actions. Нельзя добавить шаги куда-то в середину. Кажется, мы тут вводим в заблуждение.


```kotlin
object SomeTestBuilder {
fun create(): Kaspresso.Builder.() -> Unit = {
beforeEachTest {
// some work
// for example: install some external app via adb
}

afterEachTest {
// some work
}
}
}
```

```kotlin
class MySomeTestCase : TestCase(
kaspressoBuilderAdditional =
SomeTestBuilder.create()
) {
```

2. Separate your test onto steps and shortly name them. You will have readable logs and easily realise where your test failed
3. Separate your screens. For example: move your dialog into separate `Screen` object. It will be easy to read your code
4. If some code used **only** in single test-case - leave it code as private methods inside your test class. But if your code duplicates two or more times - move it into some stateless object in `common` package.

## Other advices

1. Don't be afraid to read source code of called methods. For example you can discover that method `androidx.test.uiautomator.UiDevice.pressBack()` calling `waitForIdle()` before perfoming click. This waiting by default keep 10 seconds timeout and may not execute if screen still drawing some changes.
And if you go deeper in code researching you can find configurator of uiAutomator and change default timeout of `waitForIdle()`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Технология называется UI Automator

2. Try to avoid calling `Thread.sleep()`, use instead [`flakySafely`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#flakysafely), [`compose`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#compose),[`continuosly`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#continuously). If waiting is really needed and construction above don't help you - better use this snippet:

```kotlin
/**
* More efficient simple waiting then Thread.sleep
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More efficient

Стоит указать, чем эффективнее

*/
fun simpleWait(timeoutMs: Long) {
val lock = ReentrantLock()
val condition = lock.newCondition()
lock.withLock {
Timer().schedule(timeoutMs) {
lock.withLock {
condition.signalAll()
}
}
condition.await()
}
}
```

3. There is difference of `device.targetContext` and `device.context`. First one used if you need context of **application witch will be tested** (host application) and second one if you need context of **application wich testing your main application** (slave application).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Откуда терминология host application и slave application?


4. [`flakySafely`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#flakysafely) and [`continuosly`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#continuously) have defailt parameters but you still can override on each use. You may know `timeoutMs` and `intervalMs` but there is `allowedExceptions` and you override list of allowed exceptions or take list of exceptions from configurator and modify it. May be helpful with some flaking of adbServer for example:

```kotlin
flakySafely(
timeoutMs = 1.min,
allowedExceptions = (params.flakySafetyParams.allowedExceptions + AdbServerException::class.java).toMutableSet()
) {
adbServer.performShell("settings put global heads_up_notifications_enabled 0")
}
```

5. Use `assertSafely` constructions. From naming you can realise that it's [`flakySafely`](https://github.com/KasperskyLab/Kaspresso/blob/master/wiki/04_How_to_write_autotests.md#flakysafely), for assertions and used when your assert may be appear with some delay. Also you can set custom message of error.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Что за конструкция assertSafely? Неплохо было бы увидеть пример. Из описания не понятно, что это и чем отличается от flakySafely