ANDROID-13796 Roborazzi screenshot tests (#296)

* POC roborazzi screenshot tests * POC roborazzi set screenshot folder * POC roborazzi using getScreenshotName() to get the screenshot file name * POC roborazzi add compare_screenshots.yml * POC roborazzi updating failure report * POC roborazzi buttons test * POC roborazzi update screenshots baseline * POC roborazzi execute update_screenshot_baseline.yml on push * Updated screenshots baseline * POC roborazzi update record screenshots baseline * POC roborazzi runs on ubuntu-latest * Updated screenshots baseline * POC roborazzi remove on push * POC roborazzi upload to azure * POC roborazzi upload to azure if failure * POC roborazzi updated upload to azure * POC roborazzi updated upload to azure * POC roborazzi force screenshots error * POC roborazzi add pngs of other directories * POC roborazzi add pngs of other directories * POC roborazzi revert brand names * POC roborazzi update baseline * POC roborazzi update baseline * POC roborazzi update baseline * POC roborazzi using Enclosed to run parametrized and not parametrized tests * POC roborazzi update screenshots * POC roborazzi update screenshots * POC roborazzi update globs * POC roborazzi also upload artifact * POC roborazzi update job name * POC roborazzi fix uppercase * POC roborazzi upload names * POC roborazzi use pixel 5 * POC roborazzi update baseline on push * Updated screenshots baseline * POC roborazzi update device * POC roborazzi force error * POC roborazzi update device * POC roborazzi force error * POC roborazzi pixel 5 + 33% image size * POC roborazzi add error on PasswordInput * POC roborazzi change report image dimensions * POC roborazzi add image click * POC roborazzi update compare_screenshots.yml AZURE_ACCOUNT_NAME * POC roborazzi update setting GraphicsMode from build.gradle file * POC roborazzi classic views test * POC roborazzi classic views in xmls * POC roborazzi remove clicked from screenshot tests * POC roborazzi update to 1.5.0 * POC roborazzi add screenshots.md file * POC roborazzi fix url * POC roborazzi update screenshots.md * POC roborazzi update test_dummy_activity.xml to the test package * POC roborazzi remove upload to github * POC roborazzi update screenshots.md * POC roborazzi extract roborazzi_version * POC roborazzi update screenshots.md removing github artifacts from there * POC roborazzi update removing checks that are also done with the screenshots * POC roborazzi split ButtonKtTest * POC roborazzi remove git status * POC roborazzi add darkTheme screenshots * ANDROID-13796 add ScreenshotsTest base class * ANDROID-13796 use ScreenshotsTest in classic view tests * ANDROID-13796 update screenshots * ANDROID-13796 add compareScreenshot methods in ScreenshotsTest * ANDROID-13796 downgrade roborazzi to 1.4.0 * ANDROID-13796 restore checkout * ANDROID-13796 downgrade roborazzi to 1.4.0 * ANDROID-13796 add generate-html-report action * ANDROID-13796 add gradle/gradle-build-action * ANDROID-13796 removing the layout tests from library * ANDROID-13796 add TextInputWithError test * ANDROID-13796 add TextInputDisabled test * ANDROID-13796 remove unneeded code * ANDROID-13796 configure sdk version * ANDROID-13796 extract repeated code to private method * ANDROID-13796 add dark mode tests for classic views --------- Co-authored-by: jeprubio <[email protected]>
Telefonica · Oct 16, 2023 · b043c2f · b043c2f
1 parent 57c66f0
commit b043c2f
Show file tree

Hide file tree

Showing 37 changed files with 605 additions and 1 deletion.
diff --git a/.github/actions/generate-html-report/action.yml b/.github/actions/generate-html-report/action.yml
@@ -0,0 +1,7 @@
+name: 'Generate HTML Report'
+description: 'Generates HTML report from screenshots'
+runs:
+  using: "composite"
+  steps:
+    - run: ${GITHUB_ACTION_PATH}/scripts/generate-html-report.sh
+      shell: bash
diff --git a/.github/actions/generate-html-report/scripts/generate-html-report.sh b/.github/actions/generate-html-report/scripts/generate-html-report.sh
@@ -0,0 +1,59 @@
+#!/bin/bash
+
+mkdir reports
+touch reports/report.html
+cp */screenshots/*_compare.png reports/
+files=$(find . -type f -name "*_compare.png" | grep "reports/")
+{
+  echo '<!doctype html>'
+  echo '<html>'
+    echo '<head>'
+      echo '<title>Screenshots failure report</title>'
+      echo '<link href="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.100.2/css/materialize.min.css" rel="stylesheet">'
+      echo '<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">'
+      echo '<meta charset="UTF-8">'
+    echo '</head>'
+    echo '<style>'
+    echo 'body {'
+      echo 'display: flex;'
+      echo 'min-height: 100vh;'
+      echo 'flex-direction: column;'
+    echo '}'
+    echo 'main {'
+      echo 'flex: 1 0 auto;'
+    echo '}'
+    echo '</style>'
+  echo '<body>'
+  echo '<nav>'
+    echo '<div class="nav-wrapper indigo darken-3">'
+      echo '<a href="#" class="brand-logo left">Screenshots failure report</a>'
+      echo '<ul id="nav-mobile" class="right hide-on-med-and-down">'
+        echo '<li><a href="https://github.com/takahirom/roborazzi">Roborazzi</a></li>'
+      echo '</ul>'
+    echo '</div>'
+  echo '</nav>'
+  echo '<main class="container">'
+    echo '<div class="section">'
+    echo '<table class="highlight responsive-table">'
+      echo '<tr><th>File name</th><th>Comparison</th></tr>'
+} >> reports/report.html
+
+      for file in $files; do
+        # Get the file name and insert newlines every 100 characters
+        fileName=$(basename "$file" | sed -r 's/(.{100})/\1<br>/g')
+        echo "<tr><td>$(basename "$file")</td>" >> reports/report.html
+        echo "<td><a href=\"$(basename "$file")\"><img src=\"$(basename "$file")\" width=\"100%\" height=\"100%\" /></a></td></tr>" >> reports/report.html
+      done
+{
+      echo '</table>'
+    echo '</div>'
+  echo '</main>'
+  echo '<footer class="page-footer indigo darken-3">'
+    echo '<div></div>'
+  echo '</footer>'
+  echo '<script src="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.100.2/js/materialize.min.js"></script>'
+  echo '</body></html>'
+} >> reports/report.html
+
+echo "Report: "
+cat reports/report.html
diff --git a/.github/workflows/compare_screenshots.yml b/.github/workflows/compare_screenshots.yml
@@ -0,0 +1,49 @@
+name: Compare Screenshots
+
+on:
+  workflow_dispatch:
+  pull_request:
+
+jobs:
+  CompareScreenshots:
+
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v3
+
+      - name: Setup Gradle
+        uses: gradle/gradle-build-action@v2
+
+      - name: Verify Screenshots (roborazzi)
+        run: 'bash ./gradlew verifyRoborazziDebug'
+
+      - id: generate-html
+        name: Generate Html Report
+        if: failure()
+        uses: ./.github/actions/generate-html-report
+
+      - name: Generate screenshots tests reports tar.gz
+        if: failure()
+        run: |
+          tar cvzf mistica-screenshots-tests-report.tar.gz reports || echo "No screenshots tests reports found"
+        shell: bash
+
+      - name: Checkout Telefonica/github-actions repo
+        if: failure()
+        uses: actions/checkout@v3
+        with:
+          repository: Telefonica/github-actions
+          token: "${{ secrets.NOVUM_PRIVATE_REPOS }}"
+          path: .github/shared-actions
+
+      - name: Upload reports to azure
+        if: failure()
+        uses: ./.github/shared-actions/azure/upload-to-storage
+        with:
+          azure-account-name: ${{secrets.AZURE_ACCOUNT_NAME}}
+          azure-account-key: ${{secrets.AZURE_ACCOUNT_KEY}}
+          globs: |
+            mistica-screenshots-tests-report.tar.gz
+          generate-summary: true
diff --git a/.github/workflows/update_screenshot_baseline.yml b/.github/workflows/update_screenshot_baseline.yml
@@ -0,0 +1,24 @@
+name: "Update screenshot baseline"
+on:
+  workflow_dispatch:
+
+jobs:
+  screenshots_baseline:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v2
+
+      - name: Run Roborazzi Record
+        run: 'bash ./gradlew clean recordRoborazziDebug'
+
+      - name: Check Git status
+        run: 'git status'
+        shell: bash
+
+      - name: Commit and push screenshots baseline
+        id: commitAndPushScreenshotsBaseline
+        uses: EndBug/add-and-commit@v7
+        with:
+          message: 'Updated screenshots baseline'
+          add: './**/screenshots/*'
diff --git a/build.gradle b/build.gradle
@@ -10,6 +10,7 @@ buildscript {
         accompanist_version = "0.30.1"
         coil_version = '2.2.2'
         constraintComposeVersion = '1.0.1'
+        roborazzi_version = "1.4.0"
     }
     repositories {
         google()
@@ -27,6 +28,7 @@ buildscript {
 plugins {
     id 'org.jetbrains.kotlin.android' version '1.5.21' apply false
     id 'io.github.gradle-nexus.publish-plugin' version '1.1.0' apply false
+    id "io.github.takahirom.roborazzi" version '1.4.0' apply false
 }
 
 allprojects {

diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
@@ -1,6 +1,6 @@
 #Fri May 19 13:07:06 CEST 2023
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-7.5-bin.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.2-bin.zip
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists
diff --git a/library/build.gradle b/library/build.gradle
@@ -3,6 +3,7 @@ plugins {
     id 'org.jetbrains.kotlin.android'
     id 'kotlin-kapt'
     id 'maven-publish'
+    id 'io.github.takahirom.roborazzi'
 }
 
 android {
@@ -51,6 +52,15 @@ android {
         sarifReport true
         checkDependencies true
     }
+
+    testOptions {
+        unitTests {
+            includeAndroidResources = true
+            all {
+                systemProperty 'robolectric.graphicsMode', 'NATIVE'
+            }
+        }
+    }
 }
 
 task sourceJar(type: Jar) {
@@ -88,7 +98,15 @@ dependencies {
 
     testImplementation 'junit:junit:4.13.2'
     testImplementation "androidx.compose.ui:ui-test-junit4:$compose_ui_version"
+    testImplementation 'androidx.compose.ui:ui-test-manifest:1.0.5'
+    testImplementation 'org.robolectric:robolectric:4.10.3'
+    testImplementation "io.github.takahirom.roborazzi:roborazzi:$roborazzi_version"
+    testImplementation "io.github.takahirom.roborazzi:roborazzi-compose:$roborazzi_version"
     testImplementation "org.mockito.kotlin:mockito-kotlin:4.0.0"
+    testImplementation "androidx.test:rules:1.5.0"
+    testImplementation "androidx.test:core-ktx:1.5.0"
+    testImplementation 'androidx.test.ext:junit-ktx:1.1.5'
+    testImplementation 'androidx.test.espresso:espresso-core:3.2.0'
 
     debugImplementation "androidx.compose.ui:ui-tooling:$compose_ui_version"
 }

diff --git a/library/screenshots/check_TextInput.png b/library/screenshots/check_TextInput.png
diff --git a/library/screenshots/check_TextInputDisabled.png b/library/screenshots/check_TextInputDisabled.png
diff --git a/library/screenshots/check_TextInputDisabled_dark.png b/library/screenshots/check_TextInputDisabled_dark.png
diff --git a/library/screenshots/check_TextInputWithError.png b/library/screenshots/check_TextInputWithError.png
diff --git a/library/screenshots/check_TextInputWithError_dark.png b/library/screenshots/check_TextInputWithError_dark.png
diff --git a/library/screenshots/check_TextInput_dark.png b/library/screenshots/check_TextInput_dark.png
diff --git a/library/screenshots/check_the_button_screenshot_BlauBrand.png b/library/screenshots/check_the_button_screenshot_BlauBrand.png
diff --git a/library/screenshots/check_the_button_screenshot_BlauBrand_dark.png b/library/screenshots/check_the_button_screenshot_BlauBrand_dark.png
diff --git a/library/screenshots/check_the_button_screenshot_MovistarBrand.png b/library/screenshots/check_the_button_screenshot_MovistarBrand.png
diff --git a/library/screenshots/check_the_button_screenshot_MovistarBrand_dark.png b/library/screenshots/check_the_button_screenshot_MovistarBrand_dark.png
diff --git a/library/screenshots/check_the_button_screenshot_O2Brand.png b/library/screenshots/check_the_button_screenshot_O2Brand.png
diff --git a/library/screenshots/check_the_button_screenshot_O2Brand_dark.png b/library/screenshots/check_the_button_screenshot_O2Brand_dark.png
diff --git a/library/screenshots/check_the_button_screenshot_TelefonicaBrand.png b/library/screenshots/check_the_button_screenshot_TelefonicaBrand.png
diff --git a/library/screenshots/check_the_button_screenshot_TelefonicaBrand_dark.png b/library/screenshots/check_the_button_screenshot_TelefonicaBrand_dark.png
diff --git a/library/screenshots/check_the_button_screenshot_VivoBrand.png b/library/screenshots/check_the_button_screenshot_VivoBrand.png
diff --git a/library/screenshots/check_the_button_screenshot_VivoBrand_dark.png b/library/screenshots/check_the_button_screenshot_VivoBrand_dark.png
diff --git a/library/screenshots/check_the_password_is_initially_not_visible.png b/library/screenshots/check_the_password_is_initially_not_visible.png
diff --git a/...heck_the_password_is_not_visible_after_toggling_twice_the_visibility_button.png b/...heck_the_password_is_not_visible_after_toggling_twice_the_visibility_button.png
diff --git a/...nshots/check_the_password_is_visible_after_the_visibility_button_is_clicked.png b/...nshots/check_the_password_is_visible_after_the_visibility_button_is_clicked.png
diff --git a/library/src/test/AndroidManifest.xml b/library/src/test/AndroidManifest.xml
@@ -0,0 +1,20 @@
+<manifest xmlns:android="http://schemas.android.com/apk/res/android"
+		package="com.telefonica.mistica"
+		>
+
+	<uses-permission android:name="android.permission.VIBRATE" />
+
+	<application>
+		<activity
+				android:name=".DummyActivity"
+				android:label="Text Input Activity"
+				android:exported="true"
+				android:theme="@style/MisticaTheme.Movistar">
+
+			<intent-filter>
+				<action android:name="android.intent.action.MAIN" />
+				<category android:name="android.intent.category.LAUNCHER" />
+			</intent-filter>
+		</activity>
+	</application>
+</manifest>
diff --git a/library/src/test/java/com/telefonica/mistica/DummyActivity.kt b/library/src/test/java/com/telefonica/mistica/DummyActivity.kt
@@ -0,0 +1,13 @@
+package com.telefonica.mistica
+
+import android.os.Bundle
+import androidx.appcompat.app.AppCompatActivity
+import com.telefonica.mistica.R
+
+class DummyActivity : AppCompatActivity() {
+
+    override fun onCreate(savedInstanceState: Bundle?) {
+        super.onCreate(savedInstanceState)
+        setContentView(R.layout.test_dummy_activity)
+    }
+}
diff --git a/library/src/test/java/com/telefonica/mistica/compose/button/ButtonBehaviourTest.kt b/library/src/test/java/com/telefonica/mistica/compose/button/ButtonBehaviourTest.kt
@@ -0,0 +1,64 @@
+package com.telefonica.mistica.compose.button
+
+import androidx.compose.foundation.layout.padding
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.test.junit4.createComposeRule
+import androidx.compose.ui.test.onNodeWithText
+import androidx.compose.ui.test.performClick
+import androidx.compose.ui.unit.dp
+import com.telefonica.mistica.compose.theme.MisticaTheme
+import com.telefonica.mistica.compose.theme.brand.MovistarBrand
+import org.junit.Assert.assertTrue
+import org.junit.Rule
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.robolectric.RobolectricTestRunner
+
+@RunWith(RobolectricTestRunner::class)
+internal class ButtonBehaviourTest {
+    @get:Rule
+    val composeTestRule = createComposeRule()
+
+    @Test
+    fun `check the button is clicked`() = test {
+        `given Button`()
+
+        `when the button is clicked`()
+
+        `then the onClickListener has been invoked`()
+    }
+
+    private fun TestScope.`given Button`() {
+        `when Button`()
+    }
+
+    private fun TestScope.`when Button`() {
+        composeTestRule.setContent {
+            MisticaTheme(brand = MovistarBrand) {
+                Button(
+                    text = textValue,
+                    onClickListener = onClickListener,
+                    modifier = Modifier.padding(16.dp)
+                )
+            }
+        }
+    }
+
+    private fun TestScope.`when the button is clicked`() {
+        composeTestRule.onNodeWithText(textValue).performClick()
+    }
+
+    private fun TestScope.`then the onClickListener has been invoked`() {
+        assertTrue(clicked)
+    }
+
+    private fun test(block: TestScope.() -> Unit) {
+        TestScope().block()
+    }
+
+    private class TestScope {
+        val textValue = "textValue"
+        var clicked = false
+        val onClickListener: () -> Unit = { clicked = true }
+    }
+}
diff --git a/library/src/test/java/com/telefonica/mistica/compose/button/ButtonKtTest.kt b/library/src/test/java/com/telefonica/mistica/compose/button/ButtonKtTest.kt
@@ -0,0 +1,68 @@
+package com.telefonica.mistica.compose.button
+
+import androidx.compose.foundation.layout.padding
+import androidx.compose.material.Surface
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.test.junit4.createComposeRule
+import androidx.compose.ui.test.onRoot
+import androidx.compose.ui.unit.dp
+import com.telefonica.mistica.compose.theme.MisticaTheme
+import com.telefonica.mistica.compose.theme.brand.BlauBrand
+import com.telefonica.mistica.compose.theme.brand.Brand
+import com.telefonica.mistica.compose.theme.brand.MovistarBrand
+import com.telefonica.mistica.compose.theme.brand.O2Brand
+import com.telefonica.mistica.compose.theme.brand.TelefonicaBrand
+import com.telefonica.mistica.compose.theme.brand.VivoBrand
+import com.telefonica.mistica.testutils.ScreenshotsTest
+import org.junit.Rule
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.robolectric.ParameterizedRobolectricTestRunner
+
+@RunWith(ParameterizedRobolectricTestRunner::class)
+internal class ButtonKtTest(private val brand: Brand, private val darkTheme: Boolean): ScreenshotsTest() {
+    @get:Rule
+    val composeTestRule = createComposeRule()
+
+    @Test
+    fun `check the button screenshot`() {
+        `when Button`(brand, darkTheme)
+
+        `then screenshot is OK`(brand, darkTheme)
+    }
+
+    private fun `when Button`(brand: Brand = MovistarBrand, darkTheme: Boolean) {
+        composeTestRule.setContent {
+            MisticaTheme(brand = brand, darkTheme = darkTheme) {
+                Surface {
+                    Button(
+                        text = "textValue",
+                        onClickListener = { },
+                        modifier = Modifier.padding(16.dp)
+                    )
+                }
+            }
+        }
+    }
+
+    private fun `then screenshot is OK`(brand: Brand, darkTheme: Boolean) {
+        compareScreenshot(composeTestRule.onRoot(), brand, darkTheme)
+    }
+
+    companion object {
+        @JvmStatic
+        @ParameterizedRobolectricTestRunner.Parameters(name = "Input: {0}")
+        fun brands() = listOf(
+            arrayOf(MovistarBrand, false),
+            arrayOf(VivoBrand, false),
+            arrayOf(O2Brand, false),
+            arrayOf(BlauBrand, false),
+            arrayOf(TelefonicaBrand, false),
+            arrayOf(MovistarBrand, true),
+            arrayOf(VivoBrand, true),
+            arrayOf(O2Brand, true),
+            arrayOf(BlauBrand, true),
+            arrayOf(TelefonicaBrand, true),
+        )
+    }
+}