DGPv2 Android integration tests #3904

adam-enko · 2024-11-05T11:09:41Z

Create integration tests for using DGPv2 with Android projects.

Additionally, create a JUnit extension for helping set up and run the tests.

Part of KT-70855 - the HTML currently contains errors, but this will be resolved in later PRs.

Overview

DokkaGradlePluginTestExtension is a JUnit extension for automatically configuring and re-running tests.
The behaviour can be configured by providing custom GradleTestProjectInitializer (for setting up the test projects),
or custom GradlePropertiesProvider (for providing custom Gradle properties).
The extension will re-run the tests multiple times with different versions.
TestedVersions.testedVersions() provides all possible, valid combinations of versions used to run the tests.
Add JUnit @Tag annotations, to help organise and categorise the tests.
Add two new test projects in dokka-integration-tests/gradle/projects: it-android and it-android-compose.
Each project has an expectedData directory containing the expected generated Dokka content.
~~Split GradlePropertiesBuilder into separate class, so it can be re-used in the integration tests~~
Moved to separate PR
Move GradlePropertiesBuilder test util to separate class #3906

Tip

This PR contains the expected HTML files, which make the PR difficult to view in GitHub.
You can filter out the files in the GitHub UI:

Create integration tests for using DGPv2 with Android projects. Additionally, create a JUnit extension for helping set up and run the tests. #### Overview - DokkaGradlePluginTestExtension is a JUnit extension for automatically configuring and re-running tests. The behaviour can be configured by providing custom GradleTestProjectInitializer (for setting up the test projects), or custom GradlePropertiesProvider (for providing custom Gradle properties). - The extension will re-run the tests multiple times with different versions. `TestedVersions.testedVersions()` provides all possible, valid combinations of versions used to run the tests. - Add JUnit `@Tag` annotations, to help organise and categorise the tests. - Add two new test projects in dokka-integration-tests/gradle/projects: it-android and it-android-compose. Each project has an `expectedData` directory containing the expected generated Dokka content. - Split GradlePropertiesBuilder into separate class, so it can be re-used in the integration tests

…id detecting CC status via an absence of log output

…ilure messages

Moving GradlePropertiesBuilder to a separate class will permit reusing it in Dokka integration tests that will be introduced in PR #3904

Significantly improve comparison between directories. Currently the failure messages are too obscure, and hard to diagnose. Example: https://ge.jetbrains.com/s/vl2jgh3ivevyi/tests/task/:dokka-integration-tests:gradle:test/details/org.jetbrains.dokka.it.gradle.AndroidProjectIT/generate%20dokka%20HTML(DokkaGradleProjectRunner)%5B1%5D?top-execution=1

…m/feat/KT-70336/android-integration-tests # Conflicts: # dokka-integration-tests/gradle/src/testExampleProjects/kotlin/ExampleProjectsTest.kt # dokka-runners/dokka-gradle-plugin/build.gradle.kts # dokka-runners/dokka-gradle-plugin/src/testFixtures/kotlin/kotestFiles.kt

…m/feat/KT-70336/android-integration-tests

Moving GradlePropertiesBuilder to a separate class will permit reusing it in Dokka integration tests that will be introduced in PR #3904

# Conflicts: # dokka-runners/dokka-gradle-plugin/src/test/kotlin/utils/ShouldBeADirectoryWithSameContentAsTest.kt # dokka-runners/dokka-gradle-plugin/src/testFixtures/kotlin/GradlePropertiesBuilder.kt # dokka-runners/dokka-gradle-plugin/src/testFixtures/kotlin/kotestFiles.kt

``` configuration cache cannot be reused because the file system entry '../../../../../../../home/teamcity/android-sdk-linux/build-tools/34.0.0/aapt' has been created. ```

whyoleg

Overall LGTM.

A lot of "magic" under the hood in junit setup (I don't have a lot of experince with writing custom extensions), but looks more or less understandable.
But I do really like how *IT.kt files look: nice and clear (and kotest also helps here)!
Thanks for improving our IT even more!

dokka-integration-tests/gradle/projects/it-android/settings.gradle.kts

dokka-integration-tests/gradle/projects/it-android-compose/settings.gradle.kts

...s/gradle/src/main/kotlin/org/jetbrains/dokka/it/gradle/junit/GradleTestProjectInitializer.kt

...tegration-tests/gradle/src/main/kotlin/org/jetbrains/dokka/it/gradle/junit/TestedVersions.kt

dokka-integration-tests/gradle/src/main/kotlin/org/jetbrains/dokka/it/gradle/junit/testTags.kt

dokka-integration-tests/gradle/src/test/kotlin/AndroidComposeIT.kt

…to lots of files

antohaby · 2024-11-18T14:05:11Z

dokka-integration-tests/gradle/src/test/kotlin/AndroidComposeIT.kt

+        project.runner
+            .addArguments(
+                ":dokkaGenerate",
+                "--build-cache",


tests appears to be testing only up-to-date checks on re-run.
why build cache is needed here?

shouldn't we also tests for some "changes" in outputs or inputs?

Thanks, good point, I've updated the assertions to test the actual Dokka generation work tasks, not just the dokkaGenerate lifecycle task (which is not cacheable since it has no inputs/outputs).

antohaby · 2024-11-18T14:06:23Z

dokka-integration-tests/gradle/src/test/kotlin/AndroidComposeIT.kt

+            project.runner.addArguments(
+                "clean",
+                ":dokkaGenerate",
+                "--no-build-cache",


why is it necessary to specify "no build cache" here? I though configuration cache can work without knowing anything about build cache as it is more related to execution phase.

Hmmm I suppose disabling build cache not strictly necessary, but I can imagine that a buggy Gradle task might produce some uncached output that could disrupt CC.

TBH I can't remember why it's there. Perhaps it could be removed, but I don't think it does any harm.

antohaby · 2024-11-18T14:08:03Z

dokka-integration-tests/gradle/src/test/kotlin/AndroidProjectIT.kt

+    }
+
+    @DokkaGradlePluginTest(sourceProjectName = "it-android")
+    fun `expect Dokka is compatible with Gradle Configuration Cache`(


also it looks like AndroidComposeIT and AndroidProjectIT have shared code. Is it ok for you to keep those duplicated? i.e. you don't see lots of maintenance with this code in the future.

Yes, quite a lot of the tests are duplicated. I intend to add more tests KT-71163, but I don't want to pre-emptively optimise and start deduplicating code just yet. I'd rather add more tests, and then see what could be extracted out.

The setup and test assertions for Android projects is more complicated than most other projects, so I'd prefer to hold off on trying to DRY until some more basic projects are tested.

gradle/gradle#5511

- Make TestedVersions more strict, with specific subclasses for the Android and AndroidCompose tests. - Split out TestedVersions generation into TestedVersionsSource.

…arents.

…am/feat/KT-70336/android-integration-tests

adam-enko added 2 commits November 5, 2024 12:04

updated expected data

6109ba0

adam-enko added the runner: Gradle plugin An issue/PR related to Dokka's Gradle plugin label Nov 5, 2024

adam-enko self-assigned this Nov 5, 2024

adam-enko added 10 commits November 5, 2024 12:12

IJ exclude dokka-integration-tests/.kotlin

4cced1f

Add more kdoc

cec0a7a

update assertions, use loadConfigurationCacheReportData util to avo…

8221b9f

…id detecting CC status via an absence of log output

Skip testing AGP8 until KT-70855 is fixed

cdbe9b0

improve loadConfigurationCacheReportData, tidy assertions

1959c37

rm commented out code

92fb66c

fix updateProjectLocalMavenDir()

5cc2de2

Improve directory-contents assertion to provide much more detailed fa…

fd6297a

…ilure messages

tidy shouldBeDirectoryWithSameContentAs assertion

51c18a4

fix gradle task dependency

f6c3d38

adam-enko added a commit that referenced this pull request Nov 5, 2024

Move GradlePropertiesBuilder test util to separate class

e4b59f3

Moving GradlePropertiesBuilder to a separate class will permit reusing it in Dokka integration tests that will be introduced in PR #3904

adam-enko mentioned this pull request Nov 5, 2024

Move GradlePropertiesBuilder test util to separate class #3906

Merged

adam-enko added a commit that referenced this pull request Nov 5, 2024

Move GradlePropertiesBuilder test util to separate class

eaa8bfa

Moving GradlePropertiesBuilder to a separate class will permit reusing it in Dokka integration tests that will be introduced in PR #3904

adam-enko added 4 commits November 5, 2024 17:14

comparing fix binary data

492a055

Merge branch 'adam/feat/improve-directory-compare-assertion' into ada…

de2834c

…m/feat/KT-70336/android-integration-tests

adam-enko force-pushed the adam/feat/KT-70336/android-integration-tests branch from e9a6f7c to de2834c Compare November 5, 2024 18:17

adam-enko added 3 commits November 5, 2024 19:55

replace non-printable characters with the codepoint

f9ccd37

tidy replaceNonPrintableWithCodepoint

9c7a32a

Merge branch 'adam/feat/improve-directory-compare-assertion' into ada…

66726d0

…m/feat/KT-70336/android-integration-tests

adam-enko added a commit that referenced this pull request Nov 7, 2024

Move GradlePropertiesBuilder test util to separate class (#3906)

58327ac

Moving GradlePropertiesBuilder to a separate class will permit reusing it in Dokka integration tests that will be introduced in PR #3904

adam-enko added 4 commits November 7, 2024 20:31

try re-building to prevent CC miss on TeamCity

76e30c1

``` configuration cache cannot be reused because the file system entry '../../../../../../../home/teamcity/android-sdk-linux/build-tools/34.0.0/aapt' has been created. ```

clear CC reports

13ef2d5

clear CC reports

01d1674

formatting

9a4e2e7

adam-enko requested review from Tapchicoma and whyoleg November 11, 2024 10:44

adam-enko added this to the Dokka 2.0.0 milestone Nov 11, 2024

adam-enko marked this pull request as ready for review November 11, 2024 10:44

adam-enko added 2 commits November 12, 2024 12:21

fix import

5053890

Merge branch 'master' into adam/feat/KT-70336/android-integration-tests

ed5b511

whyoleg approved these changes Nov 13, 2024

View reviewed changes

adam-enko added 2 commits November 13, 2024 14:45

order properties alphabetically

2b37748

Only update versions in specific files, to avoid reading and writing …

ddcd84c

…to lots of files

antohaby self-requested a review November 18, 2024 13:43

antohaby reviewed Nov 18, 2024

View reviewed changes

adam-enko added 9 commits November 19, 2024 09:47

add workaround for Gradle not logging aborted tests

7913ad6

gradle/gradle#5511

- rename test tag annotations

6e66872

- Make TestedVersions more strict, with specific subclasses for the Android and AndroidCompose tests. - Split out TestedVersions generation into TestedVersionsSource.

fix Gradle version

c0f8c79

Fix finding the annotations for the tested element by searching the p…

9015efa

…arents.

fix Gradle version

0d4370d

Fix test assertion

4b86360

Merge branch 'adam/fix/OverrideJsonConfig-fix-test-assertion' into ad…

dd87e63

…am/feat/KT-70336/android-integration-tests

Improve shouldHaveOutcome assertion message

09999fc

Fix build cache tests

a774f69

adam-enko requested a review from antohaby November 19, 2024 16:37

antohaby approved these changes Nov 20, 2024

View reviewed changes

adam-enko merged commit 0c87119 into master Nov 20, 2024
14 checks passed

adam-enko deleted the adam/feat/KT-70336/android-integration-tests branch November 20, 2024 09:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DGPv2 Android integration tests #3904

DGPv2 Android integration tests #3904

adam-enko commented Nov 5, 2024 •

edited

Loading

whyoleg left a comment

antohaby Nov 18, 2024

adam-enko Nov 19, 2024 •

edited

Loading

antohaby Nov 18, 2024

adam-enko Nov 19, 2024

antohaby Nov 18, 2024

adam-enko Nov 19, 2024

DGPv2 Android integration tests #3904

DGPv2 Android integration tests #3904

Conversation

adam-enko commented Nov 5, 2024 • edited Loading

Overview

whyoleg left a comment

Choose a reason for hiding this comment

antohaby Nov 18, 2024

Choose a reason for hiding this comment

adam-enko Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

antohaby Nov 18, 2024

Choose a reason for hiding this comment

adam-enko Nov 19, 2024

Choose a reason for hiding this comment

antohaby Nov 18, 2024

Choose a reason for hiding this comment

adam-enko Nov 19, 2024

Choose a reason for hiding this comment

adam-enko commented Nov 5, 2024 •

edited

Loading

adam-enko Nov 19, 2024 •

edited

Loading