Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI step to build and test swift ui example using maestro #4395

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

krystofwoldrich
Copy link
Member

@krystofwoldrich krystofwoldrich commented Oct 3, 2024

📜 Description

This PR adds a new UI test executed by Maestro in GH CI. This is part of the exploration to make UI Tests more stable.

💚 How did you test it?

Programatically rerun the Maestro Crash test 100 times.

Results: 1/100 tests failed

Record of the results -> https://github.com/getsentry/sentry-cocoa/actions/runs/11162209598/job/31068682803?pr=4395

The script:

GITHUB_REPO_OWNER="getsentry"
GITHUB_REPO_NAME="sentry-cocoa"
WORKFLOW_RUN="11162209598"
JOB_TO_REREUN_INDEX="1"
DEBOUNCE_TIMEOUT_MS=300 #5 minutes

for i in {1..1000}; do
  workflow=$(gh api \
    -H "Accept: application/json" \
    -H "X-GitHub-Api-Version: 2022-11-28" \
    "/repos/${GITHUB_REPO_OWNER}/${GITHUB_REPO_NAME}/actions/runs/${WORKFLOW_RUN}/jobs")

  job_id=$(echo "$workflow" | jq -r ".jobs[${JOB_TO_REREUN_INDEX}].id")
  job_status=$(echo "$workflow" | jq -r ".jobs[${JOB_TO_REREUN_INDEX}].status")
  job_conclusion=$(echo "$workflow" | jq -r ".jobs[${JOB_TO_REREUN_INDEX}].conclusion")
  job_run_attempt=$(echo "$workflow" | jq -r ".jobs[${JOB_TO_REREUN_INDEX}].run_attempt")

  echo "Attempt number $job_run_attempt ($job_id), status $job_status, conclusion $job_conclusion"

  if [[ "$job_status" == "completed" ]]; then
    rerun=$(gh api \
      --method POST \
      -H "Accept: application/json" \
      -H "X-GitHub-Api-Version: 2022-11-28" \
      /repos/${GITHUB_REPO_OWNER}/${GITHUB_REPO_NAME}/actions/jobs/${job_id}/rerun)
    echo "Rerun requested"
  fi

  sleep $DEBOUNCE_TIMEOUT_MS
done

📝 Checklist

You have to check all boxes before merging:

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
  • I updated the docs if needed.
  • Review from the native team if needed.
  • No breaking change or entry added to the changelog.
  • No breaking change for hybrid SDKs or communicated to hybrid SDKs.

#skip-changelog

🔮 Next steps

  • Add more iOS versions and other sample apps to fully replicated the current UI Tests for the Crash test
  • Add the remaining UI Tests

Copy link

codecov bot commented Oct 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.434%. Comparing base (4eea4d1) to head (5bbccd0).
Report is 4 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##              main     #4395       +/-   ##
=============================================
+ Coverage   91.424%   91.434%   +0.009%     
=============================================
  Files          628       628               
  Lines        50530     50541       +11     
  Branches     18255     18239       -16     
=============================================
+ Hits         46197     46212       +15     
+ Misses        4240      4237        -3     
+ Partials        93        92        -1     

see 4 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4eea4d1...5bbccd0. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant