Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(sanity): migrate sanity package to vitest #7578

Merged
merged 3 commits into from
Oct 14, 2024
Merged

Conversation

binoy14
Copy link
Contributor

@binoy14 binoy14 commented Oct 3, 2024

Description

Migrates sanity tests to vitest

What to review

Changes makes sense and all tests pass

Testing

All tests should pass

Notes for release

N/A

Copy link

vercel bot commented Oct 3, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 14, 2024 1:06pm
performance-studio ✅ Ready (Inspect) Visit Preview Oct 14, 2024 1:06pm
test-compiled-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 14, 2024 1:06pm
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 14, 2024 1:06pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 14, 2024 1:06pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Oct 14, 2024 1:06pm

Copy link
Contributor Author

binoy14 commented Oct 3, 2024

Copy link
Contributor

github-actions bot commented Oct 3, 2024

No changes to documentation

Copy link
Contributor

github-actions bot commented Oct 3, 2024

Component Testing Report Updated Oct 14, 2024 1:54 PM (UTC)

✅ All Tests Passed -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 43s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 8s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 31s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 39s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 18s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 11s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 3m 0s 0 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 45s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 41s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 16s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 8s 3 9 0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx ✅ Passed (Inspect) 27s 9 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 18s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 36s 12 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Oct 3, 2024

⚡️ Editor Performance Report

Updated Mon, 14 Oct 2024 14:05:46 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 17.9 efps (56ms) 17.7 efps (57ms) +1ms (+0.9%)
article (body) 53.1 efps (19ms) 56.2 efps (18ms) -1ms (-5.6%)
article (string inside object) 17.9 efps (56ms) 18.5 efps (54ms) -2ms (-3.6%)
article (string inside array) 14.3 efps (70ms) 14.5 efps (69ms) -1ms (-1.4%)
recipe (name) 31.3 efps (32ms) 29.9 efps (34ms) +2ms (+4.7%)
recipe (description) 35.1 efps (29ms) 33.3 efps (30ms) +2ms (+5.3%)
recipe (instructions) 99.9+ efps (7ms) 99.9+ efps (7ms) +0ms (-/-%)
synthetic (title) 14.9 efps (67ms) 15.4 efps (65ms) -2ms (-3.0%)
synthetic (string inside object) 16.3 efps (62ms) 15.9 efps (63ms) +2ms (+2.4%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 56ms 63ms 82ms 212ms 1508ms 14.2s
article (body) 19ms 24ms 66ms 305ms 481ms 7.3s
article (string inside object) 56ms 61ms 71ms 287ms 1369ms 10.0s
article (string inside array) 70ms 76ms 100ms 283ms 1970ms 10.7s
recipe (name) 32ms 34ms 37ms 109ms 86ms 8.9s
recipe (description) 29ms 30ms 33ms 106ms 65ms 6.3s
recipe (instructions) 7ms 8ms 10ms 11ms 0ms 3.4s
synthetic (title) 67ms 70ms 117ms 385ms 2359ms 15.9s
synthetic (string inside object) 62ms 65ms 74ms 199ms 1872ms 10.4s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 57ms 62ms 104ms 206ms 1405ms 13.6s
article (body) 18ms 20ms 23ms 225ms 344ms 5.9s
article (string inside object) 54ms 60ms 69ms 308ms 1250ms 9.4s
article (string inside array) 69ms 74ms 97ms 205ms 1855ms 10.3s
recipe (name) 34ms 35ms 39ms 86ms 86ms 9.3s
recipe (description) 30ms 33ms 42ms 103ms 37ms 6.7s
recipe (instructions) 7ms 10ms 11ms 45ms 0ms 3.4s
synthetic (title) 65ms 70ms 78ms 440ms 2407ms 15.3s
synthetic (string inside object) 63ms 69ms 75ms 407ms 2437ms 10.8s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

@ryanbonial
Copy link
Contributor

@binoy14 The test pass, but I am seeing the addition of about 35 warnings about Warning: An update to ... inside a test was not wrapped in act(...). warnings.

CleanShot 2024-10-03 at 13 58 22@2x

@binoy14
Copy link
Contributor Author

binoy14 commented Oct 4, 2024

@binoy14 The test pass, but I am seeing the addition of about 35 warnings about Warning: An update to ... inside a test was not wrapped in act(...). warnings.

CleanShot 2024-10-03 at 13 58 22@2x

Pretty sure they existed during jest as well but were getting swallowed by jest. Do you see any instances where it could be causing this?

My suggestion is if other changes looks good we merge this and do a follow up PR that fixes the warnings

ryanbonial
ryanbonial previously approved these changes Oct 4, 2024
Copy link
Contributor

@ryanbonial ryanbonial left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Sounds like a good plan for a follow-up PR that fixes the warnings

@bjoerge
Copy link
Member

bjoerge commented Oct 14, 2024

I just rebased this against next and migrated a couple of new tests that came in while this was waiting to be merged.

bjoerge
bjoerge previously approved these changes Oct 14, 2024
Copy link
Member

@bjoerge bjoerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this over the finishing line, @binoy14! <3

Copy link
Contributor

@RitaDias RitaDias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great work thank you @binoy14 and @bjoerge you're legends ❤️ ✨

@bjoerge bjoerge added this pull request to the merge queue Oct 14, 2024
Merged via the queue into next with commit 6d23434 Oct 14, 2024
52 checks passed
@bjoerge bjoerge deleted the test-migrate-repo branch October 14, 2024 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants