Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(se, all): Improve test output #4674

Merged
merged 2 commits into from
Jan 26, 2024
Merged

test(se, all): Improve test output #4674

merged 2 commits into from
Jan 26, 2024

Conversation

miguelff
Copy link
Contributor

@miguelff miguelff commented Jan 26, 2024

  • run schema engine tests with default log level
  • omit status output of skipped tests

* run schema engine tests with default log level
* omit status output of skipped tests
* reduce number of tries to a total of 4
@miguelff miguelff requested a review from a team as a code owner January 26, 2024 09:52
@miguelff miguelff requested review from laplab and jkomyno and removed request for a team January 26, 2024 09:52
@miguelff miguelff changed the base branch from main to janpio/shadow-drop-with-force January 26, 2024 09:53
Copy link
Contributor

WASM Size

Engine This PR Base branch Diff
WASM 2.329MiB 2.329MiB 0.000B
WASM (gzip) 906.378KiB 906.378KiB 0.000B

Copy link

codspeed-hq bot commented Jan 26, 2024

CodSpeed Performance Report

Merging #4674 will not alter performance

Comparing schema-tests-output (2e4762c) with main (ac49383)

Summary

✅ 11 untouched benchmarks

Copy link
Contributor

github-actions bot commented Jan 26, 2024

✅ WASM query-engine performance won't change substantially (0.998x)

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.0 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - 25000)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  302.32 ms/iter (299.48 ms … 307.21 ms) 306.68 ms 307.21 ms 307.21 ms
Web Assembly: Latest    367.57 ms/iter (364.52 ms … 373.38 ms)  370.4 ms 373.38 ms 373.38 ms
Web Assembly: Current   365.51 ms/iter (363.37 ms … 368.86 ms) 368.52 ms 368.86 ms 368.86 ms
Node API: Current       233.33 ms/iter  (222.06 ms … 242.4 ms) 242.34 ms  242.4 ms  242.4 ms

summary for movies.findMany() (all - 25000)
  Web Assembly: Current
   1.57x slower than Node API: Current
   1.21x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.25 ms/iter   (12.06 ms … 14.23 ms)  12.21 ms  14.23 ms  14.23 ms
Web Assembly: Latest     14.98 ms/iter   (14.77 ms … 16.46 ms)  14.91 ms  16.46 ms  16.46 ms
Web Assembly: Current    14.87 ms/iter   (14.72 ms … 15.61 ms)  14.87 ms  15.61 ms  15.61 ms
Node API: Current        9,025 µs/iter   (8,676 µs … 12.59 ms)  8,892 µs  12.59 ms  12.59 ms

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.65x slower than Node API: Current
   1.21x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,934 µs/iter   (1,832 µs … 3,349 µs)  1,917 µs  2,802 µs  3,349 µs
Web Assembly: Latest     2,376 µs/iter   (2,267 µs … 3,844 µs)  2,352 µs  3,457 µs  3,844 µs
Web Assembly: Current    2,347 µs/iter   (2,268 µs … 2,882 µs)  2,351 µs  2,807 µs  2,882 µs
Node API: Current        1,521 µs/iter   (1,457 µs … 1,745 µs)  1,534 µs  1,715 µs  1,745 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.54x slower than Node API: Current
   1.21x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.12 ms/iter   (12.04 ms … 12.46 ms)  12.14 ms  12.46 ms  12.46 ms
Web Assembly: Latest     14.87 ms/iter   (14.79 ms … 15.01 ms)  14.91 ms  15.01 ms  15.01 ms
Web Assembly: Current    14.83 ms/iter   (14.75 ms … 14.91 ms)  14.86 ms  14.91 ms  14.91 ms
Node API: Current        8,923 µs/iter   (8,734 µs … 9,196 µs)  8,944 µs  9,196 µs  9,196 µs

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.66x slower than Node API: Current
   1.22x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,913 µs/iter   (1,819 µs … 2,826 µs)  1,913 µs  2,505 µs  2,826 µs
Web Assembly: Latest     2,314 µs/iter   (2,258 µs … 2,869 µs)  2,314 µs  2,714 µs  2,869 µs
Web Assembly: Current    2,371 µs/iter   (2,248 µs … 3,909 µs)  2,320 µs  3,763 µs  3,909 µs
Node API: Current        1,487 µs/iter   (1,405 µs … 2,014 µs)  1,504 µs  1,689 µs  2,014 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.59x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    12.1 ms/iter   (11.99 ms … 12.51 ms)  12.13 ms  12.51 ms  12.51 ms
Web Assembly: Latest     14.85 ms/iter   (14.75 ms … 15.02 ms)  14.88 ms  15.02 ms  15.02 ms
Web Assembly: Current    14.78 ms/iter    (14.7 ms … 15.06 ms)  14.81 ms  15.06 ms  15.06 ms
Node API: Current        8,942 µs/iter   (8,812 µs … 9,684 µs)  8,972 µs  9,684 µs  9,684 µs

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.65x slower than Node API: Current
   1.22x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,873 µs/iter   (1,808 µs … 2,279 µs)  1,882 µs  2,201 µs  2,279 µs
Web Assembly: Latest     2,315 µs/iter   (2,263 µs … 2,566 µs)  2,320 µs  2,523 µs  2,566 µs
Web Assembly: Current    2,301 µs/iter   (2,254 µs … 2,557 µs)  2,308 µs  2,459 µs  2,557 µs
Node API: Current        1,486 µs/iter   (1,417 µs … 1,947 µs)  1,508 µs  1,691 µs  1,947 µs

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.55x slower than Node API: Current
   1.23x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  936.02 µs/iter  (881.86 µs … 1,539 µs) 930.11 µs  1,444 µs  1,539 µs
Web Assembly: Latest     1,144 µs/iter   (1,105 µs … 1,892 µs)  1,149 µs  1,350 µs  1,892 µs
Web Assembly: Current    1,152 µs/iter   (1,106 µs … 1,800 µs)  1,154 µs  1,525 µs  1,800 µs
Node API: Current       822.41 µs/iter     (744 µs … 1,065 µs) 845.51 µs  1,028 µs  1,065 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.4x slower than Node API: Current
   1.23x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  893.83 µs/iter  (852.44 µs … 1,427 µs) 898.32 µs  1,146 µs  1,427 µs
Web Assembly: Latest     1,124 µs/iter   (1,080 µs … 1,890 µs)  1,121 µs  1,693 µs  1,890 µs
Web Assembly: Current    1,123 µs/iter   (1,087 µs … 1,756 µs)  1,127 µs  1,323 µs  1,756 µs
Node API: Current       825.56 µs/iter  (748.68 µs … 1,565 µs) 844.11 µs 942.31 µs  1,565 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.36x slower than Node API: Current
   1.26x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

After changes in 2e4762c

@miguelff miguelff changed the base branch from janpio/shadow-drop-with-force to main January 26, 2024 10:08
@miguelff miguelff merged commit c2e5bf2 into main Jan 26, 2024
127 of 136 checks passed
@miguelff miguelff deleted the schema-tests-output branch January 26, 2024 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants