Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Max parallel jobs = 2. Slower, but more runner freedom. #1620

Merged
merged 1 commit into from
Sep 11, 2024

Conversation

noonio
Copy link
Contributor

@noonio noonio commented Sep 10, 2024

Hack to avoid using all our runners on these tasks.

@noonio noonio force-pushed the hotfix-concurrent-network-tests branch from d3d1038 to ecf9952 Compare September 10, 2024 12:49
Copy link

github-actions bot commented Sep 10, 2024

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-09-11 09:22:56.707533554 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial 2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff 3799
νCommit 2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d 1743
νHead 2ee477c60839936be49a50030690865b5bed4db8cd2f05bf255ac680 10068
μHead a1610f6e64843161f4a88229c0286176f5325de3e2f773eec2b1d818* 4508
νDeposit c2117fd9ebdee3e96b81fd67ff7092d638926415c10f1f7e5a267ad0 2791
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 5096 5.71 2.25 0.44
2 5301 7.17 2.84 0.46
3 5499 8.37 3.30 0.48
5 5901 11.12 4.39 0.53
10 6907 18.21 7.20 0.65
57 16356 82.89 32.79 1.78

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 569 10.52 4.15 0.29
2 755 13.86 5.65 0.34
3 940 17.33 7.20 0.38
5 1319 24.65 10.44 0.48
10 2260 45.22 19.36 0.75
20 4129 95.99 40.76 1.40

CollectCom transaction costs

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 560 22.17 8.68 0.42
2 113 671 32.96 13.05 0.54
3 170 782 47.08 18.79 0.70
4 227 893 57.57 23.31 0.82
5 282 1004 75.57 30.73 1.03
6 339 1116 92.01 37.67 1.21

Cost of Decrement Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 640 17.70 7.78 0.38
2 727 17.82 8.52 0.39
3 859 19.10 9.74 0.41
5 1248 25.40 13.68 0.51
10 1967 32.86 20.25 0.65
50 8014 98.99 75.15 1.84

Close transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 602 19.69 8.69 0.40
2 783 21.49 10.39 0.43
3 919 22.96 11.79 0.46
5 1352 26.67 15.32 0.54
10 2016 34.36 22.62 0.68
50 8096 98.07 83.36 1.90

Contest transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 671 25.89 11.13 0.47
2 892 28.10 13.02 0.51
3 964 29.49 14.19 0.53
5 1279 33.40 17.53 0.60
10 2001 42.91 25.67 0.77
40 6406 98.48 73.77 1.76

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4991 17.43 7.58 0.57
2 5067 25.09 10.86 0.65
3 5226 41.66 18.36 0.85
4 5374 58.46 25.90 1.05
5 5443 74.51 33.03 1.23

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4935 7.30 3.08 0.45
5 1 57 4968 8.83 3.96 0.47
5 5 284 5104 13.55 6.89 0.53
5 10 570 5274 18.81 10.28 0.61
5 20 1135 5610 30.52 17.57 0.77
5 30 1708 5955 42.04 24.77 0.93
5 40 2276 6292 52.98 31.73 1.08
5 50 2846 6633 64.51 38.94 1.24
5 81 4608 7681 99.56 61.02 1.73

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-09-11 09:26:02.488826203 UTC

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 4.905382186
P99 9.628830349999998ms
P95 7.248170149999995ms
P50 4.514454000000001ms
Number of Invalid txs 0

Three local nodes

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 25.288767151
P99 110.25439676000049ms
P95 34.7556047ms
P50 22.943684ms
Number of Invalid txs 0

Copy link

github-actions bot commented Sep 10, 2024

Test Results

490 tests  ±0   485 ✅ ±0   20m 0s ⏱️ -10s
155 suites ±0     5 💤 ±0 
  6 files   ±0     0 ❌ ±0 

Results for commit bbcc336. ± Comparison against base commit 7025d78.

♻️ This comment has been updated with latest results.

@noonio noonio force-pushed the hotfix-concurrent-network-tests branch from ecf9952 to 72333dd Compare September 11, 2024 07:42
@noonio noonio force-pushed the hotfix-concurrent-network-tests branch from 72333dd to bbcc336 Compare September 11, 2024 08:38
Copy link
Collaborator

@ch1bo ch1bo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not call this a hack, looks good.

@noonio noonio merged commit 3101449 into master Sep 11, 2024
38 checks passed
@noonio noonio deleted the hotfix-concurrent-network-tests branch September 11, 2024 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants