Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DEMO] reproducer / visualizer for i2095 #2120

Closed
wants to merge 5 commits into from

Conversation

stenolog
Copy link
Contributor

@stenolog stenolog commented Feb 23, 2024


The unstable test is

property("transaction with too many inputs should be rejected") {

it is timing-related.

3 observations:

  • first execution take more time, it goes down later (cache? )
  • the overall speed has increased significantly since v5.0.0, test was not adjusted
  • timing behavior on CI is different (Timeout is slower in general, higher numbers)

It looks like halving the Timeout should do it.

Local Machine

sbt   "testOnly *ErgoTransactionSpec -- -z inputs"

v5.0.20

[info] ErgoTransactionSpec:
1: 455 > 155
2: 276 > 149
3: 240 > 145
4: 319 > 145
5: 160 > 144
6: 164 > 144
7: 159 > 150
8: 151 > 145
9: 201 > 150
10: 147 > 145
11: 486 > 257
12: 151 > 152
[info] - transaction with too many inputs should be rejected *** FAILED ***
[info]   151 was not greater than 152 (ErgoTransactionSpec.scala:375)

v5.0.4

[info] ErgoTransactionSpec:
1477 > 150
921 > 150
570 > 141
374 > 149
390 > 143
393 > 140
396 > 140
451 > 148
439 > 162

v5.0.0

[info] ErgoTransactionSpec:
1286 > 148
767 > 150
390 > 143
432 > 177
410 > 141
463 > 142
536 > 144
366 > 150
511 > 152
551 > 142
435 > 147
387 > 143

On CI

master (v5.0.20 +)

(halved the Timeout value)

using "testOnly *ErgoTransactionSpec -- -z inputs"

[info] ErgoTransactionSpec:
1: 419 > 108 (Timeout/2)
2: 265 > 99 (Timeout/2)
3: 256 > 97 (Timeout/2)
4: 191 > 97 (Timeout/2)
7: 128 > 97 (Timeout/2)
10: 140 > 97 (Timeout/2)
11: 126 > 97 (Timeout/2)
12: 141 > 97 (Timeout/2)
13: 141 > 97 (Timeout/2)
14: 155 > 97 (Timeout/2)
15: 169 > 96 (Timeout/2)
16: 186 > 98 (Timeout/2)
17: 127 > 97 (Timeout/2)
18: 129 > 98 (Timeout/2)
19: 156 > 99 (Timeout/2)
20: 127 > 98 (Timeout/2)
21: 123 > 98 (Timeout/2)
22: 158 > 98 (Timeout/2)
23: 124 > 98 (Timeout/2)
24: 143 > 98 (Timeout/2)
25: 124 > 97 (Timeout/2)
26: 177 > 100 (Timeout/2)
27: 156 > 98 (Timeout/2)
28: 130 > 98 (Timeout/2)
29: 128 > 98 (Timeout/2)
30: 166 > 98 (Timeout/2)
31: 161 > 98 (Timeout/2)
32: 161 > 98 (Timeout/2)
33: 153 > 97 (Timeout/2)
34: 125 > 102 (Timeout/2)
35: 129 > 97 (Timeout/2)
36: 167 > 98 (Timeout/2)
37: 173 > 98 (Timeout/2)
38: 134 > 97 (Timeout/2)
39: 128 > 98 (Timeout/2)
40: 181 > 98 (Timeout/2)
41: 182 > 97 (Timeout/2)
42: 124 > 98 (Timeout/2)
43: 128 > 98 (Timeout/2)
44: 125 > 98 (Timeout/2)
45: 178 > 98 (Timeout/2)
46: 127 > 98 (Timeout/2)
47: 130 > 97 (Timeout/2)
48: 127 > 98 (Timeout/2)
49: 131 > 97 (Timeout/2)
50: 129 > 98 (Timeout/2)

using the full ci run

1: 210 > 100 (Timeout/2)
2: 222 > 103 (Timeout/2)
3: 210 > 99 (Timeout/2)
4: 204 > 101 (Timeout/2)
5: 250 > 99 (Timeout/2)
6: 275 > 102 (Timeout/2)
7: 218 > 101 (Timeout/2)
8: 242 > 101 (Timeout/2)
9: 239 > 100 (Timeout/2)
10: 282 > 99 (Timeout/2)
11: 273 > 101 (Timeout/2)
12: 208 > 99 (Timeout/2)
13: 239 > 99 (Timeout/2)
14: 266 > 99 (Timeout/2)
15: 205 > 101 (Timeout/2)
16: 199 > 106 (Timeout/2)
17: 197 > 102 (Timeout/2)
18: 206 > 101 (Timeout/2)
19: 284 > 99 (Timeout/2)
20: 201 > 101 (Timeout/2)
21: 205 > 101 (Timeout/2)
22: 229 > 100 (Timeout/2)
23: 197 > 101 (Timeout/2)
24: 235 > 102 (Timeout/2)
25: 205 > 101 (Timeout/2)
26: 265 > 100 (Timeout/2)
27: 208 > 98 (Timeout/2)
28: 220 > 100 (Timeout/2)
29: 202 > 100 (Timeout/2)
30: 233 > 99 (Timeout/2)
31: 208 > 103 (Timeout/2)
32: 239 > 101 (Timeout/2)
33: 277 > 99 (Timeout/2)
34: 257 > 100 (Timeout/2)
35: 206 > 99 (Timeout/2)
36: 204 > 99 (Timeout/2)
37: 194 > 99 (Timeout/2)
38: 200 > 101 (Timeout/2)
39: 211 > 100 (Timeout/2)
40: 201 > 99 (Timeout/2)
41: 198 > 100 (Timeout/2)
42: 283 > 98 (Timeout/2)
43: 241 > 105 (Timeout/2)
44: 240 > 99 (Timeout/2)
45: 222 > 102 (Timeout/2)
46: 254 > 99 (Timeout/2)
47: 309 > 98 (Timeout/2)
48: 214 > 99 (Timeout/2)
49: 204 > 98 (Timeout/2)
50: 288 > 100 (Timeout/2)
[info] - transaction with too many inputs should be rejected

Run A

https://github.com/ergoplatform/ergo/actions/runs/8022318975/job/21916370871?pr=2120#step:5:8207

1 -209 > 212
[info] - transaction with too many inputs should be rejected *** FAILED ***
[info]   209 was not greater than 212 (ErgoTransactionSpec.scala:373)

Run B

1 -212 > 207
2 -224 > 202
3 -206 > 201
4 -234 > 201
5 -209 > 200
6 -209 > 217
[info] - transaction with too many inputs should be rejected *** FAILED ***
[info]   209 was not greater than 217 (ErgoTransactionSpec.scala:373)

@stenolog stenolog changed the title reproducer / visualizer for i2095 [DEMO] reproducer / visualizer for i2095 Feb 23, 2024
@stenolog
Copy link
Contributor Author

Closing this DEMO/reproducer PR. Fix is within:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant