feat: new "scan throughput" objective in sampling compressor #1068
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Vortex bytes_at
| Benchmark suite | Current: b72b1fb | Previous: af290fc | Ratio |
|---|---|---|---|
bytes_at/array_data |
755.530466784667 ns (1.2435506789787496) |
734.0197890279654 ns (0.9748507570764673) |
1.03 |
bytes_at/array_view |
537.4593149261243 ns (0.7280168170078696) |
535.3736656666007 ns (0.8694249013215085) |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
DataFusion
| Benchmark suite | Current: b72b1fb | Previous: af290fc | Ratio |
|---|---|---|---|
arrow/planning |
768365.930168632 ns (1941.7431989887264) |
759131.6851852029 ns (1571.356140103424) |
1.01 |
arrow/exec |
1356418.2775881507 ns (4296.495727451751) |
1349942.1699783506 ns (3244.4641477082623) |
1.00 |
vortex-pushdown-compressed/planning |
474562.4211203304 ns (1393.7525557483314) |
469657.8263937358 ns (825.556854185852) |
1.01 |
vortex-pushdown-compressed/exec |
2624185.784 ns (14222.43725625053) |
2670176.552631579 ns (13363.746032895055) |
0.98 |
vortex-pushdown-uncompressed/planning |
474226.2486482615 ns (1059.3670447697805) |
469699.21025000256 ns (1123.2526276204153) |
1.01 |
vortex-pushdown-uncompressed/exec |
2521125.5635 ns (3907.566881250823) |
2523513.4880000013 ns (2602.907812499907) |
1.00 |
vortex-nopushdown-compressed/planning |
784670.9077058451 ns (1390.7695446668076) |
770468.006451082 ns (1044.4453110002214) |
1.02 |
vortex-nopushdown-compressed/exec |
3057983.2252941174 ns (27119.545941176126) |
2870182.354444444 ns (24527.4233611112) |
1.07 |
vortex-nopushdown-uncompressed/planning |
780895.192227083 ns (1983.1598266228102) |
768087.5889392112 ns (1173.9160802288097) |
1.02 |
vortex-nopushdown-uncompressed/exec |
4591341.682727273 ns (8198.59544318309) |
4667867.675454547 ns (15698.754636364989) |
0.98 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Random Access
| Benchmark suite | Current: b72b1fb | Previous: af290fc | Ratio |
|---|---|---|---|
random-access/vortex-tokio-local-disk |
2485862.8442857135 ns (24422.696779762162) |
2735624.9428571425 ns (50935.692125000525) |
0.91 |
random-access/vortex-local-fs |
4098785.040769233 ns (38274.13508653897) |
4363358.146666668 ns (36548.05696874997) |
0.94 |
random-access/parquet-tokio-local-disk |
228076700.83333334 ns (3989482.1120833308) |
224307841.1 ns (2621397.4666666687) |
1.02 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
TPC-H
| Benchmark suite | Current: b72b1fb | Previous: af290fc | Ratio |
|---|---|---|---|
tpch_q1/vortex-in-memory-no-pushdown |
562967614.3 ns (2443413.648750007) |
560903059.7 ns (2315942.443750024) |
1.00 |
tpch_q1/vortex-in-memory-pushdown |
453303106.85 ns (705614.5249999762) |
453279161.35 ns (1050434.5074999928) |
1.00 |
tpch_q1/arrow |
542683013.8 ns (1937726.4237499833) |
541180368.7 ns (1829295.388749957) |
1.00 |
tpch_q1/parquet |
684381715.5 ns (1992708.8475000262) |
680816510.7 ns (2331700.2024999857) |
1.01 |
tpch_q1/vortex-file-compressed |
493311471.1 ns (2747244.800000012) |
490824112.85 ns (651307.9693749845) |
1.01 |
tpch_q1/vortex-file-uncompressed |
531869907.8 ns (3795302.9724999964) |
526143190.9 ns (2270819.5) |
1.01 |
tpch_q2/vortex-in-memory-no-pushdown |
120857963.77650794 ns (943091.616662696) |
119481519.74734128 ns (858322.6932227239) |
1.01 |
tpch_q2/vortex-in-memory-pushdown |
117239486.05142859 ns (1043214.6738095134) |
117738979.70678572 ns (813874.6121249944) |
1.00 |
tpch_q2/arrow |
116055433.24488096 ns (829902.7831101269) |
115820249.6738492 ns (451816.92477430403) |
1.00 |
tpch_q2/parquet |
147327868.9804762 ns (587813.6945714355) |
147328338.5947222 ns (923552.8610659689) |
1.00 |
tpch_q2/vortex-file-compressed |
160615899.7108333 ns (2217498.801666662) |
161553767.1765873 ns (953834.1931865215) |
0.99 |
tpch_q2/vortex-file-uncompressed |
166967028.26194444 ns (1276862.4501875043) |
166661916.6426984 ns (1794221.692936495) |
1.00 |
tpch_q3/vortex-in-memory-no-pushdown |
159674376.64515874 ns (1461364.6101468205) |
159573944.92067462 ns (761153.0081944317) |
1.00 |
tpch_q3/vortex-in-memory-pushdown |
177052312.2523016 ns (1006070.5580952317) |
174251689.6133333 ns (1098273.4842916727) |
1.02 |
tpch_q3/arrow |
147886320.19277778 ns (1173379.5065486133) |
148078683.90432543 ns (473691.1423665583) |
1.00 |
tpch_q3/parquet |
335724923.2 ns (1622214.4162500203) |
335320221.65 ns (864424.4956249893) |
1.00 |
tpch_q3/vortex-file-compressed |
283065200.1 ns (1096972.400000006) |
281685837.5 ns (1101609.5024999976) |
1.00 |
tpch_q3/vortex-file-uncompressed |
261866234.3 ns (1697493.7668749988) |
263040689.45 ns (1470559.5287500024) |
1.00 |
tpch_q4/vortex-in-memory-no-pushdown |
111610825.70202382 ns (440548.08142857254) |
110549582.626627 ns (374243.2343055606) |
1.01 |
tpch_q4/vortex-in-memory-pushdown |
131816474.68698414 ns (327361.8428680524) |
128763600.21821427 ns (479690.68556548655) |
1.02 |
tpch_q4/arrow |
180814779.88361114 ns (608397.644760415) |
183203671.76666668 ns (441619.89999999106) |
0.99 |
tpch_q4/parquet |
205365092.06666666 ns (1095508.971666649) |
202293055.5 ns (1098659.3666666448) |
1.02 |
tpch_q4/vortex-file-compressed |
241097772.56666666 ns (1172721.4020833373) |
236078555.96666664 ns (806675.8241666853) |
1.02 |
tpch_q4/vortex-file-uncompressed |
242135917.46666664 ns (1459202.5666666478) |
235104135.40000004 ns (673102.3666666746) |
1.03 |
tpch_q5/vortex-in-memory-no-pushdown |
282503311.1 ns (1797579.0737499893) |
282986968.1 ns (1055003.259375006) |
1.00 |
tpch_q5/vortex-in-memory-pushdown |
291108729.45 ns (2264306.5112499893) |
285310084.95 ns (1991238.1299999952) |
1.02 |
tpch_q5/arrow |
268343385.15 ns (2141004.248124987) |
269064060.8 ns (2064783.2043749988) |
1.00 |
tpch_q5/parquet |
444923946.8 ns (1949350.7306250036) |
440469871.25 ns (1695036.349999994) |
1.01 |
tpch_q5/vortex-file-compressed |
367659896.2 ns (2183367.4724999666) |
358153976.4 ns (2271668.946875006) |
1.03 |
tpch_q5/vortex-file-uncompressed |
351211161.8 ns (2961775.175000012) |
349522609.8 ns (2442973.574999988) |
1.00 |
tpch_q6/vortex-in-memory-no-pushdown |
34664159.361534394 ns (195071.93084821105) |
35070552.60165344 ns (164481.65159391984) |
0.99 |
tpch_q6/vortex-in-memory-pushdown |
69218682.60996032 ns (197181.1116458401) |
68705098.32916667 ns (151486.16480208933) |
1.01 |
tpch_q6/arrow |
24915494.391775794 ns (150541.08740079403) |
24914071.70181548 ns (208577.34863616154) |
1.00 |
tpch_q6/parquet |
136037704.75638887 ns (408636.15560764074) |
136700963.5486508 ns (347315.7490783632) |
1.00 |
tpch_q6/vortex-file-compressed |
20297218.656190477 ns (346531.2007023804) |
20925454.9445873 ns (512816.7214184515) |
0.97 |
tpch_q6/vortex-file-uncompressed |
243693965.06666666 ns (658318.1908333302) |
240608022.70000005 ns (866799.1762500256) |
1.01 |
tpch_q7/vortex-in-memory-no-pushdown |
555066898.4 ns (2074297.949999988) |
540264375.1 ns (4470869.902499974) |
1.03 |
tpch_q7/vortex-in-memory-pushdown |
581308484 ns (2548068.649999976) |
569877128.8 ns (3707505.1887500286) |
1.02 |
tpch_q7/arrow |
538766645.8 ns (1580436.4712500572) |
527073182.3 ns (2441852.692499995) |
1.02 |
tpch_q7/parquet |
667889541.9 ns (6157865.397499979) |
669651357.8 ns (3410206.9674999714) |
1.00 |
tpch_q7/vortex-file-compressed |
688955130.9 ns (2609535.600000024) |
676006481.3 ns (3353048.3312499523) |
1.02 |
tpch_q7/vortex-file-uncompressed |
681130765.5 ns (4840387.631249964) |
670146264.7 ns (4101105.9712499976) |
1.02 |
tpch_q8/vortex-in-memory-no-pushdown |
221811638.4666667 ns (910449.5724999905) |
220497324.9666667 ns (1972420.8945833445) |
1.01 |
tpch_q8/vortex-in-memory-pushdown |
225975237.86666664 ns (1102432.5137500316) |
227317401.63333336 ns (1233026.6383333504) |
0.99 |
tpch_q8/arrow |
208247865.20000002 ns (1115019.9987499863) |
207199928.4 ns (1041959.1670833379) |
1.01 |
tpch_q8/parquet |
488163198.5 ns (3973198.1306249797) |
483563624.35 ns (2013434.3224999905) |
1.01 |
tpch_q8/vortex-file-compressed |
309705614.55 ns (2759521.7568750083) |
307277488 ns (2970428) |
1.01 |
tpch_q8/vortex-file-uncompressed |
303085170.1 ns (1878551.0181249678) |
301205403.9 ns (3261701.719999999) |
1.01 |
tpch_q9/vortex-in-memory-no-pushdown |
410984508.4 ns (2476448.1537500024) |
417702865.9 ns (2314668.8400000036) |
0.98 |
tpch_q9/vortex-in-memory-pushdown |
412361204.7 ns (2462216.9174999893) |
415878165.95 ns (3537223.835624993) |
0.99 |
tpch_q9/arrow |
399038414.6 ns (3766606.900000006) |
390901032.35 ns (3053577.8274999857) |
1.02 |
tpch_q9/parquet |
699646415.7 ns (2752273.872499943) |
689686081.8 ns (3946607.4025000334) |
1.01 |
tpch_q9/vortex-file-compressed |
489272050.05 ns (4159700.875) |
482609323.4 ns (3672309.0775000155) |
1.01 |
tpch_q9/vortex-file-uncompressed |
466556058.9 ns (4521050.761250019) |
467941974.25 ns (4539477.199999988) |
1.00 |
tpch_q10/vortex-in-memory-no-pushdown |
279260781.95 ns (1630089.3250000179) |
276221219.35 ns (1538522.3643749952) |
1.01 |
tpch_q10/vortex-in-memory-pushdown |
300527673.35 ns (951177.4262500107) |
301016572.25 ns (1393885.4193749726) |
1.00 |
tpch_q10/arrow |
263807922.8 ns (1294803.8012499958) |
261369735.7 ns (1517030.1012499928) |
1.01 |
tpch_q10/parquet |
499967692.85 ns (893176.5456250012) |
495938950.8 ns (1619341.7250000238) |
1.01 |
tpch_q10/vortex-file-compressed |
425619609.65 ns (2237984.200000018) |
420995800.2 ns (1557524.949999988) |
1.01 |
tpch_q10/vortex-file-uncompressed |
417246892.3 ns (2200325.544999987) |
414439327.55 ns (1758985.7400000095) |
1.01 |
tpch_q11/vortex-in-memory-no-pushdown |
178520357.070873 ns (517703.021893844) |
177768156.54376984 ns (678823.5634290576) |
1.00 |
tpch_q11/vortex-in-memory-pushdown |
176255722.45710316 ns (815129.5631150901) |
177281814.48579365 ns (840664.9464464337) |
0.99 |
tpch_q11/arrow |
174407314.84162697 ns (743061.9987500012) |
175179317.98261905 ns (736402.2469047755) |
1.00 |
tpch_q11/parquet |
183258577.37547618 ns (719813.7152261883) |
184554186.90000004 ns (938321.1112500131) |
0.99 |
tpch_q11/vortex-file-compressed |
263486006.55 ns (2082622.884374991) |
264731475.4 ns (2912306.392499998) |
1.00 |
tpch_q11/vortex-file-uncompressed |
263639380.95 ns (1510587.8349999934) |
266718340.25 ns (2175068.128124997) |
0.99 |
tpch_q12/vortex-in-memory-no-pushdown |
238897157.46666664 ns (497755.7041666955) |
230191016.7333333 ns (515290.7612499893) |
1.04 |
tpch_q12/vortex-in-memory-pushdown |
253141895.4 ns (793822.2406250089) |
249369825.4 ns (908228.575000003) |
1.02 |
tpch_q12/arrow |
187192726.63333336 ns (782580.1241666675) |
186193101.20000002 ns (357079.9037500024) |
1.01 |
tpch_q12/parquet |
337053524.2 ns (827132.8137499988) |
335763480.1 ns (825246.9599999785) |
1.00 |
tpch_q12/vortex-file-compressed |
447830217.95 ns (1605042.8431249857) |
444172044.35 ns (1290687.9500000179) |
1.01 |
tpch_q12/vortex-file-uncompressed |
433408359.45 ns (766931.3225000203) |
428021485.6 ns (1647048.1887499988) |
1.01 |
tpch_q13/vortex-in-memory-no-pushdown |
177270742.18892857 ns (2228953.232297614) |
176089729.1199603 ns (1400648.6520714462) |
1.01 |
tpch_q13/vortex-in-memory-pushdown |
172907429.10210317 ns (1721267.9609300643) |
174621087.4619841 ns (1546067.799839288) |
0.99 |
tpch_q13/arrow |
169405765.6479365 ns (1954533.8880158812) |
170528597.70515874 ns (2122133.1986011714) |
0.99 |
tpch_q13/parquet |
329468079.25 ns (1731675.8493749797) |
326697221.65 ns (2131685.4575000107) |
1.01 |
tpch_q13/vortex-file-compressed |
202696231.23333332 ns (1126612.0833333433) |
198006708.9 ns (900881.2500000149) |
1.02 |
tpch_q13/vortex-file-uncompressed |
201740333.93333334 ns (1608940.443749994) |
203018926.96666667 ns (1555533.7229166627) |
0.99 |
tpch_q14/vortex-in-memory-no-pushdown |
43758461.62723545 ns (392162.9830952436) |
43995271.40107144 ns (431898.5514880903) |
0.99 |
tpch_q14/vortex-in-memory-pushdown |
73983657.30075397 ns (356675.1773164645) |
73861513.73878969 ns (279046.1784682572) |
1.00 |
tpch_q14/arrow |
34866360.17189153 ns (384259.3683998026) |
35005566.83378307 ns (544230.4778174572) |
1.00 |
tpch_q14/parquet |
226921362.9 ns (688855.853333354) |
225298053.8 ns (458414.78874997795) |
1.01 |
tpch_q14/vortex-file-compressed |
118216529.79654762 ns (508508.4525491148) |
122325305.66412696 ns (957488.1992460266) |
0.97 |
tpch_q14/vortex-file-uncompressed |
134947078.1400397 ns (749224.5644380003) |
135371978.90940478 ns (710691.7249345183) |
1.00 |
tpch_q15/vortex-in-memory-no-pushdown |
71759328.26936507 ns (386141.04746031016) |
71386459.67698415 ns (451511.19861903787) |
1.01 |
tpch_q15/vortex-in-memory-pushdown |
103914839.23436508 ns (413156.8720238209) |
102943096.1073016 ns (535685.1348948553) |
1.01 |
tpch_q15/arrow |
56100014.32390873 ns (251544.594143603) |
56840715.733134925 ns (347459.639961306) |
0.99 |
tpch_q15/parquet |
298075914.4 ns (1223236.832499981) |
299025033.15 ns (748578.5637500286) |
1.00 |
tpch_q15/vortex-file-compressed |
240026753.86666664 ns (883615.1741666645) |
241549942.5 ns (746663.4691666663) |
0.99 |
tpch_q15/vortex-file-uncompressed |
273306300.9 ns (1276728.6743749976) |
274840948 ns (1046676) |
0.99 |
tpch_q16/vortex-in-memory-no-pushdown |
105667355.79626983 ns (381584.5176706314) |
109636016.60361111 ns (942274.3052777648) |
0.96 |
tpch_q16/vortex-in-memory-pushdown |
118245168.3420635 ns (205878.0737113133) |
120313284.79809526 ns (780756.9583333209) |
0.98 |
tpch_q16/arrow |
104806115.44507937 ns (709693.9593353122) |
107158909.03246032 ns (684914.3588888943) |
0.98 |
tpch_q16/parquet |
115322338.37297618 ns (544759.1422142908) |
117999241.89964285 ns (273186.118026793) |
0.98 |
tpch_q16/vortex-file-compressed |
124954812.87785713 ns (232542.0109077394) |
131117100.82833333 ns (518181.4666666612) |
0.95 |
tpch_q16/vortex-file-uncompressed |
130258392.24936506 ns (624478.2398948371) |
131588242.50603175 ns (683042.7460674718) |
0.99 |
tpch_q17/vortex-in-memory-no-pushdown |
540125577.7 ns (4836323.5) |
541022795.3 ns (15753799.300000012) |
1.00 |
tpch_q17/vortex-in-memory-pushdown |
616233765.4 ns (8543599) |
614427463.4 ns (11087474.991249979) |
1.00 |
tpch_q17/arrow |
520097404.6 ns (7158556.311250001) |
519226655.6 ns (9341847.721249998) |
1.00 |
tpch_q17/parquet |
642750784.2 ns (3961809.19750005) |
639609082.9 ns (3548323.3887500763) |
1.00 |
tpch_q17/vortex-file-compressed |
622332776 ns (5554417.206250012) |
630376411.5 ns (3842823.7174999714) |
0.99 |
tpch_q17/vortex-file-uncompressed |
610303856 ns (5431913.201250017) |
614690871.9 ns (4024374.5) |
0.99 |
tpch_q18/vortex-in-memory-no-pushdown |
1076670211 ns (6797084.2062500715) |
1067953455.2 ns (5622745.899999976) |
1.01 |
tpch_q18/vortex-in-memory-pushdown |
1061137484.6 ns (5128788.47875005) |
1062575542.5 ns (9149481.186250031) |
1.00 |
tpch_q18/arrow |
1067069222.1 ns (4499243.725000024) |
1064386538.3 ns (3296407.2375000715) |
1.00 |
tpch_q18/parquet |
1222777601.5 ns (9554642.133750081) |
1222627123.6 ns (4869863.181250095) |
1.00 |
tpch_q18/vortex-file-compressed |
1149554294.2 ns (3992946.3499999046) |
1117678411 ns (7137087.849999905) |
1.03 |
tpch_q18/vortex-file-uncompressed |
1093749041.4 ns (5716855.272499919) |
1087651370.5 ns (4801718.551249981) |
1.01 |
tpch_q19/vortex-in-memory-no-pushdown |
179605762.08253965 ns (431240.237771824) |
183105953.53333333 ns (349018.7841666937) |
0.98 |
tpch_q19/vortex-in-memory-pushdown |
248320569.23333335 ns (477435.1466666609) |
249583400.3666667 ns (861001.0087500066) |
0.99 |
tpch_q19/arrow |
164344860.65892857 ns (440176.85877381265) |
164943269.94273812 ns (316730.15093305707) |
1.00 |
tpch_q19/parquet |
454366474.65 ns (872369.2381250262) |
453879642.6 ns (941293.3368750215) |
1.00 |
tpch_q19/vortex-file-compressed |
446240188.05 ns (2157446.525000006) |
441962979.3 ns (2793428.878125012) |
1.01 |
tpch_q19/vortex-file-uncompressed |
413381236.7 ns (2076644.5881250203) |
408758113.5 ns (2294387.431250006) |
1.01 |
tpch_q20/vortex-in-memory-no-pushdown |
242413892.8333333 ns (1592507.134166658) |
253734775.55 ns (1977479.5268750042) |
0.96 |
tpch_q20/vortex-in-memory-pushdown |
262333924.85 ns (2727057.790624991) |
265195640.8 ns (2394636.3081249893) |
0.99 |
tpch_q20/arrow |
234960823.4666667 ns (1373721.5133333355) |
240602142.6333333 ns (2330877.4987499863) |
0.98 |
tpch_q20/parquet |
354528307.6 ns (1296847.849999994) |
356190900.5 ns (1882446.7668749988) |
1.00 |
tpch_q20/vortex-file-compressed |
366249878.8 ns (3418929.237500012) |
365537942.1 ns (1882865.3524999917) |
1.00 |
tpch_q20/vortex-file-uncompressed |
374440581.65 ns (2304314.6799999774) |
376023028.65 ns (2500746.938749969) |
1.00 |
tpch_q21/vortex-in-memory-no-pushdown |
882031083.5 ns (4410312.544999957) |
880308554.5 ns (4760410.742500007) |
1.00 |
tpch_q21/vortex-in-memory-pushdown |
904297830.8 ns (5797759.943750024) |
902299720.9 ns (4950253.931250036) |
1.00 |
tpch_q21/arrow |
849979612 ns (5117954.412500024) |
854936445.5 ns (3980942.9862499833) |
0.99 |
tpch_q21/parquet |
972985498.3 ns (3648476.9599999785) |
971147870.7 ns (4416560.558749974) |
1.00 |
tpch_q21/vortex-file-compressed |
1157335147.9 ns (9962618.147499919) |
1153845450.5 ns (4266209.518749952) |
1.00 |
tpch_q21/vortex-file-uncompressed |
1144070474.4 ns (6006278.747499943) |
1138981817.8 ns (6960775.726249933) |
1.00 |
tpch_q22/vortex-in-memory-no-pushdown |
76645742.63325396 ns (267921.0321309492) |
76428218.86363097 ns (323328.4835736528) |
1.00 |
tpch_q22/vortex-in-memory-pushdown |
76478821.67357144 ns (230200.42927678674) |
75637256.04857144 ns (204647.15291666985) |
1.01 |
tpch_q22/arrow |
74953995.52476189 ns (139045.53338392824) |
74615386.48684523 ns (163933.9377380982) |
1.00 |
tpch_q22/parquet |
93315142.06376985 ns (329506.84411656857) |
92057154.83924603 ns (594755.854478173) |
1.01 |
tpch_q22/vortex-file-compressed |
116825201.15595236 ns (506739.16130952537) |
118218258.26976189 ns (301813.7342946455) |
0.99 |
tpch_q22/vortex-file-uncompressed |
115152960.29047617 ns (322314.09033630043) |
115622304.98892859 ns (467586.2596487999) |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Vortex Compression
| Benchmark suite | Current: b72b1fb | Previous: af290fc | Ratio |
|---|---|---|---|
compress time/taxi |
1787599830 ns (6504149.518749952) |
1852900028 ns (6473768.674999952) |
0.96 |
compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd compress time/taxi |
1810922580.4 ns (6836711.961250067) |
1877386086.9 ns (4196133.518749952) |
0.96 |
parquet_rs-zstd compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
decompress time/taxi |
463142272.05 ns (5039689.074999988) |
465841654.65 ns (5376231.049999982) |
0.99 |
decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd decompress time/taxi |
334894267.6 ns (1377551.6587499678) |
346792239.4 ns (2454667.724999994) |
0.97 |
parquet_rs-zstd decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
vortex:parquet-zstd size/taxi |
1.0830146311320368 ratio |
1.0390993268324744 ratio |
1.04 |
vortex:raw size/taxi |
0.12872790830957145 ratio |
0.12350890783030273 ratio |
1.04 |
vortex size/taxi |
60606248 bytes |
58149096 bytes |
1.04 |
compress time/AirlineSentiment |
889126.6719798474 ns (1432.0274023693055) |
875539.2473065477 ns (1583.6764234203729) |
1.02 |
compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd compress time/AirlineSentiment |
57863.50546864286 ns (363.4479830191485) |
56724.94239820457 ns (50.70282864126057) |
1.02 |
parquet_rs-zstd compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
decompress time/AirlineSentiment |
45081.2784577573 ns (84.46104407420353) |
44665.686922212066 ns (78.21100454114276) |
1.01 |
decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd decompress time/AirlineSentiment |
34321.11686297589 ns (94.48760696719546) |
33337.660411390745 ns (38.80686999772297) |
1.03 |
parquet_rs-zstd decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
vortex:parquet-zstd size/AirlineSentiment |
9.042399172699069 ratio |
9.042399172699069 ratio |
1 |
vortex:raw size/AirlineSentiment |
4.328712871287129 ratio |
4.328712871287129 ratio |
1 |
vortex size/AirlineSentiment |
8744 bytes |
8744 bytes |
1 |
compress time/Arade |
3815788961.3 ns (8916392.5087502) |
3882397198.4 ns (15198750.482499838) |
0.98 |
compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd compress time/Arade |
3109342046.5 ns (8418179.558750153) |
3342218867 ns (22190957.350000143) |
0.93 |
parquet_rs-zstd compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
decompress time/Arade |
724383836.3 ns (3938713.004999995) |
751039760.6 ns (7217271.245000005) |
0.96 |
decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd decompress time/Arade |
722473041.5 ns (4262805.348749995) |
760566223.9 ns (6254295.653750002) |
0.95 |
parquet_rs-zstd decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
vortex:parquet-zstd size/Arade |
0.517090033186272 ratio |
0.5033079295106381 ratio |
1.03 |
vortex:raw size/Arade |
0.20064692328983816 ratio |
0.19530149890265067 ratio |
1.03 |
vortex size/Arade |
157913896 bytes |
153706920 bytes |
1.03 |
compress time/Bimbo |
15481658840.5 ns (39142189.723750114) |
20689921314.8 ns (160920342.65374947) |
0.75 |
compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd compress time/Bimbo |
21953323789.4 ns (156795254.94999886) |
23998902974.9 ns (81974147.1637516) |
0.91 |
parquet_rs-zstd compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
decompress time/Bimbo |
4913982559.9 ns (22239798.951250553) |
5569028308.8 ns (36961456.731250286) |
0.88 |
decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd decompress time/Bimbo |
4125490626.3 ns (21577728.996249914) |
5085578759 ns (37208689.722500324) |
0.81 |
parquet_rs-zstd decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
vortex:parquet-zstd size/Bimbo |
1.3758515662088904 ratio |
1.445633239046137 ratio |
0.95 |
vortex:raw size/Bimbo |
0.07499130772360806 ratio |
0.07879478407943727 ratio |
0.95 |
vortex size/Bimbo |
534038120 bytes |
561123944 bytes |
0.95 |
compress time/CMSprovider |
18647159807.7 ns (48521157.02875137) |
19742297398.7 ns (117016075.22249985) |
0.94 |
compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd compress time/CMSprovider |
20652748722.7 ns (51249333.79375076) |
22417341798.4 ns (101832125.1012516) |
0.92 |
parquet_rs-zstd compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
decompress time/CMSprovider |
7179183962.2 ns (25980557) |
6611003261.6 ns (50127905.77374983) |
1.09 |
decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd decompress time/CMSprovider |
5913941747 ns (19694393.18874979) |
6564990926.3 ns (42526687.787499905) |
0.90 |
parquet_rs-zstd decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
vortex:parquet-zstd size/CMSprovider |
1.2403441296386775 ratio |
1.2509461401621398 ratio |
0.99 |
vortex:raw size/CMSprovider |
0.18536400651316695 ratio |
0.18694802120324328 ratio |
0.99 |
vortex size/CMSprovider |
954462248 bytes |
962618536 bytes |
0.99 |
compress time/Euro2016 |
2818181433.5 ns (6030405.091250181) |
3319840536.7 ns (5195315.376250029) |
0.85 |
compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd compress time/Euro2016 |
1599600611.6 ns (1931916.7474999428) |
1679229005.7 ns (4728128.488749981) |
0.95 |
parquet_rs-zstd compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
decompress time/Euro2016 |
415000040.85 ns (1782508.275000006) |
350616869.25 ns (2588359.8631249964) |
1.18 |
decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd decompress time/Euro2016 |
502214299 ns (1695725.900000006) |
541641053.5 ns (7467047.75) |
0.93 |
parquet_rs-zstd decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
vortex:parquet-zstd size/Euro2016 |
1.4901589532916901 ratio |
1.4512970463242234 ratio |
1.03 |
vortex:raw size/Euro2016 |
0.4505108427325507 ratio |
0.4387619548575802 ratio |
1.03 |
vortex size/Euro2016 |
177164840 bytes |
172544552 bytes |
1.03 |
compress time/Food |
1651776019 ns (4697749.973749876) |
1695892996 ns (8940631.879999995) |
0.97 |
compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd compress time/Food |
1130441595.9 ns (1257786.1012499332) |
1187962203.1 ns (6661061.254999876) |
0.95 |
parquet_rs-zstd compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
decompress time/Food |
235178218.83333334 ns (785844.5912500024) |
221167005.8 ns (2213518.0666666925) |
1.06 |
decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd decompress time/Food |
233056816.66666666 ns (354910.2666666657) |
245724732.36666664 ns (1136420.0537500083) |
0.95 |
parquet_rs-zstd decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
vortex:parquet-zstd size/Food |
1.6437883144011016 ratio |
1.315371098541974 ratio |
1.25 |
vortex:raw size/Food |
0.17899333072009108 ratio |
0.14323165924281234 ratio |
1.25 |
vortex size/Food |
59554344 bytes |
47655784 bytes |
1.25 |
compress time/HashTags |
3177373552.2 ns (4296862.789999962) |
3245604444.2 ns (9704047.825000048) |
0.98 |
compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd compress time/HashTags |
2527098581.6 ns (4321621.942499876) |
2678352332.1 ns (10982505.856250048) |
0.94 |
parquet_rs-zstd compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
decompress time/HashTags |
661037858.6 ns (3026307.300000012) |
627720500.7 ns (5394146.713750005) |
1.05 |
decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd decompress time/HashTags |
839919175.8 ns (10474188.006250024) |
936707888.4 ns (10514493.796249986) |
0.90 |
parquet_rs-zstd decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
vortex:parquet-zstd size/HashTags |
1.5909971685396191 ratio |
1.7353314060012677 ratio |
0.92 |
vortex:raw size/HashTags |
0.26493453304092185 ratio |
0.2889693098529743 ratio |
0.92 |
vortex size/HashTags |
213138664 bytes |
232474536 bytes |
0.92 |
compress time/TPC-H l_comment chunked without fsst |
4439879281.6 ns (40743423.617500305) |
5141461573.2 ns (50561490.549999714) |
0.86 |
compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst |
915846629.8 ns (1648308.4075000286) |
944935260.9 ns (2770896.3149999976) |
0.97 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked without fsst |
113618245.14023809 ns (623466.4886309505) |
115517285.8298016 ns (1207647.2349379957) |
0.98 |
decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst |
259531211.05 ns (1171789.376249984) |
257500115.45 ns (878486) |
1.01 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked without fsst |
4.609640919691308 ratio |
4.609683221378485 ratio |
1.00 |
vortex:raw size/TPC-H l_comment chunked without fsst |
1.0532105330764496 ratio |
1.0532138717992252 ratio |
1.00 |
vortex size/TPC-H l_comment chunked without fsst |
262457000 bytes |
262457832 bytes |
1.00 |
compress time/TPC-H l_comment chunked |
1934578830.4 ns (8454666.900000095) |
1391738946.4 ns (11802452.669999957) |
1.39 |
compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked |
909466444.8 ns (1684909.7275000215) |
936046688.8 ns (3891402.3750000596) |
0.97 |
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked |
143373909.39849207 ns (1723087.4719474316) |
130533347.56115082 ns (1598685.1166527793) |
1.10 |
decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked |
262149622.7 ns (1649254.2637500018) |
259723031.35 ns (1499274.6518750042) |
1.01 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked |
1.5448887049492066 ratio |
1.352371462393247 ratio |
1.14 |
vortex:raw size/TPC-H l_comment chunked |
0.35297609614943737 ratio |
0.30898790993105096 ratio |
1.14 |
vortex size/TPC-H l_comment chunked |
87960616 bytes |
76998888 bytes |
1.14 |
compress time/TPC-H l_comment canonical |
1964679179.5 ns (3325675.09375) |
1397408904.65 ns (5462976.174999952) |
1.41 |
compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment canonical |
913613664.2 ns (2861101.425000012) |
944932510.4 ns (2837475.473124981) |
0.97 |
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
decompress time/TPC-H l_comment canonical |
140538613.37446427 ns (698003.5613095313) |
130965288.73747353 ns (1667422.3400798663) |
1.07 |
decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical |
256623011.32371035 ns (675653.522019878) |
261326303.98117065 ns (1474004.2798134983) |
0.98 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment canonical |
1.5448946743466152 ratio |
1.3523699422412019 ratio |
1.14 |
vortex:raw size/TPC-H l_comment canonical |
0.3529760734861825 ratio |
0.3089878900921105 ratio |
1.14 |
vortex size/TPC-H l_comment canonical |
87960616 bytes |
76998888 bytes |
1.14 |
This comment was automatically generated by workflow using github-action-benchmark.
lwwmanning
changed the title
lwwmanning
force-pushed
the
wm/scanperf
branch
from
093a9b2
to
eddfe59
Compare
lwwmanning
added a commit
that referenced
this pull request
Nov 8, 2024
No real changes, just moving code around to make the diff on #1068 smaller. The `SamplingCompressor` from `lib.rs` into its own file. Additionally, the previously standalone `objective_function` is now `Objective::evaluate`
lwwmanning
force-pushed
the
wm/scanperf
branch
from
eddfe59
to
b72b1fb
Compare
robert3005
reviewed
Nov 11, 2024
lwwmanning
force-pushed
the
wm/scanperf
branch
from
9c10d08
to
39cdcf3
Compare
lwwmanning
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
new objective function that biases more strongly towards the fastest codecs, while still taking compressed size into account