Releases: medcl/esm
Releases · medcl/esm
v0.7.0
v0.6.1
v0.6.0
v0.5.0
- Add buffer_count to control memory consumed by ESM
- Speedup, several performance optimization
A 3 nodes cluster(3 * c5d.4xlarge， 16C，32GB，10Gbps)
./esm -s https://localhost:8000 -d https://localhost:8000 -x kibana_sample_data_logs -y logs-test -m elastic:medcl123 -n elastic:medcl123 --regenerate_id --repeat_times=5
./esm -s https://localhost:8000 -d https://localhost:8000 -x logs-test -y logs-test1 -m elastic:medcl123 -n elastic:medcl123 --regenerate_id -w 20 --sliced_scroll_size=20 -b 20 --repeat_times=500
./esm -s https://localhost:8000 -d https://localhost:8000 -x logs-test1 -y logs-test -m elastic:medcl123 -n elastic:medcl123 --regenerate_id -w 20 --sliced_scroll_size=20 -b 20 --repeat_times=500
[12-19 06:29:40] [INF] [main.go:537,main] data migration finished.
root@ip-172-31-13-181:/tmp# ./esm -s https://localhost:8000 -d https://localhost:8000 -x logs1kw -y logs122 -m elastic:medcl123 -n elastic:medcl123 -w 40 --sliced_scroll_size=60 -b 5 --buffer_count=2000000 --regenerate_id
[12-19 06:31:20] [INF] [main.go:506,main] start data migration..
Scroll 10064570 / 10064570 [============================================] 100.00% 55s
Bulk 10062602 / 10064570 [=============================================] 99.98% 55s
[12-19 06:32:15] [INF] [main.go:537,main] data migration finished.
Migrate 10,000,000 documents within 1 minute (Nginx log generated from kibana_sample_data_logs).
v0.4.5
v0.4.4
0.4.4_SNAPSHOT
0.4.4_SNAPSHOT