Skip to content

Retrieve all historical candlestick data from crypto exchange Binance and upload it to Kaggle.

License

Notifications You must be signed in to change notification settings

aliel/candlestick_retriever

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

46 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

candlestick_retriever

Retrieve all historical candlestick data from crypto exchange Binance and upload it to Kaggle.

Dependencies

  • pandas
  • requests
  • pyarrow
  • kaggle

Usage

./main.py --help

usage: main.py [-h] [-dt DTYPE] [-p PAIRS] [-i INTERVAL] [--limit LIMIT] [--start-at START_AT] [--parquet PARQUET] [--upload UPLOAD] [--key KEY] [--secret SECRET] [--timeout TIMEOUT]
               [--check-trade CHECK_TRADE]

Download historical candlestick data for all available trading pairs and historical trades, @see https://github.com/binance/binance-spot-api-docs/blob/master/rest-api.md to understand related parameters

optional arguments:
  -h, --help            show this help message and exit
  -dt DTYPE, --dtype DTYPE
                        Provide which data to fetch candle|trade (default candle)
  -p PAIRS, --pairs PAIRS
                        Provide pair names to fetch (default all)
  -i INTERVAL, --interval INTERVAL
                        Set time frame interval (default 1m) only if type is candle
  --limit LIMIT         Set max results returned per request (default 1000)
  --start-at START_AT   Set start time or index according to data type (default 0)
  --parquet PARQUET     Build parquet as well (default False)
  --upload UPLOAD       Upload parquet to kaggle (default False)
  --key KEY             Binance api key
  --secret SECRET       Binance secret key
  --timeout TIMEOUT     Stop waiting for a response from Binance after (x) Seconds (default 30)
  --check-trade CHECK_TRADE
                        Check trade consistency (provide a csv filepath)

Running

Get all pairs with default interval value (1m)

Simply run ./main.py to either download or update every single pair available:

Get single pair with specific interval

./main.py --pairs 'ETH-USDT' --interval 1d

Get more pairs

./main.py --pairs 'ETH-USDT, ETH-BTC, ADA-USDT' --interval 1d

[...]
2020-08-22 17:44:24.178846 959/970 Wrote 83000 new lines to file for ETH-USDT 
2020-08-22 17:45:13.963455 960/970 Wrote 83000 new lines to file for ETH-BTC 
2020-08-22 17:45:14.573595 961/970 Already up to date with ADA-USDT
2020-08-22 17:46:06.781870 962/970 Wrote 83000 new lines to file for ETH-BTC 
[...]

Once that is completed you should end up with a directory with a Parquet file for each pair, currently 970 files totaling ~12GB if --parquet arguments is set.

Add --upload if you want to upload parquet files to kaggle

About

Retrieve all historical candlestick data from crypto exchange Binance and upload it to Kaggle.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%