Web service that returns a JSON description of a *.StormReplay file from Blizzard's game Heroes of the Storm. Developed by Karl Gluck.
This code is built to be deployed to a Heroku instance, but can be run locally with Ubuntu on either a virtual or a dedicated machine.
This project is based on code from:
It uses the following technologies:
- Python programming language
- Django web framework
- Celery distributed task queue
- Redis key-value data store
- Heroku cloud application platform
- Amazon S3 for production API replay file & result storage
- Boto
Special thanks to Ben, the admin of HOTS Logs, for helping me figure out the GameEvents binary format!
This code is insecure because it is in Django developer mode. I'm still developing this library so I don't have the production configuration set up. If you want to get started right now, fix this yourself before deploying anywhere public.
This server is intended to run on Linux and was developed on Ubuntu 14.04.1 LTS. I have gotten parts of it to run on Windows but it requires some tinkering and a lot of installed apps. It's much easier to just develop in a virtual machine with Ubuntu and VMware Player. VMware Player is free, by the way. The 'trial' only applies if you activate pro features, which we don't need.
To get started, grab a copy of this repository:
git clone https://github.com/karlgluck/heroes-of-the-storm-replay-parser.git
If you haven't already, install the Heroku Toolbelt and get a Redis server running.
Follow these instructions on Heroku to set up your environment for a local copy of a Python server.
Note: When running pip install
on Ubuntu for the first time, you may run into an error with psycops. Run sudo apt-get install libpq-dev python-dev
to fix it.
Wait for installation to complete. Start up the Redis server in another terminal:
redis-server
Note: The first time you start redis-server
, it may complain about two things that will affect latency. Fix them if it does:
- vm.overcommit_memory should be 1
- Transparent huge pages must be disabled
Once Redis is running, launch a Celery worker:
foreman start worker
Next, launch the replay parser web server:
foreman start web
If you visit http://localhost:5000
with your browser, the app will now be running. If you're running inside a virtual machine like I suggested, only the browser inside the VM will resolve localhost. It is possible to access the server from your host computer by using ifconfig
to find the VM's ip address and typing [ip]:5000
into the host's browser's address bar.
You will need to set the following environment variables. In Heroku you can set them as 'config vars' in the panel or with the Toolbelt command heroku config set VAR=VALUE
. For local deployment, Foreman can read these from the .env
file in the root of your project. Do not check in the .env
file!
AWS_ACCESS_KEY_ID=<Amazon Web Services user access key>
AWS_SECRET_ACCESS_KEY=<Amazon Web Services user secret signing key>
AWS_BUCKET_NAME=<Your bucket on Amazon S3>
The bucket you create for storing replays should have the following CORS configuration:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
The access and secret keys from above come from an Amazon user that you create for this project. DO NOT use your Amazon account root credentials and NEVER add these values to any public repository (don't check in your .env
file, for example). Bad things that cost you a lot of money can happen if someone gets ahold of your secret key. You have been warned!
The user needs to have permissions to access your bucket. To do this:
- Go to your Security Credentials on AWS
- Go into the IAM users panel
- Create a new user
- Download a copy of your user's access credentials to get the access key ID and secret access key
- Hit Attach User Policy
- Select the Policy Generator
- On the Edit Permissions page, pick
Allow
,AWS Service
=Amazon S3
, and check the actions:GetObject
GetObjectAcl
PutObject
PutObjectAcl
- For
Amazon Resource Name
enterarn:aws:s3:::<bucket>/*
with<bucket>
replaced by the name of your bucket - Save
- Redo API design and make its operation clear / not hard-coded
- Document global environment variables
- API keys for request rate limiting
- Protocol version 29666 (circa April 2014)
- Protocol version 30027 (circa May 2014)
- Protocol version 31566 (circa August 2014)
- Protocol version 31948 (circa September 2014)
- Protocol version 32524 (circa October 2014)