This is a proof of concept for using blockchain incentive to obtain AI data.
You can test entire flow from obtaining data for AI training in exchange for tokens on a ethereum private network to train model to serve service.
This example is for voice recognition system.
you can try it without blockchain parts if you want
For more information about the idea, please check website or blog.
- Import texts from epub for data contributors to read out
- Data contributors can record voice. the voice and meta data is saved on mongodb.
- Issue token on private ethereum network in exchange for data
- Train AI model on jupyter notebook
- Test the model at webstie
For client side scripts, please check here
Here's a live demo This is only for testing purpose.
mkdir workspace
cd workspace/
geth (optional only when you want to issue token)
How to install geth
mkdir privatenet
cd privatenet
geth account new --datadir ./data
> <address #1>
geth account new --datadir ./data
> <address #2>
geth account new --datadir ./data
> <address #3> (optional)
puppeth is a tool to generate a json file to create a genesis block.
puppeth
+-----------------------------------------------------------+
| Welcome to puppeth, your Ethereum private network manager |
| |
| This tool lets you create a new Ethereum network down to |
| the genesis block, bootnodes, miners and ethstats servers |
| without the hassle that it would normally entail. |
| |
| Puppeth uses SSH to dial in to remote servers, and builds |
| its network components out of Docker containers using the |
| docker-compose toolset. |
+-----------------------------------------------------------+
Please specify a network name to administer (no spaces, please)
> ratel_network_demo
What would you like to do? (default = stats)
1. Show network stats
2. Configure new genesis
3. Track new remote server
4. Deploy network components
> 2
Which consensus engine to use? (default = clique)
1. Ethash - proof-of-work
2. Clique - proof-of-authority
> 2
How many seconds should blocks take? (default = 15)
> 5
Which accounts are allowed to seal? (mandatory at least one)
> 0x<address #1>
> 0x
Which accounts should be pre-funded? (advisable at least one)
> 0x<address #1>
> 0x<address #2>
> 0x
Specify your chain/network ID if you want an explicit one (default = random)
>
Anything fun to embed into the genesis block? (max 32 bytes)
>
What would you like to do? (default = stats)
1. Show network stats
2. Manage existing genesis
3. Track new remote server
4. Deploy network components
> 2
1. Modify existing fork rules
2. Export genesis configuration
> 2
Which file to save the genesis into? (default = ratel_network_demo.json)
>
geth --datadir ./data init ratel_network_demo.json
RPC should be running at 127.0.0.1.
0.0.0.0 is only for testing purpose.
nohup geth --datadir ./data --nodiscover --networkid 60263 --rpc --rpcapi db,eth,net,web3,personal,miner,admin --cache=128 --rpcport 8545 --rpcaddr 0.0.0.0 --rpccorsdomain "*" &
geth attach http://127.0.0.1:8545
web3.personal.unlockAccount(web3.eth.accounts[0], <your password>, 0)
miner.start()
At workspace directory,
git clone https://github.com/kouohhashi/ratel_voice_server_example.git
cd workspace/ratel_voice_server_example/smart_contract
npm -g install truffle
truffle compile
truffle migrate
Save RatelExampleToken smart contract address and Custodian smart contract address.
RatelExampleToken: <RatelExampleToken address>
Custodian: <Custodian address>
Running migration: 1_initial_migration.js
Deploying Migrations...
... 0x767ca90172567755e77a71e4934b428103035208688bc393f35bd1c92d0e5bda
Migrations: 0x5d3dc37050617946a3614863f68a66d2ab44562e
Saving successful migration to network...
... 0xcb336a73b014a22234f30e59f6c871483ed2faac6af5879685b16bb1c1301e93
Saving artifacts...
Running migration: 2_deploy_contracts.js
Deploying RatelExampleToken...
... 0xfa7a2660df71725f76696201fa31b06e81cf16e324e2a13b609e2201b087c764
RatelExampleToken: <RatelExampleToken address>
Deploying Custodian...
... 0xd6c257d9fd838c349b10629d3387556421df3ad881250b5dab18d4e600eecb07
Custodian: <Custodian address>
Saving successful migration to network...
... 0x0db5e1e06a842ad4a029e8cf5900b8043ee60497f0370b854156f65043ad623d
Saving artifacts...
- open remix.ethereum.org
- change rpc server address to your server.
- load RatelExampleToken and Custodian by at addresses above.
- execute transferCustodian method to update Custodian address.
- execute transferMinter to update oracle address.
- switch address to oracle address.
- unlock oracle address by
web3.personal.unlockAccount(web3.eth.accounts[1], 'your password')
- execute mint method to mint enough tokens from oracle address. (example: 1,000,000,000 tokens)
by the way, at current test environment, anyone who knows your RPC server address can access your blockchain server. This is not secure at all. we need to run at least 2 nodes and RPC node should not be an oracle or a sealer.
mongodb
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
sudo apt-get update
sudo apt-get install mongodb-org
sudo service mongod start
anaconda
cd /tmp/
curl -O https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh
bash Anaconda3-5.0.1-Linux-x86_64.sh
avconv
sudo apt install libav-tools
tensorflow
pip install tensorflow
pip install -r requirements.txt
requirements.txt is located at workspace/ratel_voice_server_example/jupyter
At workspace directory,
git clone https://github.com/kouohhashi/ratel_voice_server_example.git
cd workspace/ratel_voice_server_example
npm install
cp server/routes/settings_example.js server/routes/settings.js
vi server/routes/settings.js
npm run init_db
npm run server
before train AI model, you need data. you can add data from client. you can test client scripts here
cd jupyter/
conda create --name aind-vui python=3.5 numpy
source activate aind-vui
pip install tensorflow
pip install -r requirements.txt
KERAS_BACKEND=tensorflow python -c "from keras import backend"
jupyter notebook --ip=0.0.0.0 --no-browser
node v8.9.1
- Ethereum blockchain
- Node
- Tensorflow/Keras
- MongoDB
- Should separate blockchain part and node server part.
- Easy Install
- Use distributed storage like IPFS
- Adopt many languages.
- Adopt other no voice recognition AI project
- Feature to share data with other AI projects
MIT. You can do whatever you want.