The setup is the one described here.
Edit the hosts.ini file by adding the hostnames or IPs of the cluster's machines under the group flink
.
Use the flink_type
variable to specify which node will be master
(JobManager) and set the others to slave
(TaskManager).
If you want you can set all the nodes to slaves
, and execute the JobManager locally to your PC.
Next, go ahead and edit the default values in the flink file.
Run the playbook with:
ansible-playbook -i hosts.ini site.yml
Visit the master
host at the port 8081 to access the Flink's web UI. If the cluster is running properly, you can then submit tasks for execution.
This setup is not highly available, meaning that if the JobManager goes down, the TaskManagers will continue the execution but no new tasks can be assigned and no monitoring is possible.
Normally, a highly available setup contains a quorum of JobManagers, which requires Zookeeper.
Flink runs as a service with systemd
, and by default it is enabled
on system startup. Edit the corresponding variables if you don't want such behavior.
Currently only deb distributions are supported (ubuntu, debian), with systemd
.
Ansible version >= 2.2 and jmespath
python package.
This repo is now quite outdated. I do not plan to update it.