Hadoop and Ansible
Here, we will learn how to configure a hadoop cluster with the help of Ansible playbook.
Problem Statement
🔰 11.1 Configure Hadoop and start cluster services using Ansible Playbook
Solution Steps
Step1:- Setup ansible inventory and the Ansible configuration file.
Step 2:- Edit core-site.xml and hdfs-site.xml file for the hadoop cluster configuration.
For Master
For Slave
Here, I will configure one master and one slave. My master IP is 192.168.43.46
Step 3:- Create playbooks for master and slave node as follows:-
For Master:-
For Slave:-
In my case , I am configuring master as localhost, i.e my controller node of Ansible.
Step 4:- Now run the playbooks with the command as:
ansible-playbook playbook.yml
For Master:-
Using the jps command we check whether the cluster is working or not. Here my name node is working fine.
For Slave:-
Our datanode is also working fine.
Step 5:- To check that the setup is working fine, use the command:
hadoop dfsadmin -report
Hence, my hadoop cluster is successfully deployed using Ansible.
