An Ansible Playbook that installs the Cloudera stack on RHEL/CentOS
- Setup an Ansible Control Machine
- Create Ansible configuration (optional):
$ vi ~/.ansible.cfg
[defaults]
# disable key check if host is not initially in 'known_hosts'
host_key_checking = False
[ssh_connection]
# if True, make ansible use scp if the connection type is ssh (default is sftp)
scp_if_ssh = True- Create Inventory of cluster hosts:
$ vi ~/ansible_hosts
[scm_server]
host1.example.com
[db_server]
host2.example.com
[krb5_server]
host3.example.com
[utility_servers:children]
scm_server
db_server
krb5_server
[gateway_servers]
host4.example.com host_template=HostTemplate-Gateway role_ref_names=HDFS-HTTPFS-1
[master_servers]
host5.example.com host_template=HostTemplate-Master1
host6.example.com host_template=HostTemplate-Master2
host7.example.com host_template=HostTemplate-Master3
[worker_servers]
host8.example.com
host9.example.com
host10.example.com
[worker_servers:vars]
host_template=HostTemplate-Workers
[cdh_servers:children]
utility_servers
gateway_servers
master_servers
worker_serversImportant: fully qualified domain name (FQDN) is mandatory in the ansible_hosts file
- Run playbook
$ ansible-playbook -i ~/ansible_hosts cloudera-playbook/site.yml
-i INVENTORY
inventory host path or comma separated host list (default=/etc/ansible/hosts)Ansible communicates with the hosts defined in the inventory over SSH. It assumes you’re using SSH keys to authenticate so your public SSH key should exist in authorized_keys on those hosts. Your user will need sudo privileges to install the required packages.
By default Ansible will connect to the remote hosts using the current user (as SSH would). To override the remote user name you can specify the --user option in the command, or add the following variables to the inventory:
[all:vars]
ansible_user=ec2-userAWS users can use Ansible’s --private-key option to authenticate using a PEM file instead of SSH keys.
The playbook can install a local MIT KDC and configure Hadoop Security. To enable Hadoop Security:
- Specify the '[krb5_server]' host in the inventory (see above)
- Set 'krb5_kdc_type' to 'MIT KDC' in
group_vars/krb5_server.yml
The playbook uses Cloudera Manager Templates to provision a cluster. As part of the template import process Cloudera Manager applies Autoconfiguration rules that set properties such as memory and CPU allocations for various roles.
If the cluster has different hardware or operational requirements then you can override these properties in group_vars/cdh_servers.
For example:
cdh_services:
- type: hdfs
datanode_java_heapsize: 10737418240
These properties get added as variables to the rendered template's instantiator block and can be referenced from the service configs.
For example roles/cdh/templates/hdfs.j2:
"roleType": "DATANODE",
"configs": [{
"name": "datanode_java_heapsize",
"variable": "DATANODE_JAVA_HEAPSIZE"
}To make integration easier, Gabor Roczei created a dynamic inventory script that allows Ansible to gather data from Cloudera Manager. Its main advantages are:
- Cache management of inventory for better performance
- Cloudera Manager’s HTTP cookie handling
- Support for multiple Cloudera Manager instances
- SSL-friendly, as the root CA check of the Cloudera Manager server can be disabled or enabled
High level architecture of Ansible dynamic inventory vs. Cloudera Managers
Step 1: Configuration of the related Cloudera Manager(s)
$ export CM_URL=https://cm1.example.com:7183,https://cm2.example.com:7183
$ export CM_USERNAME=usernameOther optional configuration parameters:
$ export CM_CACHE_TIME_SEC=3600
$ export CM_DISABLE_CA_CHECK=True
$ export CM_TIMEOUT_SEC=60
$ export CM_DEBUG=FalseNote: We recommend adding these environment variables to the startup file of your shell. For example: $HOME/.bashrc
Step 2: Installation of the git package:
# yum install git
Step 3: Installation of the Ansible package:
# yum install ansible
Step 4: Clone the cloudera-playbook git repository:
$ git clone https://github.com/cloudera/cloudera-playbook
Note: The cloudera-playbook git repository is not officially supported by Cloudera, but the authors recommend using it.
Step 5: Setup the default Ansible inventory and other useful Ansible parameters:
$ vi $HOME/.ansible.cfg
[defaults]
inventory = $HOME/cloudera-playbook/dynamic_inventory_cm
# Do not gather the host information (facts) by default. This can give significant speedups for large clusters.
gathering = explicit
# Disable key check if host is not initially in 'known_hosts'
host_key_checking = False
[ssh_connection]
# If it is True, make ansible use scp if the connection type is ssh (default is sftp)
scp_if_ssh = TrueNote: Update the inventory path of the dynamic_inventory_cm if necessary
Step 6: Change the working directory to cloudera-playbook
$ cd cloudera-playbook
Step 7: The available Cloudera Manager clusters (Ansible groups, such as Cluster_1, Balaton) can be listed with the following command:
$ ./dynamic_inventory_cm --list
Note: The cache of the Cloudera Manager inventory can be refreshed with the following command:
$ ./dynamic_inventory_cm --refresh-cache
Step 8: Setup the SSH public key authentication for remote hosts:
The big advantage of this is that with ad-hoc commands, you do not need to enter your password each time you run the command, but only the first time you enter the private key password.
If the ~/.ssh/id_rsa.pub and ~/.ssh/id_rsa files do not exist, they need to be generated with the ssh-keygen command prior to connecting to the managed hosts.
Launch a subshell with the following command:
$ ssh-agent bash
You will execute the rest of the commands in this How-To article in the subshell.
Add the SSH private key into the SSH authentication agent:
$ ssh-add ~/.ssh/id_rsa
Validate:
$ ssh-add -L
Upload the SSH public key (id_rsa.pub) to the managed hosts:
$ ansible all -m authorized_key -a key="{{ lookup('file', '~/.ssh/id_rsa.pub') }} user=$USER" --ask-pass -u $USER --become-user $USER
For example, you can use the root user:
$ ansible all -m authorized_key -a key="{{ lookup('file', '~/.ssh/id_rsa.pub') }} user=root" --ask-pass -u root
Note: If you do not want to use SSH public key authentication, add the --ask-pass parameter each time you run the Ansible command.
Step 9: Test remote host connectivity (optional):
$ ansible all -m ping -u $USER --become-user $USER
For example, you can execute the command with the root user:
$ ansible all -m ping -u root
Step 10: The ad-hoc command feature enables running single and arbitrary Linux commands on all hosts. You can use this to troubleshoot slow group resolution issues.
The following commands are example ad-hoc commands where Balaton is a group of hosts that is a cluster in Cloudera Manager:
$ ansible Balaton -u $USER --become-user $USER -m command -o -a "time id -Gn $USER"
$ ansible all -u $USER --become-user $USER -m command -o -a "date"
The following example uses the root user:
$ ansible Balaton -m command -o -a "time id -Gn testuser" -u root
$ ansible all -m command -o -a "date" -u root
Further information about dynamic inventory and ad-hoc commands can be found in the Ansible documentation:
Step 1: Edit the default variables in group_vars/all:
krb5_realm: AD.SEC.EXAMPLE.COM
ad_domain: "{{ krb5_realm.lower() }}"
kdc: w2k8-1.ad.sec.example.com
computer_ou: ou=computer_hosts,ou=hadoop_prd,dc=ad,dc=sec,dc=example,dc=com
ldap_group_search_base: OU=groups,OU=hadoop_prd,DC=ad,DC=sec,DC=example,DC=com
ldap_user_search_base: DC=ad,DC=sec,DC=example,DC=com?subtree?(memberOf=CN=hadoop_users,OU=groups,OU=hadoop_prd,DC=ad,DC=sec,DC=example,DC=com)
override_gid: 999999
ad_site: Default-First-Site-Name
Step 2: Enable kerberos on the hosts:
If necessary, update this template file (See the Ansible Templating (Jinja2) documentation for more information):
roles/krb-client/templates/krb5.conf.j2
Run this command to apply it on the managed hosts:
$ ansible-playbook -u root enable_kerberos.yaml
Step 3: Join the host(s) to realm:
If necessary, update these template files (See the Ansible Templating (Jinja2) documentation for more information):
roles/realm/join/templates/sssd.conf.j2
roles/realm/join/templates/realmd.conf.j2
roles/realm/join/templates/nscd.conf.j2
Run this command to apply it on all managed hosts:
$ ansible-playbook -u root realm_join.yaml
bind user: administrator
bind password:
Run this command to apply it on a cluster (for example: Balaton) (See the Ansible Best Practices documentation for more information):
$ ansible-playbook --limit Balaton -u root realm_join.yaml
bind user: administrator
bind password:
Remove all hosts from the realm with this command:
$ ansible-playbook -u root realm_leave.yaml
Remove the Balaton hosts from the realm with this command (See the Ansible Best Practices documentation for more information):
$ ansible-playbook --limit Balaton -u root realm_leave.yaml
You need to first sign and return an ICLA and CCLA before we can accept and redistribute your contribution. Once these are submitted you are free to start contributing to cloudera-playbook. Submit these to [email protected].
- Fork the repo and create a topic branch
- Push commits to your repo
- Create a pull request!
We use Github issues to track bugs for this project. Find an issue that you would like to work on (or file one if you have discovered a new issue!). If no-one is working on it, assign it to yourself only if you intend to work on it shortly.
Please write a good, clear commit message, with a short, descriptive title and a message that is exactly long enough to explain what the problem was, and how it was fixed.