Add additional cluster nodes

By default Chef-Bach brings up a three node cluster with two hadoop head (master) nodes and one data (slave) node. If an user wants to bring up additional nodes during cluster creation the following are the steps. Changes shown are to add an additional node (bcpc-vm4) to the cluster.

• Modify vbox create.sh shell script function create cluster VMs to create additional VMs

 for vm in bcpc-vm1 bcpc-vm2 bcpc-vm3 bcpc-vm4; do

• Modify enroll_cobbler.sh shell script to enroll new VMs in Cobbler

for i in bcpc-vm1 bcpc-vm2 bcpc-vm3 bcpc-vm4; do
  MAC=`$VBM showvminfo --machinereadable $i | grep macaddress1 | cut -d \" -f 2 | sed 's/.\{2\}/&:/g;s/:$//'`

• Modify tests/automated_install.sh script logic which starts all the machines

for i in 1 2 3 4; do
  vboxmanage showvminfo bcpc-vm$i | grep -q '^State:.*running' || vms_started="True"
  vboxmanage showvminfo bcpc-vm$i | grep -q '^State:.*running' || VBoxManage snapshot bcpc-vm$i take Shoe-less
  vboxmanage showvminfo bcpc-vm$i | grep -q '^State:.*running' || VBoxManage startvm bcpc-vm$i --type headless
done

• Modify tests/automated_install.sh script logic which verifies that all the machines have started successfully

while ! nc -w 1 10.0.100.11 22 || \
         !  nc -w 1 10.0.100.12 22 || \
         !  nc -w 1 10.0.100.13 22 || \
         !  nc -w 1 10.0.100.14 22
do
  sleep 60
  printf "Hosts down: "
  for m in 11 12 13 14; do
    nc -w 1 10.0.100.$m 22 > /dev/null || echo -n "10.0.100.$m "
  done
  printf "\n"
done

• Modify tests/automated_install.sh script logic which takes a snapshot of the VMs after being successfully started before cheffing

printf "Snapshotting post-Cobbler\n"
[[ "$vms_started" == "True" ]] && VBoxManage snapshot bcpc-vm1 take Post-Cobble
[[ "$vms_started" == "True" ]] && VBoxManage snapshot bcpc-vm2 take Post-Cobble
[[ "$vms_started" == "True" ]] && VBoxManage snapshot bcpc-vm3 take Post-Cobble
[[ "$vms_started" == "True" ]] && VBoxManage snapshot bcpc-vm4 take Post-Cobble

• Modify tests/automated_install.sh script logic which chefs all nodes

for i in 1 2 3 4; do
  printf "#### Chef machine bcpc-vm${i}\n"
  vagrant ssh -c "cd chef-bcpc; ./cluster-assign-roles.sh $ENVIRONMENT Hadoop bcpc-vm$i"
done

• Modify tests/automated_install.sh script logic which takes a snapshot of the VMs after cheffing of all nodes

printf "Snapshotting post-Cobbler\n"
VBoxManage snapshot bcpc-vm1 take Full-Shoes
VBoxManage snapshot bcpc-vm2 take Full-Shoes
VBoxManage snapshot bcpc-vm3 take Full-Shoes
VBoxManage snapshot bcpc-vm4 take Full-Shoes

• Modify cluster.txt with the role which need to be assigned to the new nodes

bcpc-vm4 08:00:27:C0:50:95 10.0.100.14 - bcpc.example.com role[BCPC-Hadoop-Worker]

The second parameter which is the mac id of the new node should be the mac id of the node's NIC 1. This can be found by executing the following command on hypervisor host.

vboxmanage showvminfo bcpc-vm4 | grep "NIC 1"