1

Hortonworks Hadoop Cluster Setup Using Apache Ambari

2

• Hardware• M3.large

• Software

• Security rules

Server Specification

3

• Login to Ambari server via ssh• yum install ntp• sudo chkconfig ntpd on• sudo service ntpd start• sudo /etc/init.d/iptables stop• Save a copy of instance as AMI for subsequent use

• On Ambari server, yum install wget• Goto cd /etc/yum.repos.d/• sudo wget

http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.0.0/ambari.repo

• sudo yum install ambari-server• sudo ambari-server setup• sudo ambari-server start

• Launch another 4 instances of Ambari Agents using previously saved AMI to form Hadoop cluster

Setup Commands

4

• Take down internal ip addresses for Ambari Agent, the reason for using internal ip is potentially you can shut/start instance again without the need of changing ip address. These are required as part of the setup process

• ip-172-31-10-92.ap-southeast-1.compute.internal• ip-172-31-10-93.ap-southeast-1.compute.internal• ip-172-31-9-88.ap-southeast-1.compute.internal• ip-172-31-7-40.ap-southeast-1.compute.internal

• HDP2.2 repos• http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.2.4.2

• HDP-UTILS-1.1.0.20• http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6

Setup Commands

5

• Access Ambari Server using Admin UI using • http://ec2-52-74-202-59.ap-southeast-1.compute.amazonaws.com:8080/• Default username and password: admin/admin

Launch Ambari Server

6

• The whole setup process is very straight forward with step by step guide• Ambari will suggest Master and Slaves components to be installed on which

agents• Ambari Server will deploy agent to the Hadoop cluster and then install all

required Hadoop components accordingly

Setup Procedures

7

Summary of Services Installed

8

Summary of Services Installed

9

• In the event the disk space has been used up, you can simply extend the disk space by attaching the instance to another new column created. Do remember to create a snapshot of data first prior to creating a new volume

Notes