![Page 1: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/1.jpg)
Install Apache Hadoop for Development/Production
using Opensource Distributions
October 2016
Dr.Thanachart NumnondaIMC Institute
![Page 2: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/2.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Agenda
● Introduction● Part 1: Hadoop Development Installation
● Part 1A: Cloudera Quickstart on Cloud● Part 1B: Hortonworks Sandbox on Azure
● Part 2: Hadoop Production● Install Cloudera Express Cluster on AWS
● Part 3: Hadoop as a Service● Launch EMR cluster on AWS
Hive.apache.org
![Page 3: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/3.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
This hands-on lab aims to guidea student to install Hadoop
for development using Sandboxand for production on Clusters
![Page 4: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/4.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hadoop Distribution
● On-Premise● Pure Apache Hadoop● Cloudera● Hortonworks● MapR● Pivotal● IBM Infosphere BigInsight
● On-Cloud (Hadoop as a Service)● Amazon EMR● Microsoft Azure HDInsight● Google Cloud Platform
Hive.apache.org
![Page 5: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/5.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
The Forrester Wave: Big Data HadoopDistributions Q1 2016
![Page 6: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/6.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
The Forrester Wave: Big Data HadoopCloud Q1 2016
![Page 8: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/8.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hadoop Development: Sandbox
● Recommended● Cloudera Quickstart● Hortonworks Sandbox
● On PC/Mac● Require to install VMWare Player or VirtualBox
● On Linux Virtual Server● Need to install Docker Engine ● Microsoft Azure has a Hortonworks Sandbox Image
which can easily launched.
Hive.apache.org
![Page 9: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/9.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
http://hortonworks.com/downloads/#sandbox
![Page 10: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/10.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
http://www.cloudera.com/downloads.html
![Page 11: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/11.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Part 1A: Install Cloudera Quickstart on Cloud Virtual Server
![Page 12: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/12.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hands-On Launch a virtual server (EC2)
on Amazon Web Services
(Note: You can skip this session if you use your owncomputer or another cloud service)
![Page 15: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/15.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Virtual Server
This lab will use a EC2 virtual server to install aCloudera Cluster using the following features:
Ubuntu Server 14.04 LTS
One m3.xLarge 4vCPU, 15 GB memory,80 GBSSD
Security group: default
Keypair: imchadoop
![Page 16: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/16.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select a EC2 service and click on Lunch Instance
![Page 17: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/17.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select an Amazon Machine Image (AMI) andUbuntu Server 14.04 LTS (PV)
![Page 18: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/18.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Choose m3.xlarge Type virtual server
![Page 22: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/22.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select Create an existing security group > Default
![Page 23: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/23.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Click Launch and choose imchadoop as a key pair
![Page 24: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/24.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Review an instance and rename one instance as amaster / click Connect for an instruction to connectto the instance
![Page 25: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/25.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Mac/Linux
![Page 26: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/26.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Can also view details of the instance such as PublicIP and Private IP
![Page 27: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/27.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Windows using Putty
![Page 29: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/29.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hands-On: Install a Docker Engine
![Page 31: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/31.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Update OS (Ubuntu)
● Command: sudo apt-get update
![Page 32: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/32.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Docker Installation
● Command: sudo apt-get install docker.io
![Page 33: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/33.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Docker commands:
● docker images● docker ps● docker attach id● docker kill id● docker commit id
● Exit from container● exit (exit & kill the running image)● Ctrl-P, Ctrl-Q (exit without killing the running image)
Hive.apache.org
![Page 34: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/34.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Install Cloudera Quickstart on Docker Container
![Page 35: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/35.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Pull Cloudera Quickstart
$ sudo docker pull cloudera/quickstart:latest
![Page 36: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/36.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Verify the image was successfully pulled
$ sudo docker images
![Page 37: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/37.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Run Cloudera quickstart
$ sudo docker run --hostname=quickstart.cloudera--privileged=true -t -i [OPTIONS] [IMAGE]/usr/bin/docker-quickstart
Example: sudo docker run--hostname=quickstart.cloudera --privileged=true -t -i -p8888:8888 cloudera/quickstart /usr/bin/docker-quickstart
![Page 39: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/39.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Login to Hue
http://ec2-54-173-154-79.compute-1.amazonaws.com:8888
![Page 40: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/40.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Option: Launch a virtual server on Microsoft Azure for Cloudera
Sandbox
(Note: You do not need to do this if you use yourown computer or another cloud service)
![Page 41: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/41.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Sign up for Visual Studio Dev Essential to get freeAzure credit.
![Page 44: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/44.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Virtual Server
This lab will use an Azure virtual server to install aCloudera Quickstart Docker using the followingfeatures:
Ubuntu Server 14.04 LTS
DS3_V2 Standard 4 Core, 14 GB memory,28GB SSD
![Page 45: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/45.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select New => Virtual Machines => Virtual Machines
![Page 46: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/46.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
On the Basics page, enter:
1. a name for the VM
2. a username for the Admin User
3. the Authentication Type set to password
4. a password
5. a resource group name
![Page 52: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/52.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Setting the inbound port for Hue (8888)
![Page 57: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/57.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Mac/Linuxssh -i ~/.ssh/id_rsa [email protected]
![Page 58: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/58.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Windows using Putty
![Page 59: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/59.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Part 1B: Install Hortonworks Sandboxon Azure
![Page 60: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/60.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Sign up for Visual Studio Dev Essential to get freeAzure credit.
![Page 62: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/62.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Uncompress the docker zip file
$ gunzip HDP_2.5_docker.tar.gz
$ sudo docker load < HDP_2.5_docker.tar
(This may take several minutes)
Load the Docker image
![Page 64: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/64.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Virtual Server
This lab will use an Azure virtual server to install aHortonworks Sandbox using the following features:
Hortonworks Sandbox with HDP 2.4
DS3_V2 Standard 4 Core, 14 GB memory,28GB SSD
![Page 65: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/65.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select New => Search => Hortonworks Sandbox
![Page 67: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/67.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
On the Basics page, enter:
1. a name for the VM
2. a username for the Admin User
3. the Authentication Type set to password
4. a password
5. a resource group name
![Page 74: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/74.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Mac/Linuxssh [email protected]
![Page 75: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/75.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Windows using Putty
![Page 76: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/76.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Login to Hortonworks >> ip_address:8888
![Page 77: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/77.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Login to Ambari >> ip_address:8080
![Page 79: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/79.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Part 2: Hadoop Productionon Cloud Cluster
![Page 80: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/80.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hadoop Production: Cluster
● Recommended● Cloudera Express● Hortonworks
● Cloudera Express● Install a cluster on 4 EC2 AWS
● Hortonworks Data Platform● Install a cluster on Microsoft Azure using Marketplace
Hive.apache.org
![Page 82: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/82.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Cloudera VM
This lab will use a EC2 virtual server on AWS to installCloudera.
![Page 83: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/83.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hands-On: Launch a virtual server on EC2 Amazon Web Services
![Page 84: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/84.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hands-On: Launch a virtual server on EC2 Amazon Web Services
![Page 86: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/86.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Virtual Server
This lab will use a EC2 virtual server to install aCloudera Cluster using the following features:
Ubuntu Server 14.04 LTS
Four m3.xLarge 4vCPU, 15 GB memory,80 GBSSD
Security group: default
Keypair: imchadoop
![Page 87: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/87.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select a EC2 service and click on Lunch Instance
![Page 88: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/88.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select an Amazon Machine Image (AMI) andUbuntu Server 14.04 LTS (PV)
![Page 89: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/89.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Choose m3.xlarge Type virtual server
![Page 93: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/93.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select Create a new security group > Add Rule asfollows
![Page 94: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/94.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Click Launch and choose imchadoop as a key pair
![Page 95: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/95.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Review an instance and rename one instance as amaster / click Connect for an instruction to connectto the instance
![Page 96: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/96.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Mac/Linux
![Page 97: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/97.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Can also view details of the instance such as PublicIP and Private IP
![Page 98: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/98.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Connect to an instance from Windows using Putty
![Page 100: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/100.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hands-On: Installing Cloudera on EC2
![Page 101: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/101.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Download Cloudera Manager
1) Type command >wgethttp://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin
2) Type command > chmod u+x cloudera-manager-installer.bin
3) Type command > sudo ./cloudera-manager-installer.bin
![Page 104: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/104.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Login to Cloudera Manager
Wait several minutes for the Cloudera Manager Server to complete its startup.
Then running web browser: http:// public-dns: 7180
![Page 108: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/108.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Provide your 4 instances <private ip>addresses in the cluster
![Page 111: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/111.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Select install Oracle Java & Java unlimited
![Page 113: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/113.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Define user as ubuntu & Browse the private key (imchadoop.pem) file which wehave downloaded in the previous part. Keep Passphrase as blank
![Page 127: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/127.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Hadoop Production: on Cloud
● Recommended● Amazon EMR● Azure HDInsight
● Amazon EMR● Install a cluster on 4 EC2 AWS
● Azure HD Insight● Install a cluster on 4 Vitual Servers Microsoft Azure
Hive.apache.org
![Page 137: Install Apache Hadoop for Development/Production](https://reader034.vdocuments.net/reader034/viewer/2022042500/587138ae1a28abf0568b6409/html5/thumbnails/137.jpg)
Thanachart Numnonda, [email protected] Oct 2016Hadoop Installation
Thank you
www.imcinstitute.comwww.facebook.com/imcinstitute