ambari meetup: what's new in ambari
DESCRIPTION
Ambari Meeting (September 2013)TRANSCRIPT
© Hortonworks Inc. 2013
Apache Ambari Meetup What’s new
September 2013
Page
© Hortonworks Inc. 2013
Quick History of Ambari
Page
Release Version
Release Date JIRAs Resolved
1.2.0 Feb 2013 441
1.2.1 Mar 2013 134
1.2.2 Apr 2013 106
1.2.3 Jun 2013 515
1.2.4 Jul 2013 115
1.2.5 Sep 2013 358
1.4.1 Oct 2013 468++ ← Current Release ← In Testing
© Hortonworks Inc. 2013
What’s New in Ambari 1.2.4
• Reduced “root SSH” requirements • Non-root SSH for cluster “Install” and “Add Hosts”
• Expanded Database Matrix • Hive Metastore and Oozie Oracle + MySQL options • Ambari Server DB Oracle option
• More Core Customizations • Configure Nagios user account • Configure Smoke Test user account • Choose Stack during cluster install
Page
© Hortonworks Inc. 2013
Reduced “root” SSH Requirements
• Configure to use “root” or a “sudoer” user for SSH • More flexibility in root-restricted environments
Page
© Hortonworks Inc. 2013
Hive Metastore Oracle Support
• Configure Hive Metastore to use Oracle Database
Page
© Hortonworks Inc. 2013
Oozie Oracle and MySQL Support
• Configure Oozie use Oracle or MySQL Database
Page
© Hortonworks Inc. 2013
Ambari DB Oracle
• Added Oracle to Ambari DB support matrix
Page
AMBARI
AMBARI WEB
RDBMS
postgres
oracle
© Hortonworks Inc. 2013
Ambari Database Matrix
Page
Hive Metastore Oozie Ambari DB
Use New Postgres
Use Existing MySQL
Use New MySQL
Use Existing Oracle
Use new Derby
© Hortonworks Inc. 2013
Customizable User Accounts
• Configure Nagios user • Configure Smoke Test user
Page
© Hortonworks Inc. 2013
What’s New in Ambari 1.2.5
• Manage Kerberos Secure Cluster • Customizable Dashboard Widgets • Improved Service Controls • Expanded Host Checks • Reduced “root” requirements • Core Security Enhancements
• Setup Ambari Server HTTPS • Master Key Property Encryption • Optional Server-Agent SSL Communication • Optional Ganglia and Nagios SSL
Page
© Hortonworks Inc. 2013
Manage Kerberos Secured Cluster
Page
• Run “Security Wizard” • Download-able CSV of
necessary principals & keytabs • Applies configuration properties
and restarts Services
© Hortonworks Inc. 2013
Customizable Dashboard Widgets
Page
© Hortonworks Inc. 2013
Improved Service Controls
• Start All and Stop All Service Controls • Live Component Status
Page
© Hortonworks Inc. 2013
(Further) Reduced “root” requirements
• Run “Ambari Server” as root or non-root account
Page
AMBARI SERVER
AMBARI WEB
RDBMS
root or non-root
HADOOP
AMBARI AGENT
HOST
AMBARI AGENT
HOST
AMBARI AGENT
HOST
AMBARI AGENT
HOST
AMBARI AGENT
HOST
AMBARI AGENT
HOST
AMBARI AGENT
HOST
AMBARI AGENT
HOST
Run as…
© Hortonworks Inc. 2013
Ambari Server HTTPS
Page
• Configure SSL for Ambari • Provide Certificate during “setup”
© Hortonworks Inc. 2013
What’s New in Ambari 1.4.1
• Install • Support for 2.0 Stack (YARN and MapReduce 2) • Host Cleanup Script • Nagios Group Customization • YARN Configs (including Capacity Scheduler) • Host List UI Improvements
• Monitoring • YARN Service and Host Alerts • YARN Dashboard Widgets and Heatmaps
Page
© Hortonworks Inc. 2013
Host Cleanup Script and expanded checks
Page
© Hortonworks Inc. 2013
Nagios Group Customization
Page
© Hortonworks Inc. 2013
YARN Configuration
Page
© Hortonworks Inc. 2013
Hosts List UI Improvements
Page
© Hortonworks Inc. 2013
Enable NameNode HA
• For Hadoop 2 Stack ONLY • Post-cluster install, wizard driven experience • Combination of manual and automated steps
Page
© Hortonworks Inc. 2013
JDK Path Setup
• Remove JDK Path field from Install Options UI • Step towards simplifying local installs
• Ambari will use JDK Path option set during setup ambari-server setup –j /my/jdk/path
• Use this option to configure JDK 7
Page
© Hortonworks Inc. 2012
Apache Ambari Ambari SCOM Management Pack for Hadoop
August 2013
Page
© Hortonworks Inc. 2013
Overview
Page
• Enable Microsoft System Center Operations Manager (SCOM) users to monitor their Hadoop cluster
• Ambari SCOM Management Pack for Hadoop gives insight into the performance and health of Hadoop
• “Ambari SCOM” leverages the Ambari framework to aggregate and expose Hadoop metrics
• Designed to work with Hadoop clusters that are running on Windows or Linux platforms
• Beta ETA early October
© Hortonworks Inc. 2013
Ambari-to-SCOM Integration
Page
Mgmt Pack
HADOOP Storage & Process
at Scale AMBARI
Ambari aggregates + exposes key Hadoop metrics
SCOM monitors cluster health and will alert in case of a problem
© Hortonworks Inc. 2013 Page
© Hortonworks Inc. 2013
Ambari (2H2013 and Beyond)
4Q2013 1H2014 2H2014
AMBARI 1.4.1 AMBARI 1.4.2 AMBARI 1.5.0 TBD
Provisioning • Hadoop 2 (YARN, MRv2)
• Host Cleanup Scripts
• Simplify Local Repo
• Accumulo • Flume
• Cluster Blueprints • Stack Inheritance
Managing • NameNode HA • YARN Configuration
• Host Exceptions • Move Masters • Remove host • HBase Multi-Master
• Customize Ports • Add Services
• Rack Awareness • Rolling Restarts • Log Search • Rolling Upgrades
Monitoring • YARN Alerts • YARN Dashboard
Widgets • YARN Heatmaps • Ambari Agent alerts
• Alert Mgmt • Maintenance Mode
Core Platform • JDK 7 Support
• Postgres 9.x
© Hortonworks Inc. 2013
Learn More
Page
Resource Location
Apache Ambari Project Page http://incubator.apache.org/ambari/
Mailing Lists http://incubator.apache.org/ambari/mail-lists.html
Ambari Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari JIRA https://issues.apache.org/jira/browse/AMBARI