Download - Ramunas Urbonas. The Journey
The JOURNEY
Adform explores Hadoop
Ramunas Urbonas @ Adform
Developer first
Development as a journey
High perspective talk
3 aspects
The JOURNEY
Direction / Planning / Equipment
It starts with a goal
Vision
Born of need
Maintenance costs
Time to market
Licensing costs
Hadoop
main data storage
alternative reporting
Attack your vision
why storage?
HDFS
Distributed
x 3
Auto-balancing
Can store big files
Resilient
50% 3%
Files vs Database
Multiple engines on same data
that brings us to...
Alternative reporting
Rich eco-system
HDFS
map-reduce
hive pig hbase
yarn & mr2
spark shark
impala
druid etc
Different purpose
Different SLAs
Emerging tools
Big community
Beta products
UI
Automation
Confident in vision
main data storage
alternative reporting
Building vision vs Maintaining
When left unattended
Narrow focus
Keep it fresh
The JOURNEY
Direction / Planning / Equipment
Travelling light
Climbing a mountain
Even harder for Adform
Linux courses
Java
Java backend administration
Memory management
Profiling
Garbage collection
Know Hadoop principles
More climbing ahead
Test everything
Consultants
Abstract
Technical calls
Time consuming
How > What
POC vs Production
The JOURNEY
Direction / Planning / Equipment
Commodity hardware
“Commodity hardware”
No SSD / Raids
Desktop cluster?
Our cluster - leftovers
6-7 years old
Still server machines
Electricity
Burning fuses
Outdated notion?
Tricky question
Best sport shoes?
Basketball?
Football / Jogging / Climbing
Common hardware
8Gb vs 300Gb
Our bottlenecks
Common bottlenecks
The JOURNEY
Continues...
Releasing products
Data imports
Growing team
Upgrading cluster
Impala / Shark
More business areas
Clear vision
Plan your learning
Careful hardware decissions
Thanks!
