data science day new york: gigaom big data market overview
TRANSCRIPT
Big Data Market Overview
Jo Maitland, Research Director, GigaOM
• 15+ years in technology research and journalism with focus on emerging infrastructure technologies including next generation storage, networking, virtualization, and cloud computing– Forrester Research (Analyst)– The 451 Group (Analyst)– TechTarget (Executive Editor)– UBM Tech (LightReading.com, Senior
Editor)– Computerwire (Senior Writer)– PC Week (Reporter)
Agenda
• Data growth, it’s big• Oh the mess we are in…• Let’s turn off all the computers• Don’t be daft!• There’s new technologies to help store and analyze all this data• Enter Hadoop, NoSQL and Hype.• It’s the apps stupid• Emerging trends• Questions to consider
How Big?
Data growth at Facebook
Data growth at Twitter
Growth of machine generated data
Data growth worldwide
Data growth in the enterprise is staggering
• Walmart handles more than 1 million customer transactions per hour
• There are about 90 trillion emails per year
• Google processes some 24 petabytes of data per day
• AT&T transfers 30PB of data per day
Business decision-makers are screwed, basically
The Answer?
What to do…
• Turn off all the computers?• Turn off some of the computers? • Stop storing everything and
classify your data?• All attempts to stem the tide
of big data will fail.
Two new technologies have come to our rescue
Hadoop
NoSQL
Commercial solutions enter the fray
• Hadoop distribution companies– Cloudera– HortonWorks– MapR– + +
• NoSQL database companies– 10gen (MongoDB)– DataStax (Cassandra)– Basho (Riak)– + +
Hadoop + big data apps = useful
Big data applications are key
• Operational intelligence– Splunk, Sumo Logic
• Sales and marketing– GoodData, Media Science, Bloomreach
• Visualization– Tableau Software, QlikTech, Palantir
• Business Intelligence– Platfora, Domo, WibiData
• Online advertizing– Collective, DataXu, RocketFuel, Turn
• Data as a service• FICO, DataSift, Bluekai
What’s next?
Emerging trends
• More data• Focus on applications• Data democratization and trust• A shift to real time
data
Emerging trends
• More data• Applications• Data democratization and trust• A shift to real time
Applications
Square
PredPol
23andMe
Emerging trends
• More data• Applications• Data democratization and trust• A shift to real time
Data democratization and trust
Emerging trends
• More data• Applications• Data democratization and trust• A shift to real time
Shift to real time
Questions to consider
Investors
• Is the company in an area that is already well funded or over-funded?– Infrastructure
• What are the emerging sub-categories?– Cloud-based services
• What’s the new angle?– ?
Customers
• Are there existing big data apps you could use instead of building a custom app?– Log file analysis
• What is your 3 year big data roadmap?– Just as companies have measured their ROI on technology
investments, they should also measure the value they receive from information.
