there's no such thing as big data

Download There's no such thing as big data

If you can't read please download the document

Upload: andrew-clegg

Post on 19-Jan-2015

3.338 views

Category:

Documents


3 download

DESCRIPTION

“Big Data” is a term that’s come from nowhere in the last 5 years or so, and is now practically ubiquitous within IT. But is it useful or even meaningful? Doesn’t it put too much emphasis on size over content or value? Does it add anything to discussions at all? Or does it actually impede communication, by obscuring crucial differences between diverse kinds of data that all require different tools, algorithms and strategies? (Talk presented at "Big Data for the Public Sector and Business Enterprise", London 2013)

TRANSCRIPT

  • 1. Theres no such thing as Big DataDr. Andrew CleggData Analytics & Visualization TeamPearson TechnologyTwitter: @andrew_cleggOpinions are my own.

2. Speaking of TwitterMany companies think they have a big data problemwhen they really have a big data problem.-- @dbasch (Diego Basch), 17 Nov 2012 Followed the next day by: I interact with companies whose Big Data problems can be solved on laptops.2 Theres no such thing as Big Data l 27/03/13 3. What is Big Data anyway?Customer preference and behaviour data is.3 Theres no such thing as Big Data l 27/03/13 4. What is Big Data anyway?Apparently sensor data is too. Image from http://en.wikipedia.org/wiki/Wind_farm (cc) Tomasz Sienicki4 Theres no such thing as Big Data l 27/03/13 5. What is Big Data anyway?So is web crawl data.5 Theres no such thing as Big Data l 27/03/13 6. What is Big Data anyway?Denitely social media data6 Theres no such thing as Big Data l 27/03/13 7. What is Big Data anyway? and social media metadata. Image from http://www.facebook.com/note.php?note_id=4697163989197 Theres no such thing as Big Data l 27/03/13 8. What is Big Data anyway?Video, audio and other media, surely.8 Theres no such thing as Big Data l 27/03/13 9. What is Big Data anyway?Health, medical and life sciences data might well be.9 Theres no such thing as Big Data l 27/03/13 10. What is Big Data anyway? So particle physics data must be. Image from Flickr (cc) Image Editor10 Theres no such thing as Big Data l 27/03/13 11. What do all these types of data have in common?11 Theres no such thing as Big Data l 27/03/13 12. What do all these types of data have in common?Nothing.12 Theres no such thing as Big Data l 27/03/13 13. What about size?13 Theres no such thing as Big Data l 27/03/13 14. Size isnt everything. All laptop-sized data sets.14 Theres no such thing as Big Data l 27/03/13 15. What about storage?15 Theres no such thing as Big Data l 27/03/13 16. Pick the right architecture for the job. Table-structured, Relational database E-commerce data transactional (good old SQL) Simple, high-volume,Columnar database, Sensor data often aggregatedHadoop sequenceleJSON-based, textual, Document database,Social feeds e.g. Twitternested structuressearch engine Networks e.g. socialDirected or undirected Graph databaseconnectionsgraphs(Aurelius, Neo4j) Processed sequentially, Distributed lesystem Multimediaopaque internal structure (HDFS, Mogile, S3)Biochemical sequencesComplex 2D or 3D Specialized le formats and models structureand tools These are just examples. Make the right choice for your application.16 Theres no such thing as Big Data l 27/03/13 17. Pick the right architecture for the job.Relational database(good old SQL)Columnar database, NoSQL?Hadoop sequenceleDocument database, These are all as different from each othersearch engine Graph database as each one of them is from this. (Aurelius, Neo4j) Distributed lesystem(HDFS, Mogile, S3) Specialized le formatsand tools17 Theres no such thing as Big Data l 27/03/13 18. What about analysis?18 Theres no such thing as Big Data l 27/03/13 19. Pick the right methodology for the job. Text topic modelling, sentiment analysis, information extraction E-commerce data propensity analysis, collaborative ltering Multimedia speech-to-text, audio ngerprinting, face recognition Clickstream logs frequent pattern mining, sequence analysis Proton-proton collisions from LHC I have absolutely no idea19 Theres no such thing as Big Data l 27/03/13 20. So why does any of this matter?20 Theres no such thing as Big Data l 27/03/13 21. Because all of these phrases rethinking our Big Data strategyenterprise-ready Big Data solutionfacing the Big Data challenge are pretty close to meaningless.21 Theres no such thing as Big Data l 27/03/13 22. And statements like these We cant do any of that clever Big Data stuff because were not collecting or storing any Big Data.We cant do any of that clever Big Data stuff because we dont have big enough hardware or expensive tools. are signs of opportunities being missed.22 Theres no such thing as Big Data l 27/03/13 23. What should we call it then, Andy?23 Theres no such thing as Big Data l 27/03/13 24. What should we call it then, Andy?Data.24 Theres no such thing as Big Data l 27/03/13 25. Thank you for listening!25 Theres no such thing as Big Data l 27/03/13