Slide 1

Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 1 EPL671: Research Methodologies in Computer Science, Graduate Course, Tuesday, Mar 19 th, 2013. Big Data - What Is It? Demetris Zeinalipour Assistant Professor Data Management Systems Laboratory Department of Computer Science University of Cyprus http://dmsl.cs.ucy.ac.cy/ Slide 2 Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 2 Demetris Zeinalipour, http://www.cs.ucy.ac.cy/~dzeina/ Objectives To provide an overview of the emerging field of Big Data Management from a wide range of perspectives: Fundamentals / Trends, Industrial / Academic, Commercial / Open, Reality / Visionary, etc. I assume that the audience has a technical background (e.g., DBAs) Lots of examples and illustrations to keep this presentation entertaining and educating. Slide 3 Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 3 Demetris Zeinalipour, http://www.cs.ucy.ac.cy/~dzeina/ Talk Outline Big Data Definitions and Background Big Data Definition by 3V Examples Velocity Sensor Monitoring, Network Monitoring, Web2.0 Media, Smartphone Services Volume Text Not Scalable HDFS designed for unreliable hardware (2-3 failures / 1000 nodes / day)"> Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 31 Demetris Zeinalipour, http://www.cs.ucy.ac.cy/~dzeina/ Volume #3: Big Data File Systems Big Data Filesystems: HDFS Namespace lookup are fast (1 Master enough!) [ 1GB Metadata = 1PB Data ] In NFS Metadata + Transfers going through same server => Not Scalable HDFS designed for unreliable hardware (2-3 failures / 1000 nodes / day) Slide 32 Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 32 Demetris Zeinalipour, http://www.cs.ucy.ac.cy/~dzeina/ Volume #3: Big Data File Systems Big Data Filesystems: How Big? Results from 2010: HDFS scalability: the limits to growth http://static.usenix.org/publications/login/2010-04/openpdfs/shvachko.pdf Slide 33 Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 33 Demetris Zeinalipour, http://www.cs.ucy.ac.cy/~dzeina/ Variety #2: File Systems NFS uses a Client/Server Architecture that is a single point of failure by default. Slide 34 Dagstuhl Seminar 10042, Demetris Zeinalipour, University of Cyprus, 26/1/2010 34 Demetris Zeinalipour, http://www.cs.ucy.ac.cy/~dzeina/ Talk Outline Big Data Definitions and Background Big Data Definition by 3V Examples Velocity Sensor Monitoring, Network Monitoring, Web2.0 Media, Smartphone Services) Volume Text