making mysql flexible with parelastic database scalability, amrith kumar, founder cto, parelastic

Download Making MySQL Flexible with ParElastic Database Scalability, Amrith Kumar, Founder CTO, ParElastic

Post on 09-May-2015

673 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

http://www.DatabaseMonth.com/database/parelastic-database-scalability

TRANSCRIPT

  • 1.Scalability and database virtualization How virtualizing your databases improves performance, and lowers costs New York City MySQL Meetup, October 3, 2013

2. Whats this presentation about? Scalability and the database tier Whats the problem? How did we get here? Some proposed solutions What are parallel databases? Whats ParElastic? How do I get ParElastic? Q&AOctober 3, 2013Tweet this presentation #parelasticScalability and the database tier | NYC MySQL Meetup2 3. What is the scalability problem?October 3, 2013Scalability and the database tier | NYC MySQL Meetup3 4. What is the scalability problem? Has many faces Connections and Concurrency Data Volume and Retention Period Databases and Tenants Read vs. Write Your problem(s) May be more than one May change over timeOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup4 5. Connections and Concurrency More [Active] Connections Worse Performance Sizing your databaseOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup5 6. Data Volume and Retention Period Longer Retention Period More Data More Data Worse Performance Progressive deterioration All data in memory All indexes in memory Not enough memory October 3, 2013Scalability and the database tier | NYC MySQL Meetup6 7. Databases and Tenants Common paradigm in SaaS applications Each tenants application instance has a database Several databases on each database instance More databases per instance Worse Performance In one customer engagement we were informed that no more than 1000 tenants could be located on one database instance before performance became unacceptable October 3, 2013Scalability and the database tier | NYC MySQL Meetup7 8. Read vs. Write Simple read (SELECT) queries could scale well Key based lookups With favorable indexes Things that cause heartburn Complex joins (with large data sets) Sorts Aggregation Reads are easier to scale than writesOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup8 9. How did we get here? A brief history lesson October 3, 2013Scalability and the database tier | NYC MySQL Meetup9 10. How did we get here? [1] A combination of factors Changes in the application user/usage Driven by the Internet and mobile computing News Cycles are getting shorter Economics Commodity computing is cheap and getting cheaper Solutions that can scale-out win, others lose Ability to leverage higher core-densities Other databases does a better job at this than MySQL MySQL would do great if you had a 20GHz processor ;)October 3, 2013Scalability and the database tier | NYC MySQL Meetup10 11. How did we get here? [2] The Evolution of the Database Management System A battle between generalized and specialized The Relational Database Management System (RDBMS) Designed for monolithic systems SMP Scale-Up Applications evolve quickly! Databases respond slowlyOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup11 12. How did we get here? [3] Moores Law Scale-Up seemed like a fine answer But there are limits October 3, 2013Scalability and the database tier | NYC MySQL Meetup12 13. How did we get here? [4] Database architectures traditionally were Shared CPU/Memory/Disk Also known as Shared-Everything But Shared-Everything doesnt scale At least not for databases A server costing twice as much doesnt always give you twice as much database power. You reach a point of diminishing returns.October 3, 2013Scalability and the database tier | NYC MySQL Meetup13 14. How did we get here? [5] You can pay more but you may not get more Source: Amazon RDS TPC-C Benchmark. Md. Borhan Uddin, Bo He, Radu Sion, Cloud Computing Center, SUNY Stony Brook. Viewed online http://digitalpiglet.org/research/sion2010cloud-rds.pdfOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup14 15. Some proposed solutionsOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup15 16. Some proposed solutions Several strategies have been advocated Cache, Cache, Cache, Get a bigger server [a.k.a. Scale-Up] Sharding [a form of Scale-Out] NoSQL or NewSQL [typically Scale-Out] Replication and variants We look at each one in more detailOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup16 17. Cache, Cache, Cache! Thats easy! Do some caching!caching transitive verb to cache cache noun Temporary computer storage used for quick retrieval of data in order to increase processing speed. Caching only addresses read; not write Social Media workloads are 'write heavy, 'interactive and highly personalized October 3, 2013Scalability and the database tier | NYC MySQL Meetup17 18. Get a bigger server [Scale-Up] I will use a bigger database server Can I even get a bigger server? What if m2.4xlarge isnt enough? Maybe I just have too much data? Maybe I have too many users?October 3, 2013Scalability and the database tier | NYC MySQL Meetup18 19. Sharding [a form of Scale-Out] Sharding will solve my problem!shard noun shrd a piece or fragment of a brittle substance ; broadly : a small piece or part sharding noun shr-di (a) to make ones application brittle or fragmented; (b) to take one big problem and make many small problems; (c) to complicate an application while claiming to solve a scalability problem; (d) to decrease developer productivity; (e) a bad idea; (f) sharding library: a mechanism that attempts (unsuccessfully) to hide the bad taste of shardingOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup19 20. NoSQL or NewSQL? You need NoSQL or NewSQL! Yes, I have to rewrite my application Yes, not all queries will work No, theres no standard query language No, most do not have ACID guarantees; hell some dont even guarantee Durability Yes, most are somewhat untried science-experiments More flavors than Ben & Jerrys Ice Cream [yes, really] But, all the cool kids are doing it!October 3, 2013Scalability and the database tier | NYC MySQL Meetup20 21. Replication and variants Replication based solutions (typically called clustering) Many copies of the data Distribute queries across the copies Keep the copies synchronized: like herding cats Write bottleneck Read/Write splitting Single Master (gets all the writes) Many Slaves (share the reads) Unpredictable latency Write bottleneckOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup21 22. What about MySQL Cluster? MySQL Cluster is a strange beast For best results, you must use the NDB interface Only supports the NDB storage engine Primarily a distributed in-memory Key-Value Store That is ACID compliant and supports joins and things if you use the SQL interface But no one tells you about the performance of this path! Published benchmarks are all FlexAsync which talk directly to the NDB interface And READ-ONLY For more details visit http://www.parelastic.com/blog/mysql-cluster-and-benchmarks Or stick around after the presentation and we can chat! October 3, 2013Scalability and the database tier | NYC MySQL Meetup22 23. What are parallel databases?October 3, 2013Scalability and the database tier | NYC MySQL Meetup23 24. What are parallel databases? 1 A database architecture proposed in 1992 Very successfully applied to many database problems Oracle Exadata, Netezza, Teradata, Greenplum, An example of the Shared Nothing database 2 paradigm1Parallel Database Systems: The future of high performance database processing [1992, Dewitt, Gray, ftp://ftp.cs.wisc.edu/pub/techreports/1992/TR1079.pdf] 2 The Case for Shared Nothing [1986, Stonebraker, http://db.cs.berkeley.edu/papers/hpts85-nothing.pdf]October 3, 2013Scalability and the database tier | NYC MySQL Meetup24 25. How parallel databases execute queriesImage from Parallel Database Systems: The future of high performance database processing [1992, Dewitt, Gray, ftp://ftp.cs.wisc.edu/pub/techreports/1992/TR1079.pdf]October 3, 2013Scalability and the database tier | NYC MySQL Meetup25 26. Benefits of parallel databases Linear improvement in reads Linear improvements in writes Better than linear improvement in joins Better than linear improvement in aggregation Better than linear improvement in sortsFor more details, refer Parallel Database Systems: The future of high performance database processing [1992, Dewitt, Gray, ftp://ftp.cs.wisc.edu/pub/techreports/1992/TR1079.pdf]October 3, 2013Scalability and the database tier | NYC MySQL Meetup26 27. Parallel Databases vs. Sharding Parallel Database Database architecture Application is data location agnostic Application perceives a single database Requires no application rewrites Application is not constrained by parallel database architecture A parallel database handles any schema October 3, 2013 Sharding Application architecture Application is data location aware Application perceives a collection of databases Requires application rewrites Application is constrained to the limitations of the sharding architecture Not all schemas are shardableScalability and the database tier | NYC MySQL Meetup27 28. What is ParElastic? Hypervisor for databasesOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup28 29. What is ParElastic? An approach to relational database virtualization Addresses issues of scalability in relational databases A parallel database architecture Built on standard MySQL or MySQL variant databases Horizontal Scalability ElasticOctober 3, 2013Scalability and the database tier | NYC MySQL Meetup29 30. ParElastic: System ArchitectureParElastic Architecture protected by US8214356, Apparatus for elastic database processing with heterogeneous data10/7/2013Flex Your Database | ParElastic Database Virtualization Engine30 31. Data Distribution: How it works User data is distributed across multiple storage nodes Queries are executed in parallel by some [or all] nodes Multiple distribution models supported Range Hash Broadcast Random ParElastic guarantees c

Recommended

View more >