mesosphere and contentteam: a new way to run cassandra

A New Way to Run Cassandra

© 2015. All Rights Reserved.

Speakers

Ben Whitehead@BenRWhitehead

Distributed Applications Engineer

2

Robert Stupp@snazy

Committer to Apache Cassandra & MVP


Disclaimer

Opinions and facts mentioned in our talk are purely our own and not necessarily those of Mesosphere or contentteam.

So, don’t take us too seriously as we are tech kids that just like to play with cool technology – seriously.

Open source software requires the input from the community – and that is you, please help us to improve it. Get involved and spread the word!

3


1 Apache Mesos intro

2 Apache Mesos architecture

3 Cassandra-Mesos framework

4 Demo

5 Q & A

4

© 2015. All Rights Reserved. 5

Mesos lets us treat a cluster of nodes...


As one big computer


Not as individual machines


Not as individual machines Not as VMs

But as computational resources like cores, memory, disk, etc.


Apache Mesos

• Open Source Apache project

• Cluster Resource Manager

• Scalable to 10,000s of nodes

• Fault-tolerant, battle-tested

• A kernel for your datacenter

10


DCOS

• Web UI and CLI

• init, cron, package management, discovery, admin routing

• Ecosystem of DCOS Services

11


DCOS CLI

12

$ dcos Command line utility for the Mesosphere Datacenter Operating System (DCOS). The Mesosphere DCOS is a distributed operating system built around Apache Mesos. This utility provides tools for easy management of a DCOS installation. Available DCOS commands: config Get and set DCOS CLI configuration properties help Display command line usage information marathon Deploy and manage applications on the DCOS node Manage DCOS nodes package Install and manage DCOS software packages service Manage DCOS services task Manage DCOS tasks


DCOS

13


DCOS

• Web UI and CLI

13


DCOS

• Web UI and CLI• init, cron, DNS

13


DCOS

• Web UI and CLI• init, cron, DNS• package management

13


DCOS

• Web UI and CLI• init, cron, DNS• package management• On-premise + cloud

13


DCOS



13


DCOS



13

Cassandra


DCOS



• Highly scalable database

13

Cassandra


DCOS



• Highly scalable database• Continuously available

13

Cassandra


DCOS




• Scale out as you need

13

Cassandra


DCOS




• Scale out as you need

13

Cassandra

Cassandra-Mesos framework



• Apache2 licensedhttp://github.com/mesosphere/cassandra-mesos

• Included in Mesosphere DCOS

14

http://github.com/mesosphere/cassandra-mesos


Cassandra-Mesos use case

15



Use-case:Create Cassandra cluster with X nodes.

Mesos provides the Cassandra cluster with X nodes!

15





• Devs needing a cluster “right now”

15





• Devs needing a cluster “right now”• Start the cluster without the hassle to setup servers

15





• Devs needing a cluster “right now”• Start the cluster without the hassle to setup servers• Flexible scale-out

15



• In the beginning: just a bunch of commodity servers

• Define Cassandra cluster size

• Seed Cassandra nodes are allocated and started

• Remaining Cassandra nodes allocated and started

• That’s it – period.

16



17



18



19



20



21



22



23



24



25



26


Cassandra-Mesos – getting started

Give it – # of nodes (and # of seeds)

– # of cores per node

– Amount of RAM per node

– Amount of disk space per node

– Name of cluster

“and press ENTER”

27


Cassandra-Mesos – add Cassandra nodes

• Add as many Cassandra nodes as you need

• Coordinated by the scheduler, fault-tolerant

• Just a REST API call

28


Cassandra-Mesos – repair

• Anti-entropy repair (aka. nodetool repair)

• Can perform cluster-wide maintenance job (“rolling”)

• Scheduler keeps track of the job, persists state, fault tolerant

• Triggered via REST API

29


Cassandra-Mesos – more cluster jobs

• Cleanup

• Replace (dead) node

• Restart

• QA-Report

• etc.

30


Cassandra-Mesos – REST API

• Manage + monitor cluster jobs

• Inspect configuration

• Inspect status / health information

• Change seed node status

• Add more nodes

31


Cassandra-Mesos – CLI tools

• Use Cassandra tools with Mesos-DNS

• Supporting CLI tool wrappers for – nodetool – cqlsh – cassandra-stress

32


Cassandra-Mesos – how to access?

33



• “Wait, how do I access the cluster? I don’t know on which nodes it runs!”

33




• Right!

33




• Right!

• Mesos-DNS

33


Cassandra-Mesos – client application?

• Mesos-DNS ! • DataStax Java Driver example

accessing cluster named “foo”Cluster cluster = Cluster.builder() .addContactPoint( "cassandra-foo-node. cassandra.foo.mesos") .build();

34


Cassandra-Mesos – upcoming

• Cassandra upgrade job

• Enhanced configuration options

• Extended multi data center support

• Backup using snapshots

• Includes restore functionality

35


Cassandra-Mesos – LIVE DEMO!

• We would really love to show that it really scales to some hundred nodes!

• We can do that in meetups! • But – time is a rare resource today…

36

Thank youhttp://github.com/mesosphere/cassandra-mesos

http://github.com/mesosphere/cassandra-mesos

mesosphere and contentteam: a new way to run cassandra

Technology