MongoDB Management Service Engineering Team, MongoDB

Cailin Nelson, Cadran Cowansage, Louisa Berger, Bard Bloom, Tim Olsen

#MongoDBWorld

Automate MongoDB with MMS

MMS: MongoDB Management Service

MMS Automation: What Does It Do?

● Create MongoDB systems of any size, and any configuration

● Manage MongoDB systems, such as adding capacity or resizing the oplog, with no application downtime

● Upgrade a deployment, with no application downtime

… all from the comfort of your web browser.

MMS Automation…. And MMS Provisioning Too

For extra fun, we’ve also added an Amazon AWS

integration which allows you to provision the servers

on which your MongoDB processes will run, directly

from MMS.

● Optional component - you can provision your base servers any way you

like, doing it via MMS Provisioning is just one way

● Future plans to integrate OpenStack, VMWare, etc.

MMS Automation: Why?

● Ease of operation● Expect the unexpected

About This Talk...

● Series of demos● Questions at the end

MMS Provisioning on AWS… from 0 to Sharded Cluster in 8 minutes or less

Raising Your Cluster Right… MongoDB management the easy way

Upgrading a Cluster Manually 1. Run db.upgradeCheckAllDBs() to check data set for compatibility2. Resolve any incompatibilites in your deployment3. Upgrade authentication model 4. Download mongoDB binaries for 2.65. Disable the balancer6. Upgrade the cluster’s meta data7. Wait for each mongos to exit on completion8. Upgrade each mongos process, one at a time9. Upgrade each config server, one at a time, with the first one upgraded last.10. Upgrade each shard, one at a time :

a. Upgrade secondaries, one at a time:i. Shut down the mongodii. Replace the 2.4 binary with the 2.6 binaryiii. Restart mongodiv. Wait for member to recover to SECONDARY state

b. Step down the primary c. Wait for another member to be elected to PRIMARYd. Upgrade previous primary

11. turn back on the balancer

Now, using Automation

How does it work?

Automation ≈ Self-Driving Car

● Tell it where you want to go.o It goes there

● If you steer manually?o It reroutes and goes toward your destination

● Bridge out?o It reroutes and goes toward your destination

● Want to go somewhere else?o It reroutes and goes toward your new destination

Architecture Introduction

● One agent per

machine● Agent talks to:

o MMS instructions

o Mongo sense control

● No agent-agent● Single executable

MMS

MongoD

MongoS

Config Server

Auto. Agent

MongoD

MongoS

Config Server

Auto. Agent

Automation Cycle

Get Instructions

Inspect Cluster

Adjust Cluster

Sleep 30 sec

Desired State?

You tell Automation what you want.Automation makes that happen.

Desired State ≈ Self-driving car’s destination.

What Automation Knows

1.What you want a. from MMS

2.What’s actually on the clustera. from inspecting the cluster

3.How to do various thingsa. Like “start Mongod” or “init replica set”b. from MongoDB documentationc. preconditions: what has to be true before.d. postconditions: what should be true after.

Planning

● Look at what you want vs. what you have● If they’re the same, yay!

o Check again in 30 seconds● Else, make a plan to fix what you have

o Basic robotics-style planning problemo precondition/postcondition make this work.

Executing Plans

● Follow the plan, one move at a time.● Each move has a bunch of actions● Check expected vs. reality for each action

o Check preconditions before doing an actiono Check postconditions afterwardso If they’re different, go make a new plan!

● How could they be different?o machine crasho killing/modifying a mongo instance by hand

Automation

● Instructions from you● Information from your cluster● Expertise about how to run Mongo● Patience to get all the details right

Advanced OperationsOplog Resizing

What is an oplog?

● A capped collection that stores an ordered history of logical writes to a MongoDB database

● Enables replication

Why resize the oplog?

1.You may want to increase its size to accommodate a high write rate and/or high replication lags to secondaries.

2.You may want to decrease its size to save on disk space.

Resizing the Oplog with Automation

How to resize an oplog

1. Shutdown a replica set member2. Start the member in standalone mode, listening on a different

port than it usually does3. Back up the last oplog entry4. Drop the oplog5. Recreate the oplog with its new size6. Insert the saved oplog entry7. Stop the member8. Start the member with normal options9. Repeat Steps 1 - 8 for each of the remaining replica set members

Or just have Automation do it

Automating MMS

MongoDB Management Service Engineering Team, MongoDB

Cailin Nelson, Cadran Cowansage, Louisa Berger, Bard Bloom, Tim Olsen

#MongoDBWorld

Questions