© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Rahul Bhartia, Partner Solution Architect

24th May 2016

MongoDB on AWSAn introduction and best-practices for AWS

MongoDB : Basics

Document Model

RDBMS

{ first_name: ‘Paul’, surname: ‘Miller’, city: ‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } ]}

MongoDB

Features

Rich Queries Find Paul’s carsFind everybody in London with a car between 1970 and 1980

Geospatial Find all of the car owners within 5km of Trafalgar Sq.

Text Search Find all the cars described as having leather seats

Aggregation Calculate the average value of Paul’s car collection

Map Reduce What is the ownership pattern of colors by geography over time (is purple trending in China?)

MongoDB: New features in 3.2

New Storage engines

In-memory

Run your most demanding, real-time apps with

the in-memory storage engine (beta)

Encrypted

Secure your data-at-rest with the encrypted

storage engine

Data Governance: Document Validation

Implement data governance without sacrificing

the agility that comes from a dynamic schema

Dynamic Lookup

Combine data from multiple collections with left

outer joins for richer analytics & more flexibility

in data modeling

Richer In-Database Analytics & Search

New Aggregation operators extend options for

performing analytics with lower developer

complexity

Array Operators

Math Operators Text

$slice,

$arrayElemAt,

$concatArrays,

$filter, $min,

$max, $avg,

$sum, and

more.

$stdDevSamp,

$stdDevPop,

$sqrt,$abs,

$trunc, $ceil,

$floor, $log,

$pow

$exp, and

more.

Case sensitive text search and support for languages such as Arabic, Farsi, Chinese, and more.

MongoDB Connector for BI

Visualize and explore multi-structured data using

SQL-based BI platforms.

Connector for BIProvides Schema

Translates QueriesTranslates Response

Compass: The GUI for MongoDB

Visually explore your data and schema.

Run ad hoc queries in seconds

Make smarter decisions about indexing,

document validation, and more.

No command line needed

MongoDB: High Availability & Scalability

Replica Sets

Replica set – 2 to 50 copies

Replica sets make up a self-healing ‘shard’

Replica sets address - High availability, Maintenance (e.g., HW swaps) and Disaster Recovery

Application

Driver

Primary

Secondary

Secondary

Replication

Replica Sets – Workload Isolation

Replica sets enable workload isolation

Example: Operational workloads on the primary

node, analytical workloads on the secondary

nodes

Application

PrimaryIn-memory

SecondaryWiredTiger

User DataSessions, Cart,Recommendations

SecondaryWiredTiger

PersistedUser Data

Elastic Scalability: Automatic Sharding

Increase or decrease capacity as you go

Automatic load balancing

Three types of sharding - Hash-based, Range-based and Tag-aware

Shard 1

Shard 2

Shard 3

Shard N

Horizontally Scalable

Query Routing

Multiple query optimization models

Each of the sharding options are appropriate for different apps / use cases

MongoDB: AWS Deployment Architectures

AWS EC2 Instance Types

General Purpose - M3, M4• Start with General Purpose instances and EBS GP2

Compute-optimized - C3, C4• WiredTiger - Write performance due to document-level concurrency control

Memory-optimized - R3• In memory storage engine: NEW!• Larger working set - MMAPv1 with read-intensive applications

Storage-optimized - I2, D2• Local instance store but lost when instance is stopped or terminated• Always use with higher replication

https://docs.mongodb.com/ecosystem/platforms/amazon-ec2/#deployment-notes

Amazon EBS Volumes

EBS GP2 for general workloads and EBS PIOS for consistent performancehttps://docs.mongodb.com/ecosystem/platforms/amazon-ec2/#storage-considerations

EBS-optimized Instances provides additional/dedicated bandwidthhttp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html

EBS snapshots for recovery and backupshttps://docs.mongodb.com/ecosystem/platforms/amazon-ec2/#backup-restore-verify

AWS Global Footprint

High Availability: Intra-Region

High Availability: Intra-Region

High Availability: Inter-Region

Disaster Recovery: Inter-Region

Disaster Recovery: Inter-Region

Go global

Deploy Global, Access Local

Replicate Data Near Users

MongoDB - Management

The Best Way to Run MongoDB

Ops Manager and Cloud Manager allows you

leverage and automate the best practices.

10x-20x more efficient operations

Complete performance visibility

Protection from data loss

Performance optimization

How they work

Ops or Cloud Manager

mongod mongodmongod

Agent Agent Agent

Amazon QuickStart

https://aws.amazon.com/quickstart/

VPC with private and public subnets

Instance role with fine-grained permissions.

Security groups

A fully customized MongoDB cluster - Replica sets, shards, and config servers and customized EBS storage

Monitoring & Alerting

Over 100+ database-related metrics

Dozens of optimized charts

Custom alerts so incidents do not become emergencies

Do-It-Yourself

Set of utilities distributed with MongoDB – mongostat, mongotop

Database commands – serverStatus, dbStats, collStats etc

Self-hosted – Ganglia, mtop, munin, nagios

Hosted (SaaS) – New Relic, Datadog, Server Density

https://docs.mongodb.com/manual/administration/monitoring/

Backup with Point-in-time Recovery

Cluster-wide snapshots of sharded clusters

Restore to precisely the moment you need, quickly and safely with point-in-time restores

Manual Backup Considerations

Consider a hidden member in a Replica set

Consider EBS Snapshots

Consider Journaling (Write Ahead Log), and allow for DB durability in case of a fault

Ensure consistency by using db.fsyncLock()

https://docs.mongodb.com/ecosystem/tutorial/backup-and-restore-mongodb-on-amazon-ec2/

Visual Query Profiler

Identify the slow-running queries across your cluster with just the click of a button

Index suggestions to improve your query performance

Automate rolling index builds to reduce operational overhead and the risk of failovers

Resources

MongoDB on AWS best practices:http://docs.mongodb.org/ecosystem/platforms/amazon-ec2/

MongoDB production Noteshttp://docs.mongodb.org/manual/administration/production-notes/

MongoDB Documentationhttp://docs.mongodb.org

Thank you!