MongoDB 3.4The Leading Non-Relational

Database Evolved

Jason MaAndrew Morgan

Introduction

Multi-model done right

Mission Critical Applications

Modernized tooling

Q&A

Agenda

Evolution of MongoDB

Hash-Based ShardingRolesKerberosOn-Prem Monitoring

2.4 2.6 3.0 3.2

$outIndex IntersectionText SearchField-Level RedactionLDAP & x509Auditing

Document Validation$lookupFast FailoverSimpler ScalabilityAggregation ++Encryption At RestIn-Memory Storage EngineBI ConnectorMongoDB CompassAPM IntegrationProfiler VisualizationAuto Index BuildsBackups to File System

Doc-Level ConcurrencyCompressionStorage Engine API≤50 replicasAuditing ++Ops Manager

Intra-cluster compressionViewsLog RedactionLinearizable ReadsGraph ProcessingDecimalCollations Faceted NavigationSpark Connector ++Zones ++Aggregation ++Auto-balancing ++ARM, Power, zSeries supportBI Connector ++Compass ++Hardware MonitoringServer PoolLDAP AuthorizationEncrypted BackupsCloud Foundry Integration

3.4

Multi-model done right. Other vendors sell you multiple products. MongoDB

gives you multiple models in one database: document, graph, key value, and

search with faceted navigation.

Mission-critical apps. Stronger security, broader platform support, and Zones

make MongoDB ready for the most demanding mission-critical deployments.

Modernized tooling. A sophisticated range of tools and integrations provide

powerful capabilities for Data Analysts, DBAs, and Operations teams.

MongoDB 3.4 Themes

Multi-Model Done Right

• Enables processing of graph data natively within MongoDB with $graphLookup operator

• Uncover indirect or transitive relationships in operational data

• Recommendation engines, MDM, fraud models, social networks, etc.

Graph Processing

$graphLookup Example

_id: 3

friends[0].id: 193

db.users.aggregate( [ {$match: {_id: 3}}, {"$graphLookup": {

from: "users", startWith: "$friends.id", connectFromField: "friends.id", connectToField: "_id", maxDepth: 2, depthField: "depth", as: "target"}

} ], {allowDiskUse: true}).pretty()

Graph Processing

{ "target": { "_id": 678, "meetupCount": 98, "details": [ { "firstName": "Jesse", "lastName": "Bennett" } ] }, "requestIntroFrom": { "id": 541, "details": [ { "firstName" : "Arthur", "lastName" : "Boyd", "contact" : { "email" : "aboydf0@canalblog.com", "phone" : "86-(656)576-9389"}}]}}

Graph Processing

https://github.com/am-MongoDB/meetupGraph

Faceted Navigation

• Grouping data into related categories for intuitive exploration & discovery

• Used in search and analytics applications • New aggregation pipeline stages for faceting,

bucketing & sorted counts across multiple dimensions

• Eliminates requirement for external search engine

https://github.com/am-MongoDB/TVnav

Faceted Navigation

Faceted Navigation

Faceted Navigationdb.products.aggregate([

{$match: {category: "TV"}},{$facet: {

price: [{$bucket: {

groupBy: "$price",boundaries: [0, 200, 699, 1000, 1500],default: "Over 1500",output: {"count": {$sum: 1}}

}},{$project: {

lowerPriceBound: "$_id",count: 1,_id: 0}}],

brands: [{$sortByCount: "$brand" },{$project: {

brand: "$_id",count: 1,_id: 0}}],

…}}

Faceted Navigation{"price" : [

{"count" : 29, "lowerPriceBound" : 0},{"count" : 272, "lowerPriceBound" : 200},{"count" : 165, "lowerPriceBound" : 699},{

"count" : 260,"lowerPriceBound" : 1000

},{"count" : 274, "lowerPriceBound" : "Over 1500"

],"brands" : [

{"count" : 277, "brand" : "SAMSUNG"},{"count" : 169, "brand" : "PANASONIC"},

• Data set, query and results:• https://github.com/am-MongoDB/TVnav

Faceted Navigation

• Powerful data processing pipeline for analytics & transformations

• 25+ enhancements simplify app code

• Performance improvements with query optimizer moving $match stage earlier to use indexes

New StagesArray

OperatorsString & Date

Operators

$graphLookup

$facet

$bucket

$bucketAuto

$sortbyCount

$addFields

$replaceRoot

$in

$indexOfArray

$range

$reverseArray

$reduce

$zip

$indexOfBytes

$indexOfCP

$split

$strLenBytes

$strLenCP

$substrBytes

$substrCP

$isoDayOfWeek

$isoWeek

$isoWeekYear

Advanced AnalyticsAggregation Framework Enhancements

• Create powerful visualizations & analytics from SQL-based BI tooling

• Auto-schema sampling

• Eliminates ETL

• Higher performance with rewritten SQL layer• More processing pushed down to the database

• Simplified installation and authentication

Advanced AnalyticsMongoDB Connector for BI

“We reduced 100+ lines of integration code to just a single line after moving to the MongoDB Spark connector.”- Early Access Tester, Multi-National Banking Group Group

Analytics Application

Scala, Java, Python, R APIs

SQLMachine Learning Libraries

Streaming Graph

Spark Worker

Spark Worker

Spark Worker

Spark Worker

MongoDB Connector for Spark

Advanced AnalyticsMongoDB Connector for Apache Spark

• Native Scala connector, certified by Databricks • Exposes all Spark APIs & libraries • Efficient data filtering with predicate pushdown,

secondary indexes, & in-database aggregations

• Locality awareness to reduce data movement • Updated with Spark 2.0 support

• Text comparisons and sorting using language-specific rules

• 100+ different languages & locales • Over 2x as many as offered by most RDBMs

• Case sensitivity, treatments of numbers• Collection | View | Index | Operation

Collations

db.customers.createIndex( {last_name: 1, first_name : 1 }, {name: "german_name_index", collation: {locale: "de"}, partialFilterExpression: {

language: "German" }})

db.createView( "germanSpeakers", "customers", [{$match: { language: "German", last_name: {$exists: true}}}], {collation: {locale: "de"}})

Collations

• Support for the IEEE 754-2008 decimal128 type in server and drivers

• Database stores exact values to eliminate rounding errors for high-precision calculations, complex financial & scientific apps

Decimal128

Decimal Data Type

Mission-Critical Applications

Partition data across distributed clusters based on data locality policies• Support distributed local writes• Easily adhere to data sovereignty requirements• Enable deployment patterns such as tiered

storage

MongoDB Zones can now be configured visually from Ops Manager

Zones

Seamless and elastic scalability in response to dynamic application demands • Improved balancing of data across nodes with

parallel data migrations• Faster replica set synchronization with optimized

initial sync process

Elastic Clusters

With snappy compression algorithm, network traffic can be compressed by up to 70%• Performance benefits in bandwidth-constrained

environments • Significantly reduce network costs

Intra-Cluster Compression

maxStalenessMS Read Preference• Choose how and when to route queries to secondary replicas• Only read from replicas that are within a defined consistency

window• Improved data quality while scaling reads across secondaries

linearizable Read Concern for the strongest consistency guarantees of any database

• Ensure that a node is the primary at the time of read• Ensure that data returned will not be rolled back if another

node is subsequently elected as primary

Improved Tunable Consistency

LDAP authentication & authorization reduces administrative overhead & TCO • Users stored in LDAP server can be mapped to

MongoDB roles• Native platform libraries to integrate with LDAP;

no need for external dependencies and configuration

LDAP Authorization

Define dynamically generated non-materialized views to expose selected data from underlying collection(s)• Reduces risk of sensitive data exposure

• Separately specified permissions levels • Allows organizations to more easily meet compliance

standards in regulated industries• Can generate additional fields

Read-Only Views

Collation/Views Exampledb.createView( "germanSpeakersRedacted", "customers", [ {$match: { language: "German", last_name: {

$exists: true}}}, {$project: { salary: 0, country: 0, language: 0 } } ], {collation: {locale: "de"}})

use admindb.createRole( { role: "germanViewer", privileges: [ {resource: { db: "production", collection: germanSpeakersRedacted"}, actions: ["find"]} ], roles: [] })

Collation/Views Example

Support the growing demand to run the database on a more diverse range of platforms

• ARM v8-64 bit support allows customers to take advantage of power-efficient servers being deployed into ultra dense data center racks

• IBM Power8 and zSeries support provides seamless migration for enterprises modernizing legacy workloads. Available for MongoDB Enterprise Server.

Expanded Platform Support

Modernized Tooling

• MongoDB 3.2• Visualize Indexes• Query Performance

• MongoDB 3.4• MongoDB Compass enhancements

• Modify documents

• Create document validation rules

• Optimize query performance with visual explain plans, index usage, and real-time statistics

• Create and remove indexes

• All controlled from a single intuitive and sophisticated GUI

MongoDB Compass 3.4

Real Time Performance Stats• View key

MongoDB metrics that are happening in real-time

• Operation statistics (inserts, deletes, queries, etc)

• Read/Write Load, Network load, Memory usage

Insert, Delete, Modify Documents

● Clone, modify existing documents, insert new documents, and delete existing documents

● For now, applicable to one document at time

Document Validation Rule Builder

● MongoDB Compass allows users to view, add, and modify document validation rules through its GUI

● Different teams can enforce schema consistency, reducing risk and increasing productivity between teams

Visualize Explain Plans● View key

information about the execution of a query

● Identify slow running queries and optimize query performance

View Index Utilization● Visualization of

index utilization● View type of

index, size of index, usage of index, and properties of index

Create and Remove Indexes● Create and

Remove indexes in GUI

● Enhances ability for users to identify and fix issues from the GUI

MongoDB Ops Manager 3.4Ops Manager allows you leverage and automate the best practices we’ve learned from thousands of deployments in a comprehensive application that helps you run MongoDB safely and reliably.

Benefits include:

10x-20x more efficient operations

Complete performance visibility

Protection from data loss

Assisted performance optimization

Meet SLAs, Cut management overhead

Scale easily, Follow best practices

Without Ops Manager● Install, Configure

○ 150+ steps

● Upgrades, downgrades○ 100+ steps

● Scale out, move servers, resize oplog, etc○ 10-180+ steps

With Ops Manager

or

Ops Mgr (app)

150+ steps -> 1 click

Ops Mgr (API)

• Ops Manager agent installed to pool via configuration management tools

• Server pools exposed to internal teams, ready for provisioning into local groups

• Allow administrators to create true, on demand database resources for private cloud environments

Ops Manager Server Pools to Automate Provisioning of MongoDB

• MongoDB and Pivotal co-engineered Cloud Foundry integration

• BOSH installs the Ops Manager agent

• Calls the Ops Manager API to provision, configure, monitor & backup the cluster

• Enables users to integrate MongoDB as an on-demand DBaaS resource within Cloud Foundry platforms

Ops Manager Cloud Foundry Integration

• Finer grained telemetry data: collected every 10 seconds vs every 60 seconds

• Configurable retention policies

• Simplified & extended management

• Single agent to collect both database and hardware telemetry

• Hardware metrics now collected for Windows & OSX hosts

Ops Manager Higher Resolution Monitoring

• Atlas and Cloud Manager have their own encrypted backup solution that is dependent on encrypted hardware

• Ops Manager 3.4

• Native backup encryption solution for PII information (HIPAA, PCI, FERPA, etc)

• Encrypt backup snapshots; encryption keys stored on KMIP server

Ops Manager Encrypted Backups

KMIP Appliance

Master Key

Multi-model done right. Other vendors sell you multiple products. MongoDB

gives you multiple models in one database: document, graph, key value, and

search with faceted navigation.

Mission-critical apps. Stronger security, broader platform support, and Zones

make MongoDB ready for the most demanding mission-critical deployments.

Modernized tooling. A sophisticated range of tools and integrations provide

powerful capabilities for Data Analysts, DBAs, and Operations teams.

MongoDB 3.4 Themes

• Take the

M034: New Features and Tools in MongoDB 3.4 training• https://university.mongodb.com/courses/M034/about

• Read the What’s New in MongoDB 3.4 Whitepaper• https://www.mongodb.com/collateral/mongodb-3-4-whats-new

3.4

Next Steps

Backup

Content Repo IoT Sensor Backend Customer Data Real-Time

AnalyticsSingle View

MongoDB Query Language: K-V, Aggregations & Transformations, Graph, Faceted Navigation, JOINs, Geospatial

MongoDB Data Model: Document, Key-Value, Graph, Tabular, Files

WT MMAP

Available in MongoDB 3.4

Man

agem

ent

Sec

urity

In-memory Encrypted 3rd party

Multimodel Done Right