Consulting Manager, 10gen Inc

Richard Kreuter

Machine to Machine: Managing Mechanically

Generated Data with MongoDB

Agenda

• What's Machine to Machine Data?• What's MongoDB?• Some use cases• Next steps

What’s Machine to Machine About?

Why this is game-changing

• Resource tracking• Process optimization• Market analysis• Real-time decision making• etc.– (… including things we haven’t thought of yet…)

Why this is technically challenging

• Massive volumes of data• High data ingestion rates• Complex data analysis requirements• Evolving data modeling needs

What’s MongoDB About?

MongoDB is a ___________ database

• Open source• High performance• Full featured• Document-oriented• Horizontally scalable

Full Featured

• Dynamic (ad-hoc) queries• Built-in online aggregation• Rich query capabilities• Traditionally consistent• Many advanced features• Support for many programming languages

Document-Oriented Database

• A document is a nestable associative array

• Document schemas are flexible

• Documents can contain various data types (numbers, text, timestamps, blobs, etc)

Horizontally Scalable (Sharding)

Replication within a shard

Use Case #1: Keep on Trucking

Suppose you’ve got a lot of trucks

• You probably care about where they are

• You probably care if they’re on schedule

• You might also care what they’re carrying (cargo and/or fuel)

What are these data?

• Vehicle tracking data is positional info available via GPS

• Cargo/fuel might be continuous sensor data (e.g., volume or weight) or inventory (e.g., RFID)

What MongoDB can do here

• For GPS type data, MongoDB has long had powerful geospatial query facilities:

– “Find all trucks within a region (within a certain time range, perhaps)”

– “Find all trucks within 100km from the warehouse/customer”

Recent additions to GeoSpatial

• MongoDB version 2.4 introduces support for indexing and querying on various GeoJSON data types (polygons, line strings)

– “Find trucking routes that intersect”

– “Find routes that pass nearby to a customer”

What about the cargo/fuel?

• Sensor data is an easy fit for MongoDB

• Cf. MMS, the MongoDB Monitoring Service– Cloud service hosted by 10gen since 2011– On the order of 1M measurements written per

second (i.e., 10B writes/day)– Continuous data rendering/alerting/analysis of

ingested data, 24x7

Use Case #2: Keeping Cool

Suppose you’ve got a building

• You probably keep it climate controlled…

• … and lit …

• … and perhaps secured with entry cards.

So you measure things

• Temperature readings and/or HVAC utilization reports

• Lights on/off

• Swipe-ins/swipe-outs through secured doors

This is “just” sensor data

• Straightforward to store in MongoDB documents

• With strategic document design, a single server can save hundreds of thousands of sensor reads per second

But how do you use this data?

• MongoDB has a built-in Aggregation Framework that supports ad-hoc analysis tasks over data sets

– “What rooms had the highest average Air Conditioning utilization bracketed daily?”

– “Which secured doors have the most ‘pass-back’ problems?”

Replication supports analytics

• E • Queries can be parceled out to different replicas

• In different DCs, say,• Or to segregate

competing workloads

Use Case #3: Who are the clients in your neighborhood?

Suppose you’ve mobile customers

• You might know where they are during their day

• You might want to pitch them offers while they’re there– Or perhaps notify partners in real-time

• And you might evolve your model progressively

Tracking locations is straightforward

• We’ve already discussed GPS data– Recording positional information– Geospatial queries for proximity etc.

• Though mobile customers might have more interesting locational data than just GPS– e.g., user U is in retail store S now– or, user U1 is near to U2 now

Evolving your customer model

• One interesting thing about people is that they differ

• And you might choose to pay attention to changing things as your business evolves

• MongoDB makes this easy

Flexible Schema supports Agility

• As you become interested in new data, you include that data in your documents– No laborious data migration processes are

necessary – just adapt the application models to record new information

– Queries and indexes work over polymorphic data models, too

What’s next?

Going further

• Where do you build your next warehouse/place your next office?

• How should you staff your retail spaces/factory floors?

• What will next year’s smart home products enable?

To recap

• MongoDB is widely used for machine-generated sensor, GPS, inventory, and other kinds of data

• Dynamic schema, rich query facilities, built-in analytic features, replication and horizontal scaling simplify M2M architectures

Consulting Manager, 10gen

Richard Kreuter

Questions?Next webinar:

How Financial Firms Create a Single Customer View with MongoDB

http://bit.ly/mgoSCV