how to monitor mongodb
DESCRIPTION
David Mytton is a MongoDB master and the founder of Server Density. In this presentation David delves deeper into what's discussed in our how to monitor MongoDB tutorial (https://blog.serverdensity.com/monitor-mongodb/), with the aim of taking you through: Key MongoDB metrics to monitor. Non-critical MongoDB metrics to monitor. Alerts to set for MongoDB on production. Tools for monitoring MongoDB.TRANSCRIPT
![Page 1: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/1.jpg)
How to monitor:MongoDB
David MyttonHangout on Air - Sept 2014
https://blog.serverdensity.com/monitor-mongodb/
![Page 2: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/2.jpg)
David Mytton
![Page 3: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/3.jpg)
![Page 4: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/4.jpg)
Server Density Architecture
![Page 5: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/5.jpg)
Server Density Architecture
● ~100 servers - Ubuntu 12.04
![Page 6: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/6.jpg)
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
![Page 7: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/7.jpg)
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 200TB/m processed data
![Page 8: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/8.jpg)
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● Nginx, Python, MongoDB
● 200TB/m processed data
![Page 9: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/9.jpg)
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 200TB/m processed data
● Nginx, Python, MongoDB
● Softlayer > 1TB RAM, 5TB SSDs
![Page 10: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/10.jpg)
Key metrics
● Oplog replication lag
● Replica state
● Lock %
● Disk i/o % utilization
![Page 11: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/11.jpg)
Oplog replication lag
● Replica sets: master/slave
![Page 12: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/12.jpg)
Oplog replication lag
● Replica sets: master/slave
● Async i.e. eventually consistent
![Page 13: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/13.jpg)
Oplog replication lag
● Replica sets: master/slave
● Async i.e. eventually consistent
● Write concern
![Page 14: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/14.jpg)
Oplog replication lag
![Page 15: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/15.jpg)
Oplog replication lag
https://blog.serverdensity.com/mongodb-benchmarks/
![Page 16: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/16.jpg)
Oplog replication lag
● Replica sets: master/slave
● Async i.e. eventually consistent
● Write concern
● Falling behind
![Page 17: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/17.jpg)
Reasons for repl falling behind
● Network problems
![Page 18: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/18.jpg)
Reasons for repl falling behind
● Network problems
● Hardware problems
![Page 19: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/19.jpg)
Reasons for repl falling behind
● Network problems
● Hardware problems
● Shard chunk migrations
![Page 20: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/20.jpg)
Reasons for repl falling behind
● Network problems
● Hardware problems
● Shard chunk migrations
● MongoDB bugs
![Page 21: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/21.jpg)
Replica state
● Primary / secondary
![Page 22: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/22.jpg)
Replica state
● Primary / secondary
● Alert on state change
![Page 23: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/23.jpg)
Lock %
● Database locking (2.6)
![Page 24: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/24.jpg)
Lock %
● Database locking (2.6)
● Sometimes a problem:
![Page 25: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/25.jpg)
Lock %
● Database locking (2.6)
● Sometimes a problem:
● Nearing 100%
![Page 26: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/26.jpg)
Lock %
● Database locking (2.6)
● Sometimes a problem:
● Nearing 100%
● Constantly high
![Page 27: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/27.jpg)
Lock %
● Database locking (2.6)
● Sometimes a problem:
● Nearing 100%
● Constantly high
● Slows replication
![Page 28: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/28.jpg)
Disk i/o % utilization
● Hardware limits
![Page 29: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/29.jpg)
Disk i/o % utilization
● Hardware limits
● Nearing 100%
![Page 30: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/30.jpg)
Disk i/o % utilization
● Hardware limits
● Nearing 100%
● Constantly high
![Page 31: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/31.jpg)
Disk i/o % utilization
● Hardware limits
● Nearing 100%
● Constantly high
● Spinning -> SSD
![Page 32: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/32.jpg)
Disk i/o % utilization
https://blog.serverdensity.com/mongodb-performance-ssds-vs-spindle-sas-drives/
![Page 33: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/33.jpg)
Disk i/o % utilization
https://blog.serverdensity.com/mongodb-benchmarks/
![Page 34: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/34.jpg)
Disk i/o % utilization
● Hardware limits
● Nearing 100%
● Constantly high
● Spinning -> SSD
● Slow queries, hangs, slow repl
![Page 35: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/35.jpg)
Non-critical metrics to watch
● Memory usage
![Page 36: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/36.jpg)
Non-critical metrics to watch
● Memory usage
● Page faults
![Page 37: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/37.jpg)
Non-critical metrics to watch
● Memory usage
● Page faults
● Connections
![Page 38: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/38.jpg)
Non-critical metrics to watch
● Memory usage
● Page faults
● Connections
● Shard chunk distribution
![Page 39: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/39.jpg)
Non-critical metrics to watch
![Page 40: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/40.jpg)
Monitoring tools
● mongostat
● mongotop
● rs.status()
● sh.status()
![Page 41: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/41.jpg)
rs.status()
![Page 42: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/42.jpg)
sh.status()
![Page 43: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/43.jpg)
Server Density
![Page 44: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/44.jpg)
MMS
![Page 45: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/45.jpg)
Summary
● Critical alerts on key metrics
![Page 46: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/46.jpg)
Key metrics
● Oplog replication lag
● Replica state
● Lock %
● Disk i/o % utilization
![Page 47: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/47.jpg)
Summary
● Critical alerts on key metrics
![Page 48: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/48.jpg)
Summary
● Critical alerts on key metrics
● Watch non-critical
![Page 49: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/49.jpg)
Summary
● Critical alerts on key metrics
● Watch non-critical
● Manual tools for real time
![Page 50: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/50.jpg)
Summary
● Critical alerts on key metrics
● Watch non-critical
● Manual tools for real time
● Set up a monitoring product
![Page 51: How to monitor MongoDB](https://reader031.vdocuments.net/reader031/viewer/2022020217/55625239d8b42a6c368b5060/html5/thumbnails/51.jpg)
Useful resources
● http://docs.mongodb.org/manual/administration/monitoring/
● https://blog.serverdensity.com
● https://blog.serverdensity.com/monitor-mongodb