open postgresql monitoring - dalibo.org postgresql monitoring a brand new & free monitoring...
TRANSCRIPT
![Page 1: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/1.jpg)
Open PostgreSQLMonitoring
A brand new & free monitoring suite
Jean-Paul Argudo / Julien Rouhaud
Dalibo - www.dalibo.org
PostgreSQL Conference Europe 2014 - October, 22
1 / 26
![Page 2: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/2.jpg)
OriginsComparison with Oracle
I Multimaster : Bi-Directional Replication (BDR)(2014-07)
I Materialized views : version 9.3 (2013-09)I Easy partitionning : TODOI Parallel queries : TODOI AWR : PoWAI Oracle Enterprise Manager, Oracle Grid : OPM!
2 / 26
![Page 3: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/3.jpg)
State of the ArtA tool is missing
I proprietary softwareI existing Open Source projects to assembleI no real Open Source tool dedicated to PostgreSQL
3 / 26
![Page 4: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/4.jpg)
Main ideas
To be accepted by the community the tool must beI completely free and Open SourceI crafted for PostgreSQLI extensible with a ”pluggable” architecture
4 / 26
![Page 5: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/5.jpg)
Objectives
I a powerful monitoring platformI the standard for PostgreSQL monitoringI must compete with proprietary software
5 / 26
![Page 6: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/6.jpg)
Functionalities
I an extensible architectureI Suitable probesI graphsI alerts
6 / 26
![Page 7: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/7.jpg)
An active project
I OPM has 5 releases alreadyI 13 releases for check_pgactivityI fast pace dev : release often, release early
7 / 26
![Page 8: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/8.jpg)
HistoryVersion 1.X
I creation of the architecture (opm_core and wh_nagios)I first metrics in check_pgactivityI graphs management
I aspect (Stacked, filled, etc)I human-readable unitsI export graphs into images
8 / 26
![Page 9: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/9.jpg)
HistoryVersion 2.0 to 2.3
I new designI new authentication and grant/revoke methodsI pluggable architecture to accept pluginsI tags managementI bug fixes
9 / 26
![Page 10: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/10.jpg)
OPM in details1/2
I A long projectI Composed of some new projectsI Still using some well-known ones, like Nagios
10 / 26
![Page 11: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/11.jpg)
OPM in details2/2
I A graphing engine opm_coreI Modules to store data, at the moment, only
wh_nagiosI Probes dedicated to PostgreSQL : check_pgactivity
11 / 26
![Page 12: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/12.jpg)
Graphing engine1/3
I PostgreSQL extension (opm_core) written in pl/pgsqlI web GUI in Perl (Mojolicious) and JavascriptI access control
12 / 26
![Page 13: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/13.jpg)
Graphing engine2/3
I discovers automatically the data found (servers,services, etc)
I generates automatically the associated graphs, andassociates them to the correspond server
I tags to aggregate various graphs
13 / 26
![Page 14: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/14.jpg)
Graphing engine3/3
I an API to facilitate writing to the warehouseI sampling functionI storing tables to inheritI . . .
14 / 26
![Page 15: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/15.jpg)
Authorizations
I groups and UsersI 1 server = 1 groupI each user is part of one or many groupsI two levels of users : Administrator and User
15 / 26
![Page 16: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/16.jpg)
wh_nagiosStorage module
I standard warehouse dedicated to Nagios data(perfdata)
I storing and archiving of all the valuesI contains a dispatcher to handle Nagios data
16 / 26
![Page 17: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/17.jpg)
check_pgactivityNagios probe (1/3)
I standard Nagios probeI very simple code, and an APII performance data oriented to graphing
I managing of units in the perfdatasI the right metric is sent, no transformation needed
17 / 26
![Page 18: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/18.jpg)
check_pgactivityNagios probe (2/3)
I better or new probes, based on our experienceI local file storage to compute deltasI probe on bgwriterI writing speed into WALsI . . .
18 / 26
![Page 19: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/19.jpg)
check_pgactivityNagios probe (3/3)
I still many probes to invent or addI roadmap
I make it more modularI create a dedicated probe for the system
19 / 26
![Page 20: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/20.jpg)
Overview
20 / 26
![Page 21: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/21.jpg)
Create a warehouse1/3
I specify a schemaI table inheritance
I servicesI metricsI series
I And all the warehousing necessary
21 / 26
![Page 22: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/22.jpg)
Create a warehouse2/3
I implement functionsI get_metric_data()I (grant/revoke)_dispatcher()I purge_service()
I and all the warehousing necessary
22 / 26
![Page 23: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/23.jpg)
Create a warehouse3/3
I optional, add elements in the interfaceI decorateors, e.g. :
I details_server()I details_service()
I or dedicated screensI e.g. : Administration screens wh_nagios
23 / 26
![Page 24: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/24.jpg)
Roadmap
I remove the dependency on Nagios or another toolI push and pull modes supportI have the dashboard customizableI metrics correlationI and much more
24 / 26
![Page 25: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/25.jpg)
Some numbers
I more than 100 servers generatesI more than 2000 servicesI more than 17000 metrics
I over 9 monthsI represents 8.6 Gb of data
25 / 26
![Page 26: Open PostgreSQL Monitoring - dalibo.org PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo- PostgreSQLConferenceEurope2014-October,22](https://reader031.vdocuments.net/reader031/viewer/2022022010/5b0627717f8b9ac33f8c691c/html5/thumbnails/26.jpg)
Demo
Because a good demo is better than boring slides !I demo.opm.ioI login : opmI pass : demo
More informations on opm.io
26 / 26