openstack ceilometer - meetupfiles.meetup.com/6706102/ceilometer introduction.pdf · 2013-10-21 ·...
TRANSCRIPT
![Page 1: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/1.jpg)
OpenStackCeilometer
Tong Li (IBM)
Brad Topol (IBM)
![Page 2: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/2.jpg)
Agenda• What is Ceilometer?
• Brief history of Ceilometer
• Meters of Ceilometer
• Workflow
• Ceilometer Architect & Building Blocks
Collecting Process Pipeline
Transformer Publisher
Collector Data store drivers
Alarm Dispatcher
• Ceilometer ReSTful API
• Ceilometer Python Client
• Ceilometer in Devstack
• Inter-Component Security & High Availability
• Who are the users of OpenStack Ceilometer
• Ceilometer Road Map
• Q & A (Demo maybe)
![Page 3: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/3.jpg)
Who are we?• Tong Li
– IBM senior software engineer
– Twitter @email4tong
– Email: [email protected]
• Brad Topol
– IBM distingushed engineer
– Twitter @bradtopol
– Email: [email protected]
![Page 4: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/4.jpg)
What is Ceilometer?
“OpenStack measurement project”
Julien Danjou Ceilometer PTL
A system that keeps track of what is happening in
OpenStack clusters. It was originally created for billing but it has been evolved into an all purpose
measurement repository. It can be extended for purposes such as performance report,
troubleshooting, capacity analytics, smart
scheduling, optimization, ultimately BIG DATA etc.
![Page 5: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/5.jpg)
Brief History of Ceilometer• Before year 2012 billing has been left out of OpenStack core so far
as it was not the primary problem and was non-trivial, yet almost every OpenStack deployment needs a way to track usage information
• Started in May 2012 by developers from Canonical, DreamHost, RedHat, Dell, Intel, AT&T, etc.
• Targeted to do metering, rating, billing
• Grizzly objective: The project aimed to become the infrastructure to collect measurements within OpenStack so that no two agents would need to be written to collect the same data.
• Havana objective: remains the same as Grizzly objective plus become an integrated OpenStack project.
• Current PTL: Julien Danjou
• 91 contributors, 7 core reviewers
![Page 6: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/6.jpg)
Meters of Ceilometer (from Nova)
![Page 7: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/7.jpg)
Meters of Ceilometer (from Neutron)
Additional measurements:
http://docs.openstack.org/developer/ceilometer/measurements.html
![Page 8: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/8.jpg)
Workflow
• Collect interesting info from OpenStack components
(or any other components), create meters (or samples)
• Transform meters into other meters if needed
• Publish meters to any destination (including Ceilometer itself)
• Store received meters into data store
• Read data via Ceilometer REST API
TransformCollect Publish Store Read
![Page 9: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/9.jpg)
Ceilometer Architecture
![Page 10: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/10.jpg)
Collecting
Neutron SwiftGlanceCinder
Notification bus (AMQP) queue name: notification.info
Notification Handlers, Central Agents
APIAPI API API
Nova
API
Publishing pipeline Publishing
Polling
![Page 11: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/11.jpg)
Ceilometercollector & agent
ReceiverReceiver
PublisherPublisher
Pipeline
TransformerMeterTransformer
Transformer
Publisher Receiver
Pipeline: a set of transformers mutating meters into something that publishers know how to send to external systems.
Configure pipelines @pipeline.yaml
New in Grizzly!
![Page 12: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/12.jpg)
Transformer
MeterName=cpu.time
Value=5Timestamp=T
Transformer
MeterName=cpu.time
Value=20
Timestamp=T+1
MeterName=cpu.time
Value=80
Timestamp=T+2
MeterName=cpu.percentage
Value=9Timestamp=T+2
Transform meters into new meters!
New in Grizzly!
![Page 13: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/13.jpg)
Publisher
MeterCeilometer
RPCPublisher
Other Publisher
CeilometerMessage queue
External system
AMQP
signed message
Billing, monitoring,
alarming, statistics,
capacity planning…
New in Grizzly!
![Page 14: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/14.jpg)
Data Store
Message Queueceilometer.collector.metering
SQL Database
MongoDB Database
Meter
Dispatchers
Multiple backend choices
Default backend
HBase
IBM DB2
New inGrizzly
New inHavana
![Page 15: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/15.jpg)
Alarm
Both alarm evaluator and alarm notifier are Ceilometer services
1. Alarm evaluator uses Ceilomter public APIs to pull meters to produce alarms according to alarm definition, then place the alarm message onto the alarm queue
2. Alarm notifier read the alarm queue then inform external systems (log, rest, test)
New inHavana
![Page 16: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/16.jpg)
DispatchersNew in
Havana
Multiple dispatchers support allows Ceilometer to be extended easily with many different outlet for all meters coming into Ceilometer. Dispatchers can be easily developed and deployed to add more features. For example, sending email, text messages.
1. Dispatcher only needs to implement one method.
2. By default, database dispatcher is used even if the configuration file has no trace of any dispatchers.
3. To configure multiple dispatchers:[ceilometer.dispatcher]file = ceilometer.collector.dispatcher.file:FileDispatcherdatabase = ceilometer.collector.dispatcher.database:DatabaseDispatcher
![Page 17: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/17.jpg)
Ceilometer ReSTful APIrequest
Database
Ceilometer APICeilometer API
Consumers
Billing, monitoring,
alarming, statistics, capacity planning…
HTTP
REST
w/ JSON &
XML
Storage abstraction
layer
Raw events:GET /v2/meters/vcpus
Statistics (sum, average, min, max…):GET /v2/meters/vcpus/statistics
Filter:GET /v2/meters/vcpus?q.field=project_id&q.value=foobar&q.op=eq
Group statistics by period:GET /v2/meters/vcpus/statistics?period=3600
New in Grizzly!
![Page 18: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/18.jpg)
Ceilometer ReSTful APIresponse
Response can be in JSON or XML
Raw events:/v2/meters/storage.object.size
Statistics (sum, average, min, max…):/v2/meters/storage.api.request/statistics
![Page 19: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/19.jpg)
Ceilometer Python Client
![Page 20: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/20.jpg)
Ceilometer in Devstack
devstack localrc
• Enable the ceilometer metering services
enable_service ceilometer-acompute,ceilometer-acentral,ceilometer-collector
• Enable the ceilometer alarming services
enable_service ceilometer-alarm-evaluator,ceilometer-alarm-notifier
• Enable the ceilometer api services
enable_service ceilometer-api
![Page 21: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/21.jpg)
Inter-Component Security
![Page 22: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/22.jpg)
Ceilometer High Availability
![Page 23: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/23.jpg)
Who uses Ceilometer?
https://etherpad.openstack.org/p/CeilometerFeeedback
![Page 24: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/24.jpg)
Roadmap
Havana Icehouse
• Integrated Project ✓• Integration with Horizon• Publishing meters to other
systems ✓• Enhance SQL driver ✓• Alarming ✓• Integration with Heat• Deprecating APIv1 ✓• Completing APIv2 ✓• Move publishing part to
Oslo and other projects ✓• Tighter integration with
Nova ✓• Nova-scheduler integration
✓
• Incubated Project ✓• Integration
with Horizon ❌• Agents for other
componentso Swift ✓o Ceph? ❌o Nicira? ❌
• SQLAlchemy storage driver ✓
• Multi-Publisher ✓• API v2 ✓
o User accessible API
✓
o More aggregation ✓
o Multi-dimension ✓
Grizzly
• OpenStack Design Summit in Hong Kong
![Page 25: OpenStack Ceilometer - Meetupfiles.meetup.com/6706102/Ceilometer Introduction.pdf · 2013-10-21 · Brief History of Ceilometer • Before year 2012 billing has been left out of OpenStack](https://reader035.vdocuments.net/reader035/viewer/2022081521/5e6bff9ed9bf3d5b7d4aee29/html5/thumbnails/25.jpg)
Questions?
http://launchpad.net/ceilometer
http://docs.openstack.org/developer/ceilometer
http://wiki.openstack.org/ceilometer
Freenode: #openstack-metering
Mailing List: openstack-dev [ceilometer]