1. 1. OCCI Monitoring Augusto Ciuoletti OCCI Monitoring Extending the OCCI API with monitoring capabilities Augusto Ciuoletti Dept. of Computer Science - Univ. of Pisa September 13, 2013
2. 2. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence
3. 3. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence SLA is tightly related with monitoring, so we start from cloud monitoring
4. 4. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring)
5. 5. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring)
6. 6. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring) In both cases, the user needs the tools to dene resource monitoring
7. 7. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring) In both cases, the user needs the tools to dene resource monitoring Keep into account the case of a composite service (many providers)
8. 8. OCCI Monitoring Augusto Ciuoletti Motivation SLA is a dened target to obtain user condence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring) In both cases, the user needs the tools to dene resource monitoring Keep into account the case of a composite service (many providers) A simple API aligned with OCCI
9. 9. OCCI Monitoring Augusto Ciuoletti Basic functions Monitoring is made of three basic activities
10. 10. OCCI Monitoring Augusto Ciuoletti Basic functions Monitoring is made of three basic activities extract operational parameters from a Resource
11. 11. OCCI Monitoring Augusto Ciuoletti Basic functions Monitoring is made of three basic activities extract operational parameters from a Resource gather the operational parameters to obtain the measure of a metric of interest
12. 12. OCCI Monitoring Augusto Ciuoletti Basic functions Monitoring is made of three basic activities extract operational parameters from a Resource gather the operational parameters to obtain the measure of a metric of interest deliver the measurement to the relevant party
13. 13. OCCI Monitoring Augusto Ciuoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources
14. 14. OCCI Monitoring Augusto Ciuoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider
15. 15. OCCI Monitoring Augusto Ciuoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider Candidate for the introduction of a new kind of resource: the Sensor
16. 16. OCCI Monitoring Augusto Ciuoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider Candidate for the introduction of a new kind of resource: the Sensor The user that wants to exert monitoring instantiates (and pays for) a Sensor
17. 17. OCCI Monitoring Augusto Ciuoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider Candidate for the introduction of a new kind of resource: the Sensor The user that wants to exert monitoring instantiates (and pays for) a Sensor Note: a cost is associated to the Sensor, although this is not explicit in the denition of monitoring
18. 18. OCCI Monitoring Augusto Ciuoletti Basic functions The monitoring function is controlled by time
19. 19. OCCI Monitoring Augusto Ciuoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are:
20. 20. OCCI Monitoring Augusto Ciuoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement
21. 21. OCCI Monitoring Augusto Ciuoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement During which time lapse it performs the measurements
22. 22. OCCI Monitoring Augusto Ciuoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement During which time lapse it performs the measurements Too many variants for aggregation/delivery: OCCI-mixins
23. 23. OCCI Monitoring Augusto Ciuoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement During which time lapse it performs the measurements Too many variants for aggregation/delivery: OCCI-mixins Mixins are classied using tags (sort of subtyping)
24. 24. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource
25. 25. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation)
26. 26. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource
27. 27. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period:
28. 28. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period: The sampling period
29. 29. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period: The sampling period The accuracy of the sampling period
30. 30. OCCI Monitoring Augusto Ciuoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period: The sampling period The accuracy of the sampling period OCCI-mixins are introduced to specify the measurement technique used
31. 31. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label
32. 32. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specication
33. 33. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specication By dening tagged mixins, the provider describes the monitoring capabilities oered to the user
34. 34. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specication By dening tagged mixins, the provider describes the monitoring capabilities oered to the user Next slides describe three such tags:
35. 35. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specication By dening tagged mixins, the provider describes the monitoring capabilities oered to the user Next slides describe three such tags: metric specic for collector links
36. 36. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specication By dening tagged mixins, the provider describes the monitoring capabilities oered to the user Next slides describe three such tags: metric specic for collector links aggregator specic for sensor resources
37. 37. OCCI Monitoring Augusto Ciuoletti Mixin sub-typing: the tags According with the OCCI core specication a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specication By dening tagged mixins, the provider describes the monitoring capabilities oered to the user Next slides describe three such tags: metric specic for collector links aggregator specic for sensor resources publisher specic for sensor resources
38. 38. OCCI Monitoring Augusto Ciuoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link;
39. 39. OCCI Monitoring Augusto Ciuoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link; The mixin that has the metric tag has the following attributes:
40. 40. OCCI Monitoring Augusto Ciuoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link; The mixin that has the metric tag has the following attributes: metric attribute strings that are used as an identier to refer to the output measurement stream
41. 41. OCCI Monitoring Augusto Ciuoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link; The mixin that has the metric tag has the following attributes: metric attribute strings that are used as an identier to refer to the output measurement stream control attributes used to control the measurement process (e.g. packet length of a ping)
42. 42. OCCI Monitoring Augusto Ciuoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation
43. 43. OCCI Monitoring Augusto Ciuoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource
44. 44. OCCI Monitoring Augusto Ciuoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by:
45. 45. OCCI Monitoring Augusto Ciuoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by: input attributes strings that are used as an identier to refer to input measurement streams
46. 46. OCCI Monitoring Augusto Ciuoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by: input attributes strings that are used as an identier to refer to input measurement streams control attributes that are parameters for the aggregation function (e.g. the gain in an EWMA)
47. 47. OCCI Monitoring Augusto Ciuoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by: input attributes strings that are used as an identier to refer to input measurement streams control attributes that are parameters for the aggregation function (e.g. the gain in an EWMA) metric attributes strings that are used as an identier to refer to the output measurement stream
48. 48. OCCI Monitoring Augusto Ciuoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure
49. 49. OCCI Monitoring Augusto Ciuoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource
50. 50. OCCI Monitoring Augusto Ciuoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource the mixin with a publisher tag are characterized by:
51. 51. OCCI Monitoring Augusto Ciuoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource the mixin with a publisher tag are characterized by: input attribute a string that is used as an identier to refer to input measurement stream
52. 52. OCCI Monitoring Augusto Ciuoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource the mixin with a publisher tag are characterized by: input attribute a string that is used as an identier to refer to input measurement stream control attributes that are parameters for the publishing function (e.g. the UDP port that is used for streaming)
53. 53. OCCI Monitoring Augusto Ciuoletti The role of metric and input attributes they describe the ow of monitoring data
54. 54. OCCI Monitoring Augusto Ciuoletti The role of metric and input attributes they describe the ow of monitoring data their values are unique identiers in a scope corresponding to a sensor and all outgoing collectors
55. 55. OCCI Monitoring Augusto Ciuoletti The role of metric and input attributes they describe the ow of monitoring data their values are unique identiers in a scope corresponding to a sensor and all outgoing collectors the provider is free to implement the transport of the measurement stream in the most convenient way
56. 56. OCCI Monitoring Augusto Ciuoletti Shortcuts for simple use cases here metric mixin, can be associated directly to the monitored resource and has a web service interface
57. 57. OCCI Monitoring Augusto Ciuoletti Shortcuts for simple use cases here metric mixin, can be associated directly to the monitored resource and has a web service interface a tagged mixin can be associated with a generic resource or link, avoiding the denition of a sensor or collector
58. 58. OCCI Monitoring Augusto Ciuoletti Shortcuts for simple use cases here metric mixin, can be associated directly to the monitored resource and has a web service interface a tagged mixin can be associated with a generic resource or link, avoiding the denition of a sensor or collector shortcuts eectively help very simple use cases, but are generally a source of ineciency and complexity
59. 59. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure The resources we want to monitor
60. 60. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Create one Sensor resource
61. 61. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Use two collectors to dene the measurement activiy
62. 62. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Associate two metric mixins to the Collector X
63. 63. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure And another two metric mixins to the Collector Y
64. 64. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Associate two aggregator mixins to the Sensor
65. 65. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure One publisher is going to use raw data from the collector
66. 66. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Another is going to receive measurements from the aggregators
67. 67. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure A frame for Collector X and its mixins
68. 68. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure ... one for Collector Y...
69. 69. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure ... one for the sensor
70. 70. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure The scope of the Sensor (for metric streams ids)
71. 71. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Feeding the aggregators: a,b,d are measurement stream ids
72. 72. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Feeding publisher 2: aggregated (f,g) and raw data (e)
73. 73. OCCI Monitoring Augusto Ciuoletti Step by step design of a monitoring infrastructure Feeding publisher 1: measurement stream b is multicast
74. 74. OCCI Monitoring Augusto Ciuoletti Also in the document Conformance proles: to accomodate the presence of providers that do not implement a monitoring interface
75. 75. OCCI Monitoring Augusto Ciuoletti Also in the document Conformance proles: to accomodate the presence of providers that do not implement a monitoring interface Security issues
76. 76. OCCI Monitoring Augusto Ciuoletti Also in the document Conformance proles: to accomodate the presence of providers that do not implement a monitoring interface Security issues A detailed example using the http rendering
77. 77. OCCI Monitoring Augusto Ciuoletti Also in the document Conformance proles: to accomodate the presence of providers that do not implement a monitoring interface Security issues A detailed example using the http rendering ...and several bugs.
78. 78. OCCI Monitoring Augusto Ciuoletti Also in the document Conformance proles: to accomodate the presence of providers that do not implement a monitoring interface Security issues A detailed example using the http rendering ...and several bugs. Thats all...