state of the art of visualization in apm toolsdiagnoseit.github.io/files/dagstuhl2018-vssp.pdf ·...
TRANSCRIPT
![Page 1: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/1.jpg)
Institute of Software Technology
Reliable Software Systems
State of the Art of Visualization in APM Tools
André van HoornDušan Okanović
Tutorial @ Visualizing Systems and Software Performance (VSSP)
July 10, 2018. Dagstuhl, Germany
![Page 2: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/2.jpg)
Institute of Software Technology
Reliable Software Systems
State of the Art of Visualization in APM Tools
Part 1/2: Introduction to APM
André van HoornDušan Okanović
Tutorial @ Visualizing Systems and Software Performance (VSSP)
July 10, 2018. Dagstuhl, Germany
![Page 3: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/3.jpg)
4
![Page 4: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/4.jpg)
5
![Page 5: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/5.jpg)
6
![Page 6: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/6.jpg)
Tutorial: State of the Art of Visualization in APM Tools 8
![Page 7: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/7.jpg)
„
“Application performance management (APM), as a core IT operations discipline, aims to achieve an adequate level of performance during operations. To achieve this,
APM comprises methods, techniques, and tools for
• continuously monitoring the state of an application system and its usage, as well as for
• detecting, diagnosing, and resolving performance-related problems using the monitored data.”
C. Heger, A. van Hoorn, D. Okanović, M. Mann:
Application performance management: State of the art and challenges for the future.
In: Proc. 8th ACM/SPEC ICPE, ACM (2017)9
![Page 8: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/8.jpg)
André van Hoorn und Stefan Siegl.
Application Performance Management (APM). Continuous Monitoring of Application Performance. (in German)
Order for free: http://www.sigs-datacom.de/wissen/fachposter.html 10
![Page 9: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/9.jpg)
11
![Page 10: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/10.jpg)
• Agents collect data from all system levels
• On application level the agents are often technology-dependent
Collecting Data from All System Levels1.
Where? What? How?
12
![Page 11: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/11.jpg)
Trace-based Metrics (Selection)
What?
Metric
Response Time
CPU Time
Method Name
Return Type
Logging Level
SQL Statement
Error Message
…
Okanović, D., van Hoorn, A., Heger, C., Wert, A., Siegl, S.:
Towards performance tooling interoperability: An open format for representing execution traces.
In: Proc. EPEW ’16. LNCS, Springer (2016) 13
![Page 12: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/12.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Monitoring (Measurement-based Performance Evaluation)
Database
Server
Client
Browser
Request
Response
Query
Result
Application
Server
Data
Recorder
0.2s 0.3s 0.1s 3.0s
0.1s0.4s
0.5s
0.3s
3.0s
4.9s
VisualizationData Analysis
(fictio
naltim
ings)
14
![Page 13: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/13.jpg)
• Data is collected from the system…
• represented as time series…
Tutorial: State of the Art of Visualization in APM Tools
Reconstructing Information from Data2.
15
![Page 14: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/14.jpg)
• Data is collected from the system…
• represented as time series…
• … and as detailed execution traces, and
used to support problem analysis
Reconstructing Information from Data2.
16
![Page 15: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/15.jpg)
• High quantity of information has to be pre-processed
• It has proven useful to use different views to show the data
• Views are navigable and can be categorized by both scope and detail level
Visualization Through Navigable Views3.
17
![Page 16: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/16.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Example: Application Topology Discovery and Visualization
© AppDynamics
18
![Page 17: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/17.jpg)
Manual or automated conclusions and actions can be derived from the
information, e.g.,
• Problem detection and alerting
• E.g., increased response times and resource utilization
• Detection, for instance, based on thresholds and baselines
• Problem diagnosis and root cause isolation
• E.g., N+1 problem, too many remote calls, poor DB queries
• Detection based on monitoring information
• System refactoring and adaptation
• E.g., auto-scaling in cloud-based architectures
Tutorial: State of the Art of Visualization in APM Tools
Interpreting and Using the Information4.
19
![Page 18: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/18.jpg)
Institute of Software Technology
Reliable Software Systems
State of the Art of Visualization in APM Tools
Part 2/2: Examples Visualizations
in APM Tools
André van HoornDušan Okanović
Tutorial @ Visualizing Systems and Software Performance (VSSP)
July 10, 2018. Dagstuhl, Germany
![Page 19: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/19.jpg)
Commercial APM Tools
![Page 20: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/20.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Magic Quadrant
22
![Page 21: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/21.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Application Overview – Dynatrace
23
![Page 22: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/22.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Application Overview – AppDynamics
24
![Page 23: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/23.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Application Overview – Instana
25
![Page 24: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/24.jpg)
Tutorial: State of the Art of Visualization in APM Tools
End User Monitoring – AppDynamics
26
![Page 25: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/25.jpg)
End User Monitoring– Instana
Tutorial: State of the Art of Visualization in APM Tools 27
![Page 26: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/26.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Database Monitoring – AppDynamics
28
![Page 27: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/27.jpg)
Server Monitoring – AppDynamics
Tutorial: State of the Art of Visualization in APM Tools
![Page 28: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/28.jpg)
Server Monitoring – Instana
Tutorial: State of the Art of Visualization in APM Tools
![Page 29: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/29.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Problem Identification – Dynatrace
31
![Page 30: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/30.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Problem Identification - Instana
32
![Page 31: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/31.jpg)
Open Source APM tools
![Page 32: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/32.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Open Source APM tools
34
Performance
Modeling
Web Performance
AnalysisSystem &
Resources Monitoring
Real User Monitoring
JRatJMemProf
Low-Level
Performance Profiling
Monitoring &
Application Deep Dive
![Page 33: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/33.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Dashboards – Grafana
35
![Page 34: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/34.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Dashboards – Grafana
36
![Page 35: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/35.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Dashboards – Kibana
37
![Page 36: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/36.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Dashboards – Kibana
38
![Page 37: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/37.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Execution Traces – inspectIT
39
![Page 38: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/38.jpg)
Tutorial: State of the Art of Visualization in APM Tools
Tracing – Zipkin
40
![Page 39: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/39.jpg)
Build Your Own Landscape
![Page 40: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/40.jpg)
• http://kieker-monitoring.net
Tutorial: State of the Art of Visualization in APM Tools
Dynamic Software Analysis and ApplicationPerformance Management
42
![Page 41: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/41.jpg)
43
Distributed Monitoring
(Java EE/SOAP)
Legacy System Analysis
(COBOL)
Legacy System Analysis
(Visual Basic 6)
3D Visualization
of Concurrency
Application (and Visualization) Examples
Tutorial: State of the Art of Visualization in APM Tools
![Page 42: State of the Art of Visualization in APM Toolsdiagnoseit.github.io/files/Dagstuhl2018-VSSP.pdf · achieve an adequate level of performance during operations. To achieve this, APM](https://reader035.vdocuments.net/reader035/viewer/2022070903/5f61bfaadf23203b8c250c03/html5/thumbnails/42.jpg)
• Commercial tools have most fancy visulizations
… how useful are they really?
• Open-source tools are flexible and can be adapted as needed
… but require a lot of work to setup
• Thanks to a more widespread adoption of (open-source) APM, it is easier
to get access to rich APM data (e.g., distributed traces) and to integrate
visualization approaches (e.g., via APIs)
… if you know what and how to visualize
Conclusion
Tutorial: State of the Art of Visualization in APM Tools 44