debugging,troubleshooting & monitoring distributed web & cloud applications at silicon...
DESCRIPTION
In the past, applications where created as monolithic entities running on a single server. If this is the past for you, too, you will have experienced the downside of modern distributed and cloud applications, as debugging, troubleshooting, and monitoring is not easily accomplished with traditional approaches. Within this session, we will explore different possibilities for collecting and analyzing the needed information to solve issues on modern distributed application and discuss the advantages and disadvantages of each approach like debugger, log files, performance counter and third party solutions. The focus of this session will be on Developer and DevOps need, as increased release cycles and third party dependency more and more result in the need for troubleshooting also on production system, rather than in an isolated test environment. This session requires a solid understanding of distributed applications and knowledge of SOA, but most principles also apply to and can be beneficial for more traditional application design approaches. The used code examples are in .NET but the shown principles generally apply to other languages, too, and shown software is often available for a variety of environments.TRANSCRIPT
Debugging, Troubleshooting
Distributed Web & Cloud Applications
Foothill College, Oct. 5th, 2013
Theo Jungeblut• Engineering manager & lead by day
at AppDynamics in San Francisco
• Coder & software craftsman by night
• Architects decoupled solutions tailored to business needs & crafts maintainable code to last
• Worked in healthcare and factory automation, building mission critical applications, framework & platforms
• Degree in Software Engineeringand Network Communications
• Enjoys cycling, running and [email protected]
www.designitright.net
Warning• Contains also proprietary
software
• This areas are pointed out
• Please walk away now,if that is not acceptable for you, it’s okay
Overview• Inquiry about the Status Quo • Some Facts -> Dev Process & Ops Reality• Monolithic Applications• Debugging & Logging• Distributed Applications• Side Effects you need to troubleshot• Performance Counter• Microsoft IntelliTrace• AppDynamics• Summary• Q & A
We need YOU!
How agile are You?
The Impact of Process
* from Wikipedia: http://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg
How many releases do your company
have a year?
Typical Releases a Year*
58%
17%
21%3%
2+ monthsmonthlyweeklydaily
http://www.appdynamics.com/blog/2011/12/14/storm-clouds-in-2012-summary-of-appdynamics-apm-customer-survey/
Development + Operations = DevOps
* from Wikipedia: http://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg
Difference between Dev & Ops
Dev• Getting / being agile: need for frequent updates
with new feature and bug fixes• Achieves stability trough running latest
Ops• Highest goal: stable execution flawless execution,
by knowing the environment and software• Achieves stability through careful rollouts
“The Past/Present”*
WebServer DBUser
* or maybe still your present?
“The Past/Present” - Architecture
Data Access Layer
Business Logic Layer
Presentation Layer
DB
How are you troubleshooting?
Traditional Debugging
Traditional Debugging
Advantages• Visibility for executed code path
Disadvantages• Performance impact• Requires software installation• Often not possible in production• Typical does not find easily distributes issues• Only real time results
Low Impact Debugger & Crash Dump Debugging
http://www.windbg.info
Automatic Crash Dump Analysis
http://blogs.msdn.com/b/debugdiag/archive/2013/10/03/debugdiag-2-0-is-now-rtw.aspx
Low Impact Debugger & Crash Dump Debugging
Advantages• Low overhead• Dump collection can be done in production• Allows for offline analysis of dump files• Can be setup to automatically collect dump files for crashes• Allows very deep analysis and debugging
Disadvantages• Automatic analysis limited• Manual analysis has steep learning curve• Offline review requires to have precise dll version available
The Book
http://www.amazon.com/Advanced-NET-Debugging-Mario-Hewardt/dp/0321578899
Log Files
Advantages• Output adjustable without restart• Accepted in Production
Disadvantages• Manual, time intense creation of log statements• Manual, time intense process of result review• Limited scope & visibility • Potentially high performance impact based on IO
“The Present/Future”- Architecture
Microsoft MSDN: http://msdn.microsoft.com/es-es/architecture/gg189193
“The Present/Future”ASP.NET MVC 3/4
(IIS)User
WAS Tomcat
Azure Web Role
SQL Server MySQL
Active Directory
Azure Worker Role
WAS
3rd Party Web Services
JavaApplication
MongoDB
3rd Party Web Services
HTTP/JSON
LDAP
WCF
WCF HTTPJDBC
ADO.NET
.NET Service Bus
.Net Service Bus
HTTP
ADO.NET
HT
TP
The (Netflix) Distributed Reality *
* http://techblog.netflix.com/2012/06/netflix-operations-part-i-going.html
What has all this to do with Debugging
and Troubleshooting?
Who does really know how their application
is performing?
What impactsApplication
Performance?
Concurrency
http://en.wikipedia.org/wiki/File:Smithposter.jpg
• Deadlocks• Raise Conditions• Unexpected complexity• …
Data Volume
http://en.wikipedia.org/wiki/File:Smithposter.jpg
• constantly increasing• Volume hides details• Handling volume
impacts resources• …
Resources
http://blog.laptopmag.com/amd-cries-foul-but-tests-show-battery-life-inferior-to-intels/battery_morpheus
• are limited• Danger of Starvation• …
Who is Performance Testing
every release?
http://www.slideshare.net/appdynamics/what-can-devops-learn-from-formula-1
Dev + Ops = Success• DevOps has to be a combined approach• Mostly no visibility in real performance
& production state
IntelliTrace™• “Flight Recorder” for your Process• Records configurable details• Allows offline analysis• Microsoft specific Tool in Visual Studio• Java Flight Recorder within the Oracle world
IntelliTrace http://msdn.microsoft.com/en-us/library/hh398365.aspx
Java FlightRecorder: http://gotocon.com/dl/goto-aar-2013/slides/RickardBckman_JavaFlightRecorderNextGeneration
DiagnosticsAndProfiling.pdf
Summary IntelliTrace
Advantages• Allows great visibility into single process/AppPool• Allows to play back test or production issue
Disadvantages• Limited to Single Process / AppPool Visibility• Should be only temporally enable in production• Requires configuration effort for balance between
details & performance
Performance Counter / Perfmon
http://msdn.microsoft.com/en-us/library/aa645516(v=vs.71).aspx
Performance Counter• Collect machine wide and process
specific data• Allows on- and offline analysis• Microsoft specific tool build into
Windows
http://www.codeproject.com/Articles/8590/An-Introduction-To-Performance-Counters
Summary Performance Counter
Advantages• Huge OOTB support in Windows/IIS/etc.• Low overhead• Extensible
Disadvantages• Collecting application specific data requires
source code changes
AppDynamics
• Application Performance Management• Optimized for Production• Monitoring instead of investigating• Providing the “Right-Data”
Let’s take a look
http://www.appdynamics.com/products-editions.php
Summary AppDynamics
Advantages• Works in production on high performance sides• Monitors, Notify, Troubleshoot• Allows instance handling in cloud environments• Free lite version• Collects required application details
Disadvantages• Full version not free• Does not include every application detail
Build your own Opinion (for Free)
• Lite Version free but limited
• 30 days free Trial of Pro & Azure Version
http://www.appdynamics.com/products-free-download.php
Summary
• Understand Dev Impact to Ops Execution• Understand the business impact of
production issues• Monolithic is simpler to Troubleshoot• Distributed is more complex• The right tools can make a Big Difference
Downloads, Feedback & Comments:
Q & A
Graphic by Nathan Sawaya courtesy of brickartist.com
[email protected] www.speakerrate.com/theoj
www. slideshare.net/theojungeblut
Blog, Rating, Slides
http://www.DesignItRight.net
www.speakerrate.com/theoj
www.slideshare.net/theojungeblut
Time to say Thank You!
The OrganizersFoothill College (team)
The volunteers (how about you?)The
Spon
sors
Pict
ures
from
htt
p://
blog
.sili
conv
alle
y-co
deca
mp.
com
… thanks for you attention!
And visit and support the
www.siliconvalley-codecamp.com
Please fill out the feedback, and…
www.speakerrate.com/theoj