bright talk if they cant use it, it doesnt work - final

http://www.evolveimages.com/media/b65376d8-3b6a-11e1-a064-8546d2b4117d-aerial-view-of-complex-highway-interchange!

User Experience and the Cloud: If the Users Can’t Use It Then It Doesn’t Work!

!Andrew has fifteen years of experience designing and managing the deployment of systems monitoring and Event Management software. Andrew has held various roles including leading the monitoring organization of a Fortune 100 company and developing solutions as a consultant for a wide variety of organizations, including the Mexican Secretaría de Hacienda y Crédito Público, Telmex, Wal-Mart of Mexico, JP Morgan Chase, Nationwide Insurance and the US Navy Facilities and Engineering Command.!

!Andrew White!User Experience Fanatic!

http://weheartit.com/entry/12433848!

Follow Us: #ITSMSummit!

GROUND RULES FOR THIS SESSION…!

1.  If you can’t tell if I am trying to be funny…!!GO AHEAD AND LAUGH!!

2.  Feel free to text, tweet, yammer, or whatever. People gotta hear this! Use !

3.  If you have a question, no need to wait until the end. Just interrupt me. Seriously… I don’t mind.!

I have a lot of experience leading !Systems and Event Management teams !

My name is Andrew White!

Latency!I am here today to share some of what I have learned about!

User Experience!

And more importantly, I am here today to talk about!

What do I mean by latency and user experience?!

Definitions:!


LA�TEN�CY – [LEYT-N-SEE]!-noun, plural -cies!1. The state of being latent!2. The time that elapses between a stimulus

and the response to it!3. The state of being not yet evident or active!

http://www.flickr.com/photos/25822731@N02/4644128723/!


EX�PE�RI�ENCE – [IK-SPEER-EE-UH’NS]!-noun!1. The apprehension of an object, thought,

or emotion through the senses or mind!2. Direct personal participation or

observation; actual knowledge or contact!3. A particular incident, feeling, etc., that a

person has undergone!-verb!4. To be emotionally or aesthetically moved

by; to feel!5. To learn by perceiving, understanding, or

remembering!

http://www.flickr.com/photos/51035626620@N01/170061976/sizes/l/in/photostream/!


When you put them together we get:!!The ultimate measure of success for any system is the perception of its performance. The less interactive a system becomes the more likely its performance will be perceived to be poor.!

Latency is the mother of inactivity!!


THE TWO DIMENSIONS OF LATENCY…!

Internal Latency vs. External Latency!

Actual Latency vs. Perceived Latency!

This is what user experience is all about!!In other words: Perceived = Fn(Internal+External)Variation )!

*Among adults who accessed the internet with a mobile phone in the past 12 months (n=1,001) – Gomez Mobile Web Experience Survey conducted by Equation Research!

58% of mobile phone users expect websites to load as quickly, almost as quickly or faster on their mobile phone, compared to the computer they use at home*!

http://www.flickr.com/photos/lucianbickerton/3858380291/sizes/l/!

*Among adults who accessed the internet with a mobile phone in the past 12 months (n=1,001) – Gomez Mobile Web Experience Survey conducted by Equation Research!

60% of mobile web users have had a problem in the past year when accessing a website on their phone*!

http://www.flickr.com/photos/rickyromero/1357938629/sizes/l/!

*Among adults who accessed the internet with a mobile phone in the past 12 months (n=602) – Gomez Mobile Web Experience Survey conducted by Equation Research!

Slow load time was the number on issue, experience by almost 75% of them*!

http://bighugelabs.com/onblack.php?id=2497744197&size=large!

Is 5 seconds really bad?!

Start…!

Start…!

Observed Maximum:!

90th Percentile:!5.44 seconds…!

15.4 seconds…!

Start…!

Start…!

Observed Maximum:!

90th Percentile:!DONE!!5.44 seconds…!

15.4 seconds…!

Start…!

Start…!

Observed Maximum:!

90th Percentile:!DONE!!

DONE!!

5.44 seconds…!

15.4 seconds…!

If you were the one on the phone with one of those customers…!

how would you fill that silence?!

We need to recognize when we have problems to solve!

Why does any of this matter?!


No complaint… is more common than that of a scarcity of money!-Adam Smith, Wealth of Nations!

QUESTION:!What value does your IT organization create for your business?!

If you can’t answer this question, how can you be sure you are doing the right things and doing them well…!

HINT: “We provide infrastructure or applications the business uses” is not a value statement!


We all want to make sure of one thing…!

The services we offer are our brand!

And the thing about brand is…!

Brand is not what you say…!

Brand is what THEY say!

• Who are we?!• What do we do?!• Why should people care?!

So it is important to know:!

Would anyone miss us!if we were not here?!

And especially…!

BRAND MATTERS!

How does IT preserve the brand value it creates?!

• 100% Uptime*!

• Scalability*!• Performance*!• Agility*!

• Good UX*!!

*To the best of our ability!

How well would THEY say you are doing?!

Maybe.!Let me show you why this is important…!

http://www.flickr.com/photos/anneacaso/3693155059/sizes/l/in/photostream/!

Objective #1: Users Love Our IT Systems…

Constructive Criticism!

If no critique is given, no improvement will happen.!


OUR PROBLEM STATEMENT:!

The business needs to reliably reach its customers and users regardless of where they may be located.

Latency forces close geographic proximity of the components and limits the quality of service provided

to geographically distributed customers. !!

If the users can’t use it, it doesn’t work.!


OUR CONSTRAINTS!At the same time, there are a few inescapable facts we face:!1.  Today’s users demand a reasonably well

performing system to do their work!2.  Our systems mirror the complexity of the

businesses they support!3.  Business continuity requires diversity in our

deployments!4.  The speed of light isn’t changing any time soon!

When all of these happen at the same time…!

Ug…!

http://www.flickr.com/photos/gregphoto/4881356366/sizes/l/in/photostream/!

Bad Experience!!!!

http://www.wealthforteens.com/wp-content/uploads/2011/07/happiness_boy.jpg!

We need to be giving more of this!


BEING USER FOCUSED…!

In addition to monitoring for system availability, we are here to help manage latency.!

The Formula…!1.  Continually map, monitor, and categorize all

sources of latency!2.  Help identify and remove all sources that are

found!

OK.!So now what?!

I am going to talk today about latency in the cloud and some new ideas that will help both them and you.!

Is it the infrastructure or the application?!The perennial problem….!


ANATOMY OF AN OUTAGE!

Corporate!LANs & VPNs!

Load Balancer!

Firewall!

Web!Servers!

Message!Queue!

zOS!CICS!

WAS!

Database!

WAS!Database!

zOS!MQ!

DB2!

IM01109089: P0 - Affecting Multiple apps & Internet Sales West!!!!!

4!

!!!!!!

3!

!!!!!!1!

5:45-ish pm: CICS ABENDS start flooding MainView but not high enough to ticket!

!!!!!!2!

6:00-ish pm: MQ flows start are interrupted and are alerting in Flow Diagnostics!

6:04pm: Synthetic transactions fail at and 6:14 the Ops Center confirms the issue and creates a P0 Incident!

6:54pm: Support teams investigate the interrupted flows and determine it is a “back-end” problem!

10:29pm: Support teams investigate MQ and ultimately and rule it out and ultimately decide to reset CICS to resolve the issue!

!!!!

5!


“How did you go bankrupt?" “Two ways. Gradually, then suddenly.” -Ernest Hemmingway, The Sun Also Rises!

HOW DID WE GET HERE?!


DRIVING THE RIGHT KIND OF ACTION!

Application!

End User Experience!

Gainesville!

Transaction 1!

Transaction 2!

Transaction N!

San Antonio!

Transaction 1!

Transaction 2!

Transaction N!

Des Moines!

Transaction 1!

Transaction 2!

Transaction N!

Columbus!

Transaction 1!

Transaction 2!

Transaction N!

Infrastructure!

Network!

KPI 1!

KPI 2!

KPI N!

Mainframe!

KPI 1!

KPI 2!

KPI N!

Storage!

KPI 1!

KPI 2!

KPI N!

Linux!

KPI 1!

KPI 2!

KPI N!

Middleware!

KPI 1!

KPI 2!

KPI N!

Database!

KPI 1!

KPI 2!

KPI N!


Application!


Gainesville!

Transaction 1!

Transaction 2!

Transaction N!

San Antonio!

Transaction 1!

Transaction 2!

Transaction N!

Des Moines!

Transaction 1!

Transaction 2!

Transaction N!

Columbus!

Transaction 1!

Transaction 2!

Transaction N!

Infrastructure!

Network!

KPI 1!

KPI 2!

KPI N!

Mainframe!

KPI 1!

KPI 2!

KPI N!

Storage!

KPI 1!

KPI 2!

KPI N!

Linux!

KPI 1!

KPI 2!

KPI N!

Middleware!

KPI 1!

KPI 2!

KPI N!

Database!

KPI 1!

KPI 2!

KPI N!



WHAT GOOD MONITORING LOOKS LIKE!


Load Balancer!

Load Balancer!

Firewall!

Switch!

Web Server Farm!

Database!

Data Power!Mainframe!

Middleware!

Load Balancer!

1.  System Availability!2.  Operating System Performance!3.  Hardware Monitoring!4.  Service/Daemon and Process Availability!5.  Error Logs!6.  Application Resource KPIs!7.  End-to-End Transactions!8.  Point of Failure Transactions!9.  Fail-Over Success!10. “Activity Monitors” and “Reverse Hockey

Stick”!

Elements of Good Monitoring!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!3!2! 4! 5! 6!1!!!!!

7!

!!!!!!!!!!!!!!!!!!8!

!!!!!!!!!!!!!!!!!!!!

9! !!!!!!

10!


Application “Think Time”…!

These two are the network!

WE HAVE A PROBLEM HOUSTON!

Here is the real problem…!


COGNITIVE DISSONANCE!


Distributed!

Database!

Mainframe!

Network!

Middleware!

Storage!

ISP!Connection!

DNS & Internet!Services!

Content Mgmt!System!

Social Network!Widgets!

Site Tracking!& Analytics!

Banner Ads & !Revenue Generators!

Multimedia &!CDN Content!

Home Wireless!& Broadband!

Mobile Broadband!

The Part You Control!

The Part They Experience!

…meanwhile the user is NOT happy

All our systems look great, SLA’s are being met…

You Have More Control Here

Than You Think!


SOMETIMES WE MISS WHAT IS GOING ON!

Say… what’s a mountain goat doing all the way up here in a cloud bank?!

Failure comes when you don’t listen. You can’t put something out there and assume it’s great. It’s up to us to make sure we’re listening to improve our chances for success—if not this time, next time !-Alan Lewis, eBay Technical Evangelist!

<TANGENT>!

All You Need!

Love!

</TANGENT>!

http://info.streamdatacenters.com/Portals/165393/Gallery/Album/6624/Richardson%20Aerial-01.png!

This is no longer the way we should think about monitoring!

Monitoring Happens Here!

Cloud Monitoring Happens Here!

Is it the infrastructure or the application?!The perennial problem….!


COGNITIVE DISSONANCE!


Distributed!

Database!

Mainframe!

Network!

Middleware!

Storage!

ISP!Connection!








Mobile Broadband!

The Part You Control!

The Part They Experience!

…meanwhile the user is NOT happy

All our systems look great, SLA’s are being met…

You Have More Control Here

Than You Think!

The Cloud!



Application!


Gainesville!

Transaction 1!

Transaction 2!

Transaction N!

San Antonio!

Transaction 1!

Transaction 2!

Transaction N!

Des Moines!

Transaction 1!

Transaction 2!

Transaction N!

Columbus!

Transaction 1!

Transaction 2!

Transaction N!

Infrastructure!

Network!

KPI 1!

KPI 2!

KPI N!

Mainframe!

KPI 1!

KPI 2!

KPI N!

Storage!

KPI 1!

KPI 2!

KPI N!

Linux!

KPI 1!

KPI 2!

KPI N!

Middleware!

KPI 1!

KPI 2!

KPI N!

Database!

KPI 1!

KPI 2!

KPI N!

The Cloud!


SOME THINGS NEVER CHANGE!


ISP!Connection!








Mobile Broadband!

Is It My Cloud Provider?!•  Configuration errors!•  Application design issues!•  Code defects!•  Insufficient infrastructure!•  Oversubscription Issues!•  Poor routing optimization!•  Low cache hit rate!

Is It a Service Provider Problem?!•  Non-optimized mobile content!•  Bad performance under load!•  Blocking content delivery!•  Incorrect geo-targeted content!

Is it an ISP Problem?!•  Peering problems!•  ISP Outages! Is it My Code or a Browser Problem?!

•  Missing content!•  Poorly performing JavaScript!•  Inconsistent CSS rendering!•  Browser/device incompatibility!•  Page size too big!•  Conflicting HTML tag support!•  Too many objects!•  Content not optimized for device!

The Cloud!

We are going to borrow some old tactics but apply them in a new way!

We aren’t going back to square one!


That is great but we need more…!


WHAT DO YOU WANT TO ACCOMPLISH?!

Your monitoring should help you answer:!

•  How will we know if the users are getting the experience they are expecting?!

•  How much capacity do we need during normal and peak times to ensure user expectations are met?!

•  How quickly can the provider we select ramp up to meet our needs if we find that the service is underperforming?!

•  How fast do we need to be able to access additional capacity once it is ready for us?!


THE CRITICAL PATH OF PERFORMANCE!

Browser Worksta?on OS

Worksta?on Hardware Client LAN Corporate

WAN Datacenter

LAN Etc.

Web Server Web Server OS

Web Server Hardware

Datacenter LAN

Middleware Server

Hardware Middleware Server OS

Middleware Applica?on Etc.

Database Server

Database Server OS

Database Server HBA

SAN Fabric Switch

Array Hardware

Array Controller

Hardware Cache

Disk Drives Etc.

Client Node

Middleware

Database


COMPOSITE APPLICATIONS!

Site Content!Search!

Session!Information!

User Login!& Identity Mgmt!







COMPOSITE APPLICATIONS ARE EVERYWHERE!

•  ATG (Oracle) – Shopping Cart!•  Estara – Click to Chat!•  Twitter Widget – Social Networking!•  Gigya – Social Networking!•  Google Maps API – GeoLocation!•  Facebook Widget – Social Networking!•  Google Analyics – User Tracking!

hJp://www.sgstool.com/images-‐dev/_products/drill_group.png

People don’t want to buy quarter-inch-drill-bits…!They want to buy quarter-inch-holes.!-Ted Levitt, Marketing Professor, Harvard University!


GAINING PERSPECTIVE REQUIRES BALANCE!

Packet Capture!

Synthetic Transactions!

Client Monitoring!

Client Monitoring!

Synthetic Transactions!

Server Probe!

1.  Client to the Server!2.  Server to the Client!3.  “3rd Party” Vantage Point!4.  Synthetic Transactions!

Four Perspectives of User Experience!


WHY MULTIPLE PERSPECTIVES?!

Know Your Customer:!§  What they do? !

§  Customers care about completing tasks NOT whether the homepage is available!

§  Where they do it from?!§  Your customers don’t live in the cloud, test from their

perspective!

§  When they do it?!§  Test at peak and normal traffic levels, to find all the

problems!

§  What expectations do customers have?!§  Is 5 seconds fast enough or does it have to be quicker?!


SEEING IS BELIEVING!

Real User Monitoring Would Report 94ms!Response Time. !

The page seemed “done” to me !

1.2 seconds later!

The time spent rendering represented 93% of the

user experienced latency!


HOW PERCEPTION IS INFLUENCED!

Source: Gomez Real-‐User Monitoring

0

5

10

15

20

25

Second

s

Load Time Perceived Render


Gomez recommends the following test criteria…!Capacity!§  Test vendors to 15-20% past estimated capacity goals!Elasticity!§  Baseline end-user performance before & after testing!§  Test during pre-deployment and in production!§  Ramp elasticity testing to peak levels!Burstability!§  Isolate the cloud elements from other

infrastructure to test!§  Test the “failover process”!

PERFORMANCE TESTING CLOUD CAPABILITIES!


DON’T CONFUSE BANDWIDTH AND SPEED!

737-‐900ER 747-‐400ER

Maximum Number of Passengers 215 524

Maximum Crusing Speed (mph) 511 570

A 737 and a 747 both travel around 500 mph but the 747 carries twice as many people. Would you say it is twice as fast?


NOT ALL CLOUDS PERFORM EQUALLY!Average Response Time of Reference Application, as measured from US backbone locations!

Source: Gomez Real-‐User Monitoring


75th Percentile!

50th Percentile!

25th Percentile!

WHY AVERAGES DON’T WORK!

0.5! 0.7! 0.9! 1.8! 2.5! 2.5! 2.6! 2.9! 3.3! 3.5!

Average!


BEST PRACTICES FOR QUALITY EXPERIENCES!1.  Everyone is responsible for UX!2.  Don’t worry about the infrastructure unless it is in

the critical path!3.  You can control a lot more outside the firewall

than you think!4.  Monitor the user experience from multiple

perspectives and multiple geographies!5.  Test like you monitor!6.  Monitor at a frequency to ensure you can resolve

issues before end-users are impacted!7.  Establish a baseline for historical analysis and

benchmark yourself against the competition!!

Positive Feedback from others is important!


The IT Culture is driven to technology for solutions. Leverage your monitoring and testing tools to help practice failure scenarios. Work on tracking potential points of failure by creating monitoring and report the rate of occurrence to the developers at the start of each new iteration.!

PLAYING TO OUR STRENGTHS!


OH… REMEMBER THE OPS USE CASE!

•  Security!•  Backups!•  High Availability!•  Upgradability!•  Deployment Process!•  Scaling and Elasticity!•  Anticipated Performance Under Load!•  Known Defects!


LET’S KEEP THE CONVERSATION GOING…!

[email protected]!

ReverendDrew!

SystemsManagementZen.Wordpress.com!

systemsmanagementzen.wordpress.com/feed/!

@SystemsMgmtZen!

ReverendDrew!

[email protected]!

614-306-3434!

bright talk if they cant use it, it doesnt work - final

Technology

lot of experience

mobile phone users

perceived latency

user experience fanatic

mobile web users

actual latency

external latency

internal latency