james a. whittaker software architect visual studio team test microsoft [email protected]
TRANSCRIPT
![Page 1: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/1.jpg)
The future of testing
James A. WhittakerSoftware ArchitectVisual Studio Team TestMicrosoft
http://blogs.msdn.com/[email protected]
![Page 2: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/2.jpg)
“Any sufficiently advanced technology is indistinguishable from
magic.”
- Arthur C. Clarke (1917-2008)
![Page 3: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/3.jpg)
In 1953 Francis Crick and James Watson described the structure of
deoxyribonucleic acid (DNA)
![Page 4: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/4.jpg)
From 1990-2003 the Humane Genome Project mapped the entire genetic blueprint of a human being
![Page 5: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/5.jpg)
In 1969 Peter van de Kamp
explained wobbles in
the motion of Barnard’s star
by the presence of a
planet 1.6 times the mass of Jupiter
![Page 6: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/6.jpg)
By 2006 Astronomers confirmed the existence of over 200 exoplanets
As of January 2009 that number has climbed to 334
It wasn’t until 2003 that the first exoplanet was confirmed to orbit the
star Gamma Cephei
That’s one new planet every 2.3 days … how long until we find one that
might support life?
![Page 7: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/7.jpg)
We’ve always considered the autistic to lack the ability to communicate
![Page 8: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/8.jpg)
Until, through the magic of software, a courageous autistic girl corrected
ushttp://www.youtube.com/watch?v=JnylM1hI2jc&eurl=http://well.blogs.nytimes.com/2008/02/28/
the-language-of-autism/?apage=2
![Page 9: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/9.jpg)
We’re going to need the magic of software
Global climate change
![Page 10: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/10.jpg)
We’re going to need the magic of software
Alternative energy
![Page 11: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/11.jpg)
We’re going to need the magic of software
Managing global finance
![Page 12: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/12.jpg)
Software is the one tool we need to save ourselves
But what’s going to save us from software?
![Page 13: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/13.jpg)
• “A second V-22 crash in December 2000 killed four Marines. The accident was pegged to a ruptured hydraulics line and a software failure.” - http://community.seattletimes.nwsource.com/archive/?date=20030226&slug=osprey26
• “Fans of the rock band Pearl Jam got a taste of cyber-gridlock Saturday, when a software failure disabled phone lines set up for ticket requests for shows in Seattle, Toledo, Ohio, and other cities.” - http://community.seattletimes.nwsource.com/archive/?date=19960814&slug=2344161
• “Patriot Missile's Tragic Failure Might Have Been Averted -- Computer Glitch Probably Allowed Scud To Avoid Intercept, Army Says” - http://community.seattletimes.nwsource.com/archive/?date=19910815&slug=1300071
• “Software Failure Halts Big Board Trading for Over an Hour” - http://query.nytimes.com/gst/fullpage.html?res=9C05E0D7173EF93AA35755C0A9679C8B63&scp=1&sq=%22software%20failure%22&st=cse
• “This is what happened in 2003, when a software failure near Toledo, Ohio, combined with sagging power lines and a simple error by a control room worker to cause a blackout in eight Northeastern and Midwestern states and Ontario.” - http://www.nytimes.com/2007/07/08/opinion/nyregionopinions/08CIperrow.html?_r=1&scp=14&sq=%22software%20failure%22&st=cse&oref=slogin
• “Nike last year had to reduce prices and fly some products to customers after a software failure resulted in excess inventory and missed shipments.” - http://query.nytimes.com/gst/fullpage.html?res=9407E2DB133EF93BA15755C0A9649C8B63&scp=24&sq=%22software%20failure%22&st=cse
![Page 14: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/14.jpg)
![Page 15: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/15.jpg)
![Page 16: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/16.jpg)
fee = 8.95fee = 0.00
fee = -100.00
![Page 17: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/17.jpg)
![Page 18: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/18.jpg)
![Page 19: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/19.jpg)
![Page 20: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/20.jpg)
![Page 21: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/21.jpg)
![Page 22: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/22.jpg)
![Page 23: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/23.jpg)
![Page 24: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/24.jpg)
If quality is this low with today’s software, what about tomorrow’s?
![Page 25: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/25.jpg)
The future I see
http://blogs.msdn.com/james_whittaker
![Page 26: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/26.jpg)
Information
• For testers, it’s everywhere– On your desk– On your drive– On your network– In your head– In other people’s heads– …
• We need information to be available, actionable and alive
![Page 27: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/27.jpg)
Information
![Page 28: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/28.jpg)
The future I see
http://blogs.msdn.com/james_whittaker
![Page 29: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/29.jpg)
Virtualization
• Hard problems:– Ain’t got no repro– The Vista lesson
• Solution:– Control the environment– Easier said than done … until now
![Page 30: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/30.jpg)
VM’s everywhere
• Virtualization for user machines– Think Watson … only better– Fool proof debugging– Libraries of virtual test machines
• Virtualization for tester machines– Virtualized test libraries, just add apps– Environment-carrying tests
![Page 31: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/31.jpg)
A New Test Market
• Virtual test machines would have value– Buy them, sell them, rent them
• Testers no longer test, they design– Determining what needs to be tested supersedes how
to test it – Why select among tests when you can have them all?
• The result– Apps released after centuries of accumulated testing
against hundreds of thousands of possible environments
– Most apps don’t require dedicated QA staff
![Page 32: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/32.jpg)
The future I see
http://blogs.msdn.com/james_whittaker
![Page 33: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/33.jpg)
Visualization
![Page 34: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/34.jpg)
Visualization
![Page 35: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/35.jpg)
What does software look like?
![Page 36: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/36.jpg)
• Well, there’s:– Input– Output– Data flow– Control flow– Modules– Dependencies– Environment variables– Files– Interfaces– Bugs
![Page 37: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/37.jpg)
Conclusion
• I’ve painted my vision of the future• A future where– late cycle heroics are cause for shame– testing is thorough, predictable,
repeatable– software just works– quality is an expectation– innovation is an epidemic
![Page 38: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/38.jpg)
Parting thoughts
• 20 years from now–Will the quality of software be taken for
granted? Will users be genuinely surprised when it fails?
–Will researchers look back in wonder that there was ever even a need for dedicated bug finders?
![Page 39: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/39.jpg)
What needs to be done now?
![Page 40: James A. Whittaker Software Architect Visual Studio Team Test Microsoft jw@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022013122/5697bf931a28abf838c8f854/html5/thumbnails/40.jpg)