testing – microsoft office david hansen twc test manager
TRANSCRIPT
![Page 1: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/1.jpg)
Testing – Microsoft OfficeDavid Hansen TWC Test Manager
![Page 2: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/2.jpg)
13 years (+2 internships) at Microsoft Started in Outlook (email, attachments, setup, automation) Now shared Office TWC team (security, performance,
reliability, feedback) Education
BS in Mechanical Engineering from BYU in 1992 MBA from University of Washington in 2006
Personal Grew up in Reno, NV. Married for 15 years to Wendy 3 children (Hayley 13, Dillon 10, Mali 8)
Contact info: [email protected]
![Page 3: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/3.jpg)
Testing’s role in shipping commercial software The kinds of challenges testers face What makes a great tester
Not (intentionally) Recruiting for Microsoft
![Page 4: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/4.jpg)
![Page 5: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/5.jpg)
![Page 6: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/6.jpg)
Typical Office/Microsoft PM/Dev/Test team ratio is 0.5 : 1 : 1
More precise definitions at http://www.microsoft.com/college
![Page 7: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/7.jpg)
![Page 8: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/8.jpg)
Micro-version of product lifecycle
Designed for quick spin of information between triad
Goal is ‘dogfoodable’ feature quality before checking into ‘main’
PlanningStart Feature Crew
Deliver Feature Specification
Deliver Dev Design Doc
Deliver Test Design Spec
Deliver Detailed Feature ScheduleSpec Inspection
Implementation
Deliver Feature Code
Deliver Test Automation
Check-in Test
VerificationPerform Code review
Run check-in tests
Test code on a private release build
Check code into main branchFeature Crew SignoffBuild Verification Test
*Feature Development
![Page 9: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/9.jpg)
Office TWC Challenges
![Page 10: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/10.jpg)
Data has significant impact on performance results
Scale and performance testing about pushing limits
How do you get all that data in there?
![Page 11: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/11.jpg)
Automation? Disadvantages
Homogenous data sets. Lists all have similar fields Stress and functionality test cramming data
Memory leak could bust you Change in functionality/paths break test
Generating lots of data takes a long time. Time is your most valuable resource as a tester.
![Page 12: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/12.jpg)
Attach existing database? Disadvantages
Toughest problem in developing a server product: upgrade
Product changes constantly break solution End up writing lots of build x to build y migration code
instead of new features
![Page 13: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/13.jpg)
Tester owns the solution Drive any product changes with Dev/PM Author tools as necessary Validate process Generate performance results that are consistent,
reliable, and accurate Then you can start entering bugs…
![Page 14: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/14.jpg)
Office TWC Challenges
![Page 15: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/15.jpg)
Targeted attacks are more popular (going after assets instead of fame)
Office installed on a vast number of PCs (especially government/business)
Fuzzing (http://en.wikipedia.org/wiki/Fuzzing) has risen as popular tool for finding buffer overruns
![Page 16: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/16.jpg)
Smart vs. Dumb fuzzing Competing against lots of motivated people with
more time. How do you scale? Distribute the problem Automate to deal with data explosion
Move the battlefront Soldiers inside the walls Sanitize (Wringer or MOICE) Source analysis (catch it in the code)
![Page 17: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/17.jpg)
Aware and constantly learning about changing attack vectors
Build infrastructure and tools to allow scaling out Validate data coming from the tools Provide ideas on moving the battlefront Validate the features
Try to break out of the box Verify box doesn’t break workflow, functionality,
performance, etc…
![Page 18: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/18.jpg)
Office TWC Challenges
![Page 19: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/19.jpg)
![Page 20: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/20.jpg)
Feature only works with Office in a bad state How do you reproduce various states Are you giving someone a broken fire extinguisher?
Validity of the data Can developers fix the bug? Minimal impact on user experience. They’re already
mad. Categorize crashes for decision making No Personally identifiable information
![Page 21: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/21.jpg)
Write test plan Automation Validate data for developers and managers Scale of the service
Tools Simulate crash and doc recovery Scale test the service Service health monitoring
Process Set goals for the organization Drive investigation and action on the data Help other testers across office use the data
![Page 22: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/22.jpg)
Fixing bugs creates bugs Beta feedback:
Verbose – Lots of data Squeaky wheel effect with customers. Not
democratic. How can you tell if you’re making it better or
worse?
![Page 23: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/23.jpg)
WinXP SP1example:
![Page 24: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/24.jpg)
![Page 25: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/25.jpg)
SharePoint survey and update feature Offline store to local store migration Null in filename Office Min/Max functionality Assembly code for exploit
![Page 26: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/26.jpg)
Passion for software and technology Curious about how things work Raw smarts and quick learners CS background and coding skill Test mentality (malicious mindset) Communication & collaboration skills Deal with ambiguity & drive for results Customer focus Detail oriented and persistence Broad and deep technical knowledge
![Page 27: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/27.jpg)
![Page 28: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/28.jpg)
Passion for software and technology Curious about how things work Raw smarts and quick learners CS background and coding skill Test mentality (malicious mindset) Communication & collaboration skills Deal with ambiguity & drive for results Customer focus Detail oriented and persistence Broad and deep technical knowledge
![Page 29: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/29.jpg)
A genuine passion for the work we do.From the day I started in Testing I loved it. I didn’t just like
it, I loved it. I love finding bugs. I love learning about how things are implemented and using that knowledge to break the software. I love coding tools that help make it easier to find bugs faster and more efficiently (wEatmem, WinWhere).
I knew only Pascal, Fortran and a little Basic when I came
to MS. I immediately set out teaching myself to program in C and write Windows apps after that. I loved learning to code and I applied what I learned to writing simple games and tools (Pegged, Dr. BlackJack, WinWhere, wEatmem, etc). I loved all of this. I had an innate genuine passion for testing and coding and applying it to making software better.
![Page 30: Testing – Microsoft Office David Hansen TWC Test Manager](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649ebc5503460f94bc4d52/html5/thumbnails/30.jpg)