The Effective Tester / Web Testing BasicRob Lambert - Director of Cultivated Management

WHAT MAKES A GREAT TESTER?

A great tester :

• Asks questions (Is curious)• Ships software / value• Communicates clearly• Is assertive• Constantly learns• Leads when needed• Expands their role when needed• Takes extreme ownership of testing

WHAT MAKES A GREAT TESTER?Asks questions

● Testing is the act of asking questions● At any time● Of anything

The better you become at asking deep, searching, critical questions - the better chance you have building the right services and products.

Ownership :

1. At what point in the process are testers involved? 2. What questions could be asked?3. Would the 3 Amigos work?

Resources : 1. http://thesocialtester.co.uk/the-phoenix-checklist/2. https://www.coursera.org/learn/mindware

WHAT MAKES A GREAT TESTER?Ships software / value

● Testing is about providing information for the business to make decisions● Testing does not improve the quality of the product● Decisions made with your information improve quality● All testing must relate to business value. The business pays you. You should be

able to articulate why your testing is adding value. ● The business owns decisions around quality● Your job is to ship software. Not to stop it.

Ownership :

1. What information does the business need? 2. Could you create templates/standards/expected deliverables?

Resources : https://leanpub.com/10behavioursofeffectiveemployees

WHAT MAKES A GREAT TESTER?Communicates clearly

● The information testers provide must be clear, timely and relevant● If you don’t communicate - it’s essentially not done● All communication has a Purpose and an Audience● The further up the project reporting line information goes, the simpler it needs to

be - think about your audience● Communication may happen in the future - imagine an auditor coming in - how will

you articulate what you tested, how, why and when?

Ownership :

1. Are there suites of information that could be consistently created for evidence?2. How best are you using TestRails - is there a consistent way that works?

Resources : https://leanpub.com/10behavioursofeffectiveemployees

WHAT MAKES A GREAT TESTER?Is assertive

● Assertiveness is about communicating your beliefs, values and ideas in a confident and non-aggressive manner.

● Assertiveness does not mean you always have to “win”● Disagree and commit is a useful approach (disagree in an assertive way, but commit even

if the decision goes against you)● Assertiveness can be taught● Testers must “own” testing and be assertive about how testing happens at your company

Ownership :

1. How do you do testing?2. Is there a way to document what we believe? What we stand for? What we want to do?

Resources : https://leanpub.com/10behavioursofeffectiveemployees

WHAT MAKES A GREAT TESTER?Constantly Learns

● Learning is a fundamental skill a tester must have○ Task based learning is the act of “doing” a task and getting better at it○ Knowledge based learning is the act of “studying” a subject and remembering it

● A combination of knowledge and task based learning is usually the most successful● On the job learning is the best form of learning. Doing something whilst pairing with

someone more knowledgeable● Build your own Knowledge Management System - a key to modern learning

Ownership :

1. Is there a learning plan that maps to business objectives/goals?2. Is there a learning group/room/resource/library?

Resources : http://cultivatedmanagement.com/personal-knowledge-management-system/

WHAT MAKES A GREAT TESTER?Leads when needed

● Leaders always emerge● Leadership is not a title● Don’t wait for someone to give you permission to make improvements - seek

forgiveness instead● We are all leaders in some things

Ownership :

1. Does everyone own something?2. What levels of ownership / permission / authority are missing?

Resources : https://leanpub.com/10behavioursofeffectiveemployees

WHAT MAKES A GREAT TESTER?

Expands their role● Expand your role to pick up problems but don’t inflict help :)● Testing can happen at ANY point of the pipeline● Testing is about creating high value products - not just about “testing”

Resources : https://leanpub.com/10behavioursofeffectiveemployees

WHAT MAKES A GREAT TESTER?Takes extreme ownership of testing

● Great testers are the authority on testing and take ownership of the consequences of their work● Non-testers have views, opinions and experience - but testing should be owned by testers● Set your stall out - improve it - take on feedback - but don’t let others tell you what sort of testing

you should do. Work with them and fine tune your approach.● It’s hard to take a stand and say what your company testing is/isn’t if it’s not been communicated

Ownership :

1. Take ownership of testing and everything around it2. Excuses suggest a lack of ownership3. Interference and mis-direction from others suggests a lack of ownership.4. Could you do something to address that?

Resources : https://leanpub.com/10behavioursofeffectiveemployees

AUTOMATION IS NOT THE ONLY FUTURE• The current market is hiring Testers who can code.

• SDETs. Technical Testers. • Markets are always playing catch up.• This solved a problem modern companies don’t have

• The “leading” companies ensure Developers write automated “checks” whilst Testers “test” and provide test assurance expertise to the team.

• Testers diversify and become skilled in many areas• Or they have singular focus and pick a niche• Or they become managers

• Test Automation will NOT replace good testers.• Learning to code is NOT the same as building test automation

Your job is to be

Effective

AND

Liked

Powerability; ability to act or do

• We all have access to relationship power• Expertise without relationship can be short term• Role power should be used sparingly

Testing Versus Checking• Checking is verifying the product/system does what it is

expected to do

• Testing is gaining knowledge about what the system can do, actually does and how it reacts to certain actions (input, output, changes, prodding, poking)

• Testing is the art of asking questions, at any time, of anything. Testing leads to more checks as we model and check our models.

• What is behind the screen? Behind the process? Behind the empty statement?

Definitions / Thoughts• TDD

○ Test Driven Development○ The art of creating tests that fail, then implementing the code to make them pass

• ATDD, EDD, BDD○ Acceptance Test Drive Development, Example Driven Development, Behaviour Driven

Development○ BDD - Cucumber, Specflow etc○ GIVEN, WHEN, THEN○ Can be used without code. ○ Best to use with automation framework

• Conversation starters - frameworks come with it• 3 Amigos works well

ET and Project Work• All ET should map to a business value

• Anything you know the answer to already, can be automated (where technically possible)

• Developers can write automated checks

• ET can be recorded in Test tools and linked to a story

• Session Based Test Management is a suitable approach (SBTM)

• Reports should be provided publicly for all to see

BASIC EXPLORATORY TESTING• EXPLORE, WITH, TO DISCOVER (Elisabeth Hendrickson – Explore. IT!)

• EXPLORE the smart meter• WITH these combination of parameters• TO DISCOVER how it responds

• EXPLORE the customer admin site• WITH multiple administrators• TO DISCOVER any permission or locking issues

• Exploratory Testing is gaining knowledge about areas we’re not certain about. • It’s about discovering unknown consequences of development. • It’s about finding out what we don’t yet know.• It’s TESTING

• ET should form a significant part of your testing approach

Question / Thought / Hunch

• Explore, With, To Discover• What do I need to find out?• What data can I use?• What can I do to discover

more information?

Explore

• Time Bound• Guided• Documented• Evidenced• Disciplined

Assets

• Further ET charters• Bugs• Questions• Insights

Communicate

• Metrics• Tied to Business Value• Simple Language• Timely

How Does SBTM ET Work?

WHAT DOES IT LOOK LIKE?• Keep your sessions a duration of time that works for you.

○ 1.5 hours or 20 minutes can work.○ Experiment.

• Book it in your calendar○ If it’s a priority then don’t let others control your time

• Ensure it maps to a User Story on the Team Project board○ There are some exceptions but they should be rare

• Ensure the session is a priority regarding business value• Turn off all notifications and avoid being interrupted• Say “NO” or “No - not right now” to distractions or interrupts or requests on your time. • Take copious notes (paper, video, screens).

○ You rarely remember as much as you think.• Develop Workplace Efficiency

○ Get good at getting to your testing point quickly.

WHAT DOES IT LOOK LIKE?

• Note down and raise clear bug reports ○ Or speak to your dev and get it fixed with no report.

• Pair with Programmers to build the software AND to test it at the same time. • List all new questions that come to mind as you test, and schedule time to

ask these questions later. ○ Or deviate right there and then. ○ There is no standard right or wrong way.○ Just your way

• Reflect on what you learned. ○ Didn’t learn anything? ○ Work out why.

• How can you improve for next time? ○ What can you do differently?

MEASURING TESTING• Don’t measure testers at an individual level• Metrics and measures should be time series (show trends)• Testers are part of a team – what are the team metrics?

You should be able to answer these questions:

1. How many live cases are reported relating to your work?2. How much failure demand is present in your work?3. What is the cycle time of a story and how do test

contribute to that cycle time?4. What testing did you do for X feature/story?

Ownership :

1. What measures do you currently capture?2. How do you know you’re getting better?

Bug AdvocacyHow to find and describe bugs. And then get them fixed.

What is a bug?

A bug is:

• Something that BUGS someone• Something that reduces the value to the person that matters (user,

customer, purchaser)• Something that increases the value to someone who shouldn’t have

value (hacker)

Where do bugs come from?

Where do bugs come from?• Unclear requirements• Miscommunication• Software complexity• Programming errors• Changing expectations• Egos• Poorly documented code• Lack of testing (or slipped through the gap)• And many more places…

Why can’t we find all bugs?

• We don’t know how many bugs there are in the product….if we did we wouldn’t test…

• We would have to test every • sequence• data combination• possible timing of input• hardware configuration• Interface to any other system• Possible way a user might use the system

Why can’t we find all bugs?

• Cont..• There are too many tests to run

• In a simple menu in an application you may have:• 5 ways to add an item to a queue using the UI• 5 x 5 ways to add items via API• 5 x 5 x 5 ways to add to the queue using third party systems

If my math is correct this is roughly about 100 trillion possible paths to add items to a queue…..that’s a lot.

How would you test all of those?

Why can’t we find all bugs?

• Nope.• Even an automated test suite would take years to write and months to

complete a test run• We would never release the product

• So the reality is, we will ALWAYS have bugs.

Can’t we automate all that?

What happens when we find a bug?

How to gather information for a good bug report

The Ten Things To Do

1. Reproduce it in the fewest steps possible2. Reproduce it a number of times to give you confidence3. Vary the data, sequence or timings etc4. Try extreme values, then nothing extreme, then something in between5. Gather relevant log information6. Gather information about the system under test7. Gather information about the requests and responses8. Keep copious notes on how to reproduce9. Describe, to yourself, the impact on the customer10. Ask critical questions about the bug and the language you use to describe it:

○ Does it “always” happen?○ Is it really “broken” or “stuck” or “unusable”.

Writing good bug reports – ten things

1. Include a simple, but descriptive, title2. Include the fewest steps to reproduce3. Include all supporting environmental information4. Include all the different ways you were able to reproduce it5. Clearly articulate the impact on the customer6. Pay attention to the audience of the bug report (next slide)7. Write the report keeping in mind the 60 Days Proof Concept (next slide)8. Include a video, screenshot or any other supporting media9. Describe where the bug was found (live, pre-production etc)10. Include the version of the software, hardware, firmware etc

60 Days ProofWrite all notes, emails, bugs and reports so that they would make sense in 60 days from today.

● When the context of your work is not fresh in your mind - will it make sense to you still?

● Do you need to include any relevant information?● Do you need to add contextual information (location, date, time, who you

were with etc)

Purpose, Audience, ContextAll communication has a Purpose, Audience and Context

● What are you trying to achieve with this communication?● Who are you trying to communicate to?● What context is this communication happening in?● The more of each you have, the harder it is to get your communication right

• Don’t fall into the trap of using the bug tracking tool as a means of communication

• The bug tracking tool is a record of the issue and the work done to complete it

• Face to face discussions with Dev, Product, Test and Support are preferred.

Bug tracking tools are not communication tools