susan windsor - critical thinking for testers - eurostar 2010

Critical Thinking for Testers

© Copyright 2010 Aqastra 1

Agenda

• Why Critical Thinking?

• Ways of Using Language

• Dishonest Tricks in Argument

• Debate


Why Critical Thinking?

test

We can be deceived by most of

what we deal with


People

• Users

• Stakeholders

• Project Managers

• Developers

• Ourselves

Artefacts

• Project Plans

• Requirements

• Designs

Infrastructure

• Environment

• Test Tools

• Test Data

Software

• Under Test

• Not Under Test

How we can be deceived - 1

• By People– They may be uninformed and/or may not be able to

express themselves well

– We cannot understand them easily

– They have agendas, prejudices and vested interests

– They push for unauthorised requirements/changes

– Our own prejudices and shortcomings get in the way

• By artefacts (requirements, designs, project plans)– Gaps in content

– Ambiguities, contradictions, defects

– Unsupported assertions

– Unjustified, unrealistic, unverified requirements.


How we can be deceived - 2

• Infrastructure

– Environments may not be sound, stable, dedicated

– Tools may not work or not work the way we expect and mislead us

– Test data may not be appropriate, complete, controlled

• Software

– Software under test may reveal its secrets slowly

– Software not under test may hide or influence behaviour

– Software does not obey the laws of physics (or any reliable law for that matter)


Critical thinking is…

• About having an open mind

• Recognising flaws in thinking and writing

• Challenging the arguments of others

• An enabler:

– to ensure we have good reasons to believe or do

what other people attempt to persuade us to

believe or do

• Etc.


Critical thinking as a process

• Identifying other people’s positions, arguments and conclusions

• Evaluating the evidence for alternative points of view

• Weighing up opposing arguments and evidence fairly

• Being able to read between the lines, seeing behind surfaces, and identifying false or unfair assumptions

Recognising techniques used to make certain positions more appealing to others, such as false logic and persuasive devices

• Reflecting on issues in a structured way, bringing logic and insight to bear

• Drawing conclusions about whether arguments are valid and justifiable, based on good evidence and sensible assumptions

• Presenting a point of view in a structured, clear, well-reasoned way that convinces others.


We need to be critical of…

• Testers are responsible for collecting, analysing and disseminating intelligence on software artefacts

• We don‟t make decisions, but others use that intelligence to do so

• We are very reliant on people, artefacts, infrastructure and software to do our job and our „genes‟ tell us to be distrustful

• So we need to be critical of:– The fallibility of our sources of information and guidance

– Our processes, approaches and tools

– Our project colleagues, our stakeholders and their leadership.


What is „critical‟

• “Involving skilful judgment as to truth, merit;”

• “Characterized by careful, exact evaluation and judgment”

• “Characterized by thoroughness and a reference to principles, as becomes a critic”

• Of course, critical has the other, more popular meaning relating to defect detection, importance etc.


References

Straight and Crooked Thinking, Thouless, Penguin

Critical Reasoning, Anne Thompson, Routledge

Critical Thinking, Bowell and Kemp, Routledge

Critical Thinking Skills, Cottrell, Palgrave

• A Rulebook for Arguments, Weston, Hackett

• Lines of Thought, Young, Oxford

• Critical Thinking, Paul and Elder, Pearson Prentice Hall

• Tools of Critical Thinking, Levy, Waveland Press


Ways of Using Language

Many ways of using language -1

• Some examples:1. To give information about a fact – “The system has

been delivered”

2. To ask for information about a fact – “was the system delivered on time?”

3. To indicate an emotional attitude – “the system is useless”

4. To indicate how a word is used – “programmers are people who write programs”

5. To ask how a word is used – “what is a „test‟?”

6. To get someone to carry out an action- “please fix this bug”

© Copyright 2010 Aqastra13

Many ways of using language - 2

• But there are many more ways – for example:

7. To make poetry

8. To make a joke

9. To make a quarrel

10. To resolve a quarrel

11. To give a greeting

• But let‟s look more closely at types 1 and 3 on the previous slide.


Factual and emotional use of words

• Example:

– “My dog is of mixed-breed”

– “Your dog is a mongrel”

• Factually, the two statements say the same thing, but the second infers a less satisfactory situation; disapproval

• This is a trivial example, but compare this with words used to describe mixed race – most are unacceptable to modern society.


Emotional words are everywhere

• Define the adjective „firm‟:– “I am firm”

– “You are obstinate”

– “He is pig-headed”

• Essentially the words mean the same, but they convey an increasing level of disapproval

• Countless words exist that reflect this notion of popularity, approval or „goodness‟

• In political contexts, different words are used, depending on whose side you are on– Freedom-fighter – terrorist

– Spirit (our army) – mentality (theirs)

– Heroism (our army) – foolhardiness (the enemy)


And in a business context?

• From a hardware company website 1/6/08

– “Today's communications consumers want

personalized and entertainment-related

services.… they have an insatiable need to

communicate…”

• But I don‟t want spam, crappy music, junk mail,

irritating web advertising, cold calls, intrusive

salesmen thank-you very much.


And in a testing context?

• For Example, who is the largest provider of testing

services in the UK?

– Mission Assurance and Testing (Mission) is the UK's largest

systems quality assurance and testing solutions provider.

– SQS-UK is the UK's leading independent provider of

system and software testing and quality improvement

services

– Cognizant is a leading provider of information technology,

consulting and business process outsourcing services.

© Copyright 2010 Aqastra

Political oratory (rhetoric)

• Political debate/oratory is full of loaded terms:

– Progress, Liberty, Democracy, Fascist, Reactionary, Racist, Liberal are all loaded terms

– So how can we hope to understand and decide?

• Political oratory has its place

– E.g. Winston Churchill‟s many war speeches

– Where we are to make an important decision (and isn‟t that what democracy is?) it has no place, has it?

• But it is so pervasive, we had better be able to recognise it!


Rhetoric

• Used to mean:– the study of the effective use of language

– the art of influencing the thought and conduct of an audience

• Common usage now is it refers to:– High-flown style

– Excessive use of verbal ornamentation

– Loud and confused and empty talk

• Rhetoric is deprecated by commentators, but it‟s persuasive power is irresistible to politicians, newspapers, salesmen, CEOs etc.


From The People 1/6/08

CASH FOR BAD PARENTS:

Lousy parents could soon be

BRIBED to make sure their kids

go to schoolhttp://www.people.co.uk/news/tm_headline=cash-for-bad-

parents%26method=full%26objectid=20590312%26siteid=93463-name_page.html


http://www.people.co.uk/news/tm_headline=cash-for-bad-parents&method=full&objectid=20590312&siteid=93463-name_page.html

http://www.people.co.uk/

From the editorial - same paper

It’s Action – not ASBOsTen years ago anti-social behaviour orders were hailed as a panacea for all the yobbery, violence and teenage terror in our communities. A decade on they have had little impact. Youth crime is now a plague infecting every corner of Britain -fuelled by binge-drinking and the apathy of irresponsible parents.

Teenagers stabbed by teenagers, dads stamped to death by boozed-up gangs, pensioners scared witless by dope-smoking hoodies. Decent, law-abiding families are crying out for tough new measures to control feral youths who now see Asbos as a badge of honour.

So they should welcome the Action Plan being launched by the Government tomorrow.

At last PARENTS will be punished if they fail to control their yob kids - they will be fined or made to clean up the neighbourhoods terrorised by their offspring. But the failure of the Asbo has proved that in the poorest communities with the most deep-rooted social breakdown it is futile to use a stick without a carrot.

And it seems the Government has got the message.

Work and Pensions Minister James Purnell has been to America to study a project in New York's impoverished ghettos. The radical scheme PAYS parents to take better care of their children. There are cash incentives to ensure youngsters go to school, get regular health checks, even use the library.

Introducing such a scheme here is bound to outrage millions of hard-up parents who are already raising bright, healthy and respectful children.

Critics will call it bribery - rewarding feckless mothers and fathers with taxpayers' cash.

But think of parents like Margaret and Barry Mizzen, Sally and Colin Knox and John and Sylvia Lancaster who have lost their children at the hands of the teen thugs of Broken Britain.

They are the reason we MUST consider all options.


Exercise: Spot the emotional language

Exercise

Emotive Terms

•

•

•

•

•

•

•

•

•

Non-Emotive Terms

•

•

•

•

•

•

•

•

•


Dishonest Tricks in Argument

Sophistry

• From the Greek “sophist” meaning sage, wise

• Sophists:– “A group of professional fifth-century B.C. Greek

philosophers and teachers later characterized by Plato as superficial manipulators of rhetoric and dialectic”

– “an impostor in argument; a captious or fallacious reasoner” (captious=carping, nitpicking, niggling, picky, testy)

• Sophistry: subtle, tricky, superficially plausible, but generally fallacious method of reasoning

• Sophisticated is not necessarily a good thing.


„All‟ and „Some‟

• Arguments such as „All Xs are Y‟ e.g.

– All pacifists are cowards

– All tobacco/oil/arms companies are evil

– All developers are poor testers

– All regression testing is better automated

– All bugs are bad

• To overthrow these assertions, we need only one contradictory example

• Some are easier than others – it depends on the context.


Using „all Xs are Y‟ as a trap

• People use the argument to make a „stronger point‟– Counter is to point out exceptions

– BUT… maintain a moderate position

• Aggressive lawyers might trick a witness into saying „all Xs are Y‟ – which they then attack– Lawyer repeatedly makes extreme assertions (“you

lied, cheated, misled,…”)

– You say, “I always told the truth”

– Lawyer reveals you lied about your age on myspace.com – undermining your credibility.


The “exception proves the rule”

• If you challenge an „all Xs are Y‟ assertion with exceptions the response might be “the exception proves the rule”

• Just point out that the exceptions do the opposite of proving a rule to be true

• Consider that prove really means test or challenge

• Respond that exceptions still confirm that a „rule‟ is false.


Diversionary tactics

• Where an assertion of yours is challenged by an irrelevant assertion e.g.

• Example:

– “X is a thief” / “but he gives money to charity”

– “the software doesn‟t work” / “but we have a deadline to meet”

– “I need X weeks to test” / “but the plan says you finish in Y”

– “You said the system could be tested in X weeks” / “but you didn‟t give me the people”


More diversionary tactics

• Personal attacks, slurs, aspersions– “he always underestimates/over-estimates/lies/

pretends/sucks up to bosses/”

– “he never…”, “he would say that, wouldn‟t he?”

• Refer to something your opponent is unaware of– “according to general systems theory, you are

correct”

– “Thank-you” (thinks: what is GST?)

– “BUT: more recent research has proven you completely WRONG”

– “Er….”


Nitpicking as diversion

• Picking on minor points, asserting victory in an argument is another diversionary tactic– If your opponent says “that‟s 1% wrong” repeatedly, the memory

of the discussion may be that “it‟s wrong”

• “You can‟t be certain” could overrule your common-sense argument– “But you can‟t be certain the system will fail in production”

– “In fact, there are 379 bugs in the system, not 377, (you are WRONG) so the system should go live”

• In fact, these are actually irrelevant assertions.


The “Lesser of two evils” Argument

• Sometimes a course of action is advocated as the „lesser of two evils‟

• But the two evils may not be related

– “we can‟t abolish war because more people die on the roads than in wars”

– “we should stop testing because complete testing is impossible”

– “we shouldn‟t worry about releasing now because the software industry has a bad reputation anyway”


Lesser of two evils - 2

• A classic defence of people accused of parking

illegally or speeding”

– “The police shouldn‟t waste time spying on and

arresting good citizens. They should be arresting

serious criminals”

• Of course the counter-argument is:

– If Y is more evil than X then this is no reason for

not trying to eliminate X, but it is a reason for

fighting Y more energetically.


The mean between two extremes

• Not obviously a trick, but watch out for the language used to state a position

• The argument is that a position is the „middle ground‟ between extremes – so must be more attractive than either

• But everything can be represented as a middle ground – it depends on what that ground is

• The political party that wins the middle ground, wins the election

• Perhaps their definition of ground won, really


Hidden agendas and laziness

• We‟ll change our process (its impossible to

change our people or culture)

• We‟ll train our people (we can‟t ask

management to change)

• We‟ll buy tools (we can‟t ask our developers

or testers to work any harder)

• We‟ll outsource (hiring more people is

expensive, and getting smarter people is too).


Debate

If a talented orator were to sell the idea that

2+2=5 how might they do it?

• 2+2=4– A backward, superstitious ideology! A corrupt idea like

flat-earth, slave-labour, communism, witch-burning and is condemned by all. Do you want to live in the dark age?

• 2+2=6– A dangerous view promoted by self-serving, ambitious and

dishonest bourgeois politniks and anarchists – Are you prepared to risk civilisation and democracy?

• 2+2=5– The one, true way. Five is God‟s chosen number and will

free mankind to achieve universal freedom and peace, and unfetter our ambition to understand our universe.


If a talented salesman were to sell you

consultancy how might they do it?

• Structured Testing– A backward, superstitious ideology! A corrupt idea like

flat-earth, slave-labour, communism, witch-burning and is condemned by all. Do you want to live in the dark age?

• Exploratory Testing– A dangerous view promoted by self-serving, ambitious and

dishonest bourgeois politniks and anarchists – Are you prepared to risk civilisation and democracy?

• “The five-point method – or “Best Practice!”– The one, true way. Five is God‟s chosen number and will

free mankind to achieve universal freedom and peace, and unfetter our ambition to understand our universe.


Debate

Invalid Statement

•

•

•

•

•

•

•

•

•

Trick being used

•

•

•

•

•

•

•

•

•


Some assertions to debate

• Some examples…

1. “Testing must be planned, documented and scripted and systematically executed with disciplined record-keeping”

2. “Test design must use formal test design techniques to be effective and efficient”

3. “Testers need perfect requirements to be effective”

4. “Testers should make the acceptance decision”

5. “Testers have no role in the acceptance decision”

6. “Automation should be the goal of all testers”

7. …

• Care to add your own?


Close

Any Questions?

susan windsor - critical thinking for testers - eurostar 2010

Technology