how to think smarter about software development
TRANSCRIPT
How to Think Smarter about Software DevelopmentNilanjan BhattacharyaCounterfactuals
What scientific concept would improve our cognitive toolkit?Edge.org is a website where some of the most renowned
intellectuals discuss ideas and express opinions. Every year, the Edge, asks a question which is answered by a large number of leaders in various scientific fields. The question in 2011 was - What scientific concept would improve everybody‘s cognitive toolkit?
There were 166 responses from eminent thinkers.All the responses are available online as well as in a printed book
In this presentation I have listed the concepts which are most relevant to software development.
Presentation formatI have included a link to the concepts, followed by brief commentary on the relevance to software development.You must read each link before my commentary. I recommend you take printouts of the web pages and highlight relevant portions. Some of these articles are heavy reading. If you wish you can read one or more every day. When reading the links, try to put aside any preconceptions you might have about software development.
Wicked ProblemsCreating software can be challenging. Software development is definitely a wicked problem. Developers and testers are good at solving difficult problems. Wicked problems require a different mindset. I like to think that a smart developer has already tried to solve a wicked problem. This makes testing even more wicked.
The Name GameThe best example of the ‘Name Game’ is the word ‘test’ (related to software). There is no one in the world (including grandma) who would admit to not knowing the meaning of the word ‘test’. In reality the word ‘test’ is highly misunderstood (in the context of software). It is important to understand what ‘test’ really means. Another word which is a victim of the ‘Name Game’ is ‘agile’.
Experimentation
ExperimentationAs in real life, when you are trying to determine what works (or may not work), you need to experiment. The agile movement has created practices which make software increments available in a few days. There is no reason not to start experimenting immediately and all the time. We would be better off if we realized that ‘user stories’ (as used in agile) are experiments.Many incorrectly view experiments as synonymous with ‘frivolous’ as opposed to deliberate. Later concepts dispel that idea.
Gedankenexperiment
GedankenexperimentIn the case of software, unlike physics, you can conduct physical experiments (on the software). However, the gedankenexperiment is a great concept to stimulate thinking and can be an alternative or precursor to physical experiments. It‘s another alternative to the heinous word ‘test’.
The Double-blind control experiment
Double-blind control experiments
It‘s worth understanding double-blind control experiments from WikipediaA control experiment is much more specific and rigorous compared to the previous two conceptsMany online companies conduct A/B testing which is a controlled experimentIt’s more powerful to understand software development as a series of experimentsInstead of solely relying on customers you can use customer support, developers, testers or other proxies
Cummulative Error
Cummulative ErrorSoftware development itself is an elaborate game of ‘postman’With agile practices, the importance of verbal communication is emphasized as opposed to ‘messages’ inscribed on paper.Many engineers still struggle with this cognitive fallacy
“The illusion of Platonic information is confounding because it can easily defeat our natural skeptical impulses”
The virtues of negative results
The virtues of negative results
Agile retrospectives result in learning from failureWe can go a step further and intentionally make software fail in order to understand flaws in design
You can show something is definitely dangerous, but not definitely safe
You can show something is definitely
dangerous….Since you cannot prove that the software being developed will cause no harm (to users), you continue to find ways in which it can cause harm (defects).Tester redeemed
Uncertainty
UncertaintyUncertainty is a virtue when dealing with complex systems, as long as it is quantifiedWith complex systems there is limited value in pursuing precision as a goal in itself.It is more valuable to qualify uncertainty.When testing software you can qualify uncertainty by describing what has been done and how you measure coverage
The uselessness of certainty
The uselessness of certainty
We don’t need ‘scientifically proven’We need enough information which will allow us to make decisions and act.Instead of Tayloresque management adopted a personal, direct approach based on valuing people and their skills and personal needsIn traditional testing, quality and management, the word ‘scientific’ still shows up as the preferred approach
The uselessness of certainty
‘Scientific’ implies well documented and detailed, and engineers who follow instructionsEven though we don’t demand following instructions, the bigger point is that there is no value in certaintyEven if the software doesn’t crash when connected to eight servers, you could be proven wrong when the environment changesThe key challenge in software development is to correctly qualify your level of uncertainty.
Science’s methods aren’t just for science
Science’s methods aren’t just for scienceScience is not about hard truths‘Science is a way of thinking, of making better approximations of how things are.’Software testers can use scientific methods such as A/B testingOf course, this isn’t about ensuring that customers see value – it’s more about quantifying the uncertainty about risks to that value
Science’s methods aren’t just for scienceScientific methods don‘t need to include conducting experiments. They could be activities like a ‘journal club’ or applying different ways of thinking such as vertical, horizontal, or learning to take meaningful notes when observing software behavior.
Q.E.D. Moments
Q.E.D. MomentsYou need to experience the divine moment of proving a logical or mathematical problemWhen developing and testing software, i.e., determining value and what might go wrong, you can conduct investigations to validate your hypothesis.