static code analysis and code review - soberit-software business

HELSINKI UNIVERSITY OF TECHNOLOGY

T-76.5613 Software testing and quality assurance

Static code analysis and code review

Mika MäntyläSoberIT

2

Different static methods

Mathematical verificationProve the program correctness with mathematical methods

Code reviewsRead the source code in order to find defects and quality issues

Tool based static analysis“Automated code inspections”

Subjective design and code quality indicators and improvement

3

Introduction – Static versus Systematic testing

Systematic software testingRequires a large set of tests

Each test tend to discover only one or few faults

Cannot be used in early stages

Static methods Does not require execution

Each error can be considered in isolation No problems where you don't know whether anomalies are from new fault or side-effects of existing one

Can consider other quality attributes as maintainability, reusability, security

4

Contents

Static methodsMathematical verification

Code review defects

Tool based analysis


5

Mathematical verification

Research in this area has been going on 40 years This can offer error free programs

Assuming that programs specification is correct

Mathematical formal verification requires Program itself must be formally specifiedSemantics of programming language must be formally defined

The formal semantics of a language is given by a mathematical model to represent the possible computations described by the languageSyntax is always formally definedSemantics of most programming languages are not formally defined

— We must use only a subset of language

6

Mathematical verification cont’d

Hoare’s triplets are example of mathematical verification {P} S {Q}

P is a precondition, S is program statements, Q is a post conditionExample { x = 0 } x := x + 2 { x ≥ 1 },

Mathematical verification is too time consuming and expensive for many programs

Can be used in systems or parts of systems that are safety critical require high reliabilityE.g. telecom systems, space-shuttle software, embedded systems

ISDN-protocol bugs enabled free calls -> discovered with mathematical verification

Automated mathematical verification, i.e. model checkingUsed to verify and debug systems specified with temporal logic

Software needs to be abstracted for the language of the model checker

State explosion problem Algorithms: symbolic algorithms, partial order reduction

7

Design-By-Contract (DBC)

DBC is practical application of Hoare’s tripletsUses contract as program specifications

Each routine has “two contracts”, pre and post conditionsCaller must satisfy preconditionsThe routine must satisfy post conditions

Inheritance and polymorphism is also supported through “subcontracting”

Pre condition may be kept or weakenPost condition may be kept or strengthen

Classes can have contracts. E.g. invariants that must hold for all instances

Program itself never verifies the contract conditionsFail hard

Contract code not used in production

8

Design-By-Contract (DBC) – Example Routine

put (x: ELEMENT; key: STRING) is

-- Insert x so that it will be retrievable through key.

require

count <= capacity

not key.empty

do

... Some insertion algorithm ...

ensure

has (x)

item (key) = x

count = old count + 1

end

9

Design-By-Contract (DBC)

Benefits:Errors are immediately detected and located -> Better error handlingProgrammers must specify their intentions -> Better software design and structureSpecified intentions serve as documentation -> Better documentation

DrawbacksLack programming language support

Eiffel is the only widely used language with built in support for DBCTools exists for many languages C++ and Java

— Wikipedia has good list of tools

Assertions can be used in DBC fashion— Discipline required

10

CleanRoom

CleanRoom is development approach utilizing statistic / mathematical techniques.

Bugs are not allowed to enter the softwareCleanRoom has

Incremental developmentPrograms are seen as mathematical functions

Stepwise refinement (blackbox->state-boxes->clear-boxes (procedural design))

Formal specification of the programsStatistic verification of program correctnessIncrement components are not executed or tested in any wayIntegrated system is not tested to find defects but to ensure reliability with operational profile

Operational profile = Probable pattern of the usage of the softwareAll possible usage patterns definedDefined based on box structure specifications of system function

— plus information on system usage probabilities obtained from prospective users, actual usage of prior versions, etc.

11

CleanRoom Cases

Ericsson OS-32 (350Kloc) operating system project70% improvement in development productivity, 100% improvement in testing productivity, Testing error rate of 1.0 errors per KLOC (all errors found in all testing) Project that contributed the most to the company in 1993

NASA Satellite Control (40Kloc)80% improvement in productivity

780 LOC/person month50% improvement in quality4,5 errors per KLOC

Other ClearRoom project domainsFlight control softwareLan Software Real-time embedded tape drive softwareMedical device software (2004)

12

Summary of Mathematical Methods

Mathematical verification is expensive, but can be used when high reliability is required

Design-by-Contract is a feasible optionTechnique based on pre and post conditions

CleanRoom is development approach utilizing statistical techniques.

13

Contents


Code review defects

Tool based analysis


14

What is a defect?

No universally adopted definition

Three approaches in code review contextanomalies that will cause a system failure

incorrect code, which may (or may not) result in a system failure

all deviations from quality are defects

In this study the last definition is used

15

Code Review - Defect types

0,00 %

10,00 %

20,00 %

30,00 %

40,00 %

50,00 %

60,00 %

70,00 %

80,00 %

90,00 %

Evolvability "True defects" FP

Industry case QIndustry case SStudents

16

What are Evolvability Defects? (1/3)

Not just simple issues dealing with layout

Three types of evolvability defectsVisual Representation

Documentation

Structure

Visual Representation has roughly 10% share of evolvability defects

This number was consistent in the three data sets (IQ 10%, Stu 11%, IS 12%)

Examples of VR: Indentation, Blank line usage, Grouping...

17


Documentation - Communicates the intent of the codeTextual

Code comments

Naming of code elements

Debuggin information

Supported by languageDeclare immutable (keyword: const, final)

Visibility of code elements (keyword: static, private)

Share of documentation fluctuats between studiesIQ 35%, Stu 46%

Textual documentation: IQ 24%, Stu 34%, IS 52%

18


StructureRedundant

Dead code, Duplicate code

Semantic dead code, Semantic duplicate code

Re-organize Move Functionality, Long sub-routine

Alternative ApproachChange function, Magic number, Create new functionality, Use Exceptions

Share of structure fluctuats between studiesIQ 55%, Stu 43%, IS 26%

19

What are functional defects (1/2)?

Data & ResourceVariable initialization

Memory management

Data & Resource manipulation

SharesIQ 27.7%, Stu 2%, IC 15.3%, IK 10%-13%,

ChecksCheck return value

Valid value returned

No error code given

Check memory allocation

Check variableParameters are valid

Checking loop variables

Check pointer

SharesIQ 16,9%, Stu 56%, IC 12.6%, IK 8%-13%,

20

What are functional defects (2/2)?

InterfaceDefect made when interacting with other parts of the software or with code library

Incorrect or missign function call

Incorrect or missignparameter

SharesIQ 16,9%, Stu 2%, IC 12.6%, IK 8%-13%,

LogicComparison

Computation

Wrong location

Off by one

Algorithm / Performance

SharesIQ 14.5%, Stu 37.5%, IC 21.6%, IK 4%-5%,

21

Summary of Code review data

What is a defect

1:3 ratio funtional defects: evolvability defects

Evolvability defectVisual representation (layout)

Documentation (Naming & Commenting)

Structure

Functional defectsData & Resource

Checks

Interface

Logic

22

Contents


Code inspections

Tool based static analysis


23

Summary of Code review defects

60% are evolvability defectsDocumentation

NamingCommenting

Visual representation (layout)Constant 10% share in studies

StructureRefundantRe-organizeAlternative Approach (re-think)

20% are functional defectsData&ResourceChecksInterfaceLogic

24

Contents


Code review defects

Tool based analysis


25

Tool Based Static analysis

“Automated code inspections”Descended from compiler technology

A compiler statically analyses the code and knows a lot about it, e.g. Variable usage; finds syntax faultsStatic analysis tools extend this knowledgeReverse-engineering tools also extend compiler knowledge

But they have different target e.g. Source Code -> UML

Goals:Check violation of code

Documentation Visual Representation

Ensure the system design and structure qualityDetect defects

26

Layout & Visual Representation

Visual representation has great impact to program comprehension

Usage of white space, indentation and other layout issues

Different layout styles: K&R Style, Allman Style, GNU Style

What can static analysis do to enforceLayout / Visual representation

Layout can be completely enforced

Pretty printers can even restore the wanted layout

27

Documenting Code (Naming)

Descriptive naming greatly increase the program comprehensionDifferent naming styles

Sun’s Java Coding ConventionsClass names are nouns Methods are verbs Variable names should reflect the variables intended use

Hungarian naming style (“Microsoft Style”)Identifiers prefixes state their type lprgst -> long pointer to array of zero terminated strings

Figuring good names right away is difficultName should be improved as better one is discoveredRename is most used “refactoring” feature in Eclipse

28

Documenting Code (Commenting)

Studies have shown that too much commenting is harmful

Commenting is often abusedComment are used as deodorant

— Using comments to explain a complex code rather than improving the code

Comments re-stating the method name— Maybe caused by outside pressure to comment all code

Commenting should be used wiselyProvide extra information not obvious from the naming

“Check needed because of bug in Java virtual machine 1.4.1”

“Used by X”

“If parameter is <0 this disables all buttons”

29

Static Analysis – Documenting

What can static analysis do to enforceDocumenting

Naming— Regular expression checks

— E.g int’s are lower case, starting with ‘i’, length< 6 letters

Commenting— Check that classes, variables, and methods have comments and

— Existence of different java doc tags can be checked (@return, @param)

— No tool can say anything about the sanity of the comments

Tool CheckStyle http://checkstyle.sourceforge.net/

Peer pressure and standards are still among the most effective tools

30

Structure

Does the following look acceptable

The method had over 500 lines of code and cyclocmaticcomplexity over 100

Writing for example a unit test or a small description of the methods behavior is impossible

31

Structure

The software structure affects many software quality attributesMaintainability, portability, reliability, and functionality

Reasons for controlling internal software structureThe future development depends on the current structure Agreement on what is acceptable software structure

What is acceptable size for method, how big can class be?

To stop bad programmersAnecdotal evidence suggest that some programmers actually do more harm than good

To prevent side effects of software evolution. Laws of software evolution from 1970’s by Lehman & Belady at IBM with OS/360 state that

— Software which is used in a real-world environment must change or become less and less useful in that environment

— As an evolving program changes, its structure becomes more complex, unless active efforts are made to avoid this phenomenon

32

Case: Construction Quality Measurement at HP

Purpose Create construction quality (maintainability) metric for software developers in the trenches

Measure combined three dimension of maintainability 1) control structure, 2) information structure, 3) typography, naming, commenting

Polynomial metric equationProcedure quality:

Equation was adjusted against developers opinions

Analysis assisted HP in Buy-versus-build decision, Controlling software entropy over several versionsIdentify change prone subcomponentsFinding targets and assessing the efforts of reengineering

))46,2sin(50()ln(2,16

)'(23,0)ln(2,5171

perCMaveLOC

gaveVaveVol

××+×

−×−×−

33

Summary: Static Analysis – Structure

Code metrics are used to control internal software qualityBased on change rate data: Cyclomatic complexity not >14

Code metrics can protect software from the harmful side-effects of software evolutionBy measuring you send a message of it’s importance Problems

You should to define your own limitsHistorical data needed

If you want to use hard evidence to base the measures

Lack qualitative elements“You may replace that entire function with one function call”“Measures don’t really tell whether it is well programmed”

Programs: CCCC, Metrics (Eclipse plugin)To measure duplication: Same and CloneFinder (commercial)

34

Static analysis – Error detection

Static analysis can also detect anomalies in the code unreachable code,

undeclared variables,

parameter type mismatches,

uncalled functions & procedures,

array bound violations

thread synchronization problems,

misuse of variables

35

Example: Errors

void input_reporter(){

int i;

while (i == 0)

{

sleep();

i = doWeHaveInput();

}

weHaveInput(i);

}

Where is the bug?

Is this even a bug?C -> Yes

Java -> No

36

Data flow analysis

Study of program variablesVariable defined where a value is stored into it

Variable used where the stored value is accessed

Variable is undefined before it is defined or when it goes out of scope

X = y + z

IF a>b THEN read(S)

x is defined, y and z are used

a and b are used, S is defined

37

Static analysis – Defect detection

Static analysis can detect anomalies in the code Static analysis can find real and difficult to spot defects

For example Microsoft has but such tools as part of their build process

Whenever warning is found it is treated as a build breakerTheir developers are using such tools in their desktops

Problems of static defect detectorsNoise = “messages people don’t care about” (not just “bogus”messages)

Too much noise — people won’t use the tool— missing all the defects

Need to make sure that all error messages are considered important

38

Summary on tool based static analysis

Static analysis is “Automated code inspections”Descended from compiler technologyGoal:

Check problems of code Documentation (Naming & Documenation)

— Important but difficult to automate

Layout / Visual representation— This is not currently so big problem

Ensure the system design and structure qualityCan good or poor design be measured? Yes and NoPrevent the code erosion caused by software evolution

Detect defectsAutomatic detection can find many hard to spot defects

Problems: NoisePeople can starting ignoring the messages and miss all defects detected

Need to make sure that all error messages are considered important

Tools often can focus one or more of these viewpointsProgramming errors (Lint, Findbugs)Documentation (Checkstyle)Layout (IDE’s auto reformat)Structure (Code metrics tools)

39

Contents


Code inspections

Tool based static analysis


40

Anti-Patterns and Bad Code Smells

Describe some of the common mistakes in software development

41

Pattern History

Design PatternsRepresent reusable designs

Strategy, Command, Singleton, etc

HistoryBased on pattern languages which focused on traditional architecture (1977)Rediscovered by software people 1987 and published by GoF in 1994

Motivation Capsulate years of industry experience on software development

Anti-PatternsRepresent frequently occurring problemsHistory

Have been around longer than Design PatternsFred Brooks and The Mythical Man-Month 1979

— “Adding manpower to a late software project makes it later”Term Anti-Pattern appeared soon after the book by GoF in 1994

MotivationLearning from mistakes

42

Anti-Patterns & Bad Code Smells

Anti-Pattern cover wider range of topics

DevelopmentSoftware Development

ArchitecturalSystems Architecture

ManagerialOrganization and Process

Bad code smellsAre in fact development level anti-patterns

43

Anti-Pattern examples – Golden Hammer

Also know as: Old Yeller, Head-in-the sand

Description: High knowledge on particular solution or vendor product

This knowledge is applied everywhere

Causes:Several successes with the particular approach

Large investment for the product

Isolated Group

44

Anti-Pattern examples – One Size Fits All

Also know as: Silver Bullet Software Process

Description: Software process does not fit in the business environment or lifecycle

Too bureaucratic process for small project

Own process created under the official one

Causes:IT management wants to have same process for all projects to avoid overrunning time and budget

45

Anti-Patterns & Bad Code Smells

Anti-Pattern cover wider range of topics

DevelopmentSoftware Development

ArchitecturalSystems Architecture

ManagerialOrganization and Process

Bad code smellsAre in fact development level anti-patterns

46

When to Refactor

Bad smells in the code tell us when to apply refactorings

Smells are structures that indicate bad software designIdea introduced by Fowler and Beck (2000)

Of course, the list of bad smells can never be complete

They are introduce with the set of refactoring that can remove the smells

Why are they called smells?Fowler & Beck think that when it comes to refactoring decision: “no set of metrics rivals informed human intuition”

47

Bad Code Smells - Taxonomy

BloatersWhen something is too large

Examples: Long Method, Large Class, Long Parameter List, These smells likely grow little bit a time

Hopefully nobody designs e.g. Long Methods

Object-Orientation abusersObject Oriented concept not fully understood and utilized

Examples: Switch statements, Temporary Field,, Alternative Classes with Different Interfaces

Change PreventersThese smells make changing the system unnecessarily difficult

Example: Shotgun surgeryViolate principle one external change should effect to one classonly

Change the database from Oracle to SQLServer

48

Bad Code Smells - Taxonomy Cont’d

DispensablesAll code needs effort to understand and maintain

If code is not used or redundant it needs to be removedExamples: Duplicate & Dead code, Speculative Generality

Couplers Low coupling between objects/classes is one the desirable goals of OO software

Examples: Feature Envy, Message Chains

Too much delegation (= reduction of coupling) can be bad as well

Examples: MiddleMan

49

Refactoring Example – Switch statement

Class Engine{ …

int cylinderCount(){

switch (type){

case FAMILY_CAR_ENGINE:

return getBaseSylinderNumber();

case EXECUTIVE_CAR_ENGINE:

return getBaseSylinderNumber() * 2;

case RACE_CAR_ENGINE:

return 8;

}

} +cylinderCount() : int

Engine

+cylinderCount() : int

FamilyCarEngine


ExecutiveCarEngine


RaceCarEngine

50

Summary subjective design and code quality indicators

Refactoring is improving the code structure with out changing it’s external behavior

Bad code smells and AntiPatterns are subjective quality indicators

They cannot be exactly measured and human mind provides the final judgment

It is possible that people have quite different opinions on them

51

ReferencesAnon (CheckStyle) http://checkstyle.sourceforge.net/Anon (Eiffel Software Inc.) Building bug-free O-O software: An introduction to Design by ContractTMhttp://archive.eiffel.com/doc/manuals/technology/contract/Bansiya, J., David, C.G. 2002, "A Hierarchical Model for Object-Oriented Design Quality", Software Engineering, IEEE Transactions on, vol. 28, no. 1, pp. 4-17.Barnard, J., Price, A. 1994, "Managing code inspection information", Software, IEEE, vol. 11, no. 2, pp. 59-69. Brown, W. J., Malveau, R. C., McCormick, H. W., & Mowbray, T. J. 1998, AntiPatternsRefactoring Software, Architectures, and Projects in Crisis Wiley, New York..Fowler, M. 2000, Refactoring: Improving the Design of Existing Code Addison-Wesley, Canada. Coleman, D., Ash, D., Lowther, B. & Oman, P.W. 1994, "Using Metrics to Evaluate Software System Maintainability", Computer, vol. 27, no. 8, pp. 44-49.Dunsmore, A., Roper, M. & Wood, M. 2003, "The development and evaluation of three diverse techniques for object-oriented code inspection", Software Engineering, IEEE Transactions on, vol. 29, no. 8, pp. 677-686. Dunsmore, A., Roper, M. & Wood, M. 2003, "Practical code inspection techniques for object-oriented systems: an experimental comparison", Software, IEEE, vol. 20, no. 4, pp. 21-29.Enseling Oliver, iContract: Design by Contract in Java, http://www.javaworld.com/javaworld/jw-02-2001/jw-0216-cooltools.htmlHoare, C.A.R, 1969 ”An Axiomatic Basis for Computer Programming”, Communications of the ACM 12,Laitenberger,Oliver; DeBaud,Jean-Marc, An encompassing life cycle centric survey of software inspection, J.Syst.Software, 2000, 50, 1, 5-31Li, W., Henry, S.M. 1993, "Object-Oriented Metrics that Predict Maintainability", Journal of Systems and Software, vol. 23, no. 2, pp. 111-122.Linger,Richard C. 1993, “Cleanroom software engineering for zero-defect software”, 2-13, IEEE Computer Society PressM. Lowry, M. Boyd, and D. Kulkarn Towards a Theory for Integration of Mathematical Verification and Empirical Testing p. 322, 13th IEEE International Conference on Automated Software Engineering (ASE'98), 1998. Martin, John C. ”Formal methods software engineering for the CARA system”, International-Journal-on-Software-Tools-for-Technology-Transfer. 2004; 5(4): 301-7McArthy J., 1962, “Towards a mathematical science of computation”, In Proceedings of IFIP 62, Munich, 21,8McConnell, S. 1993. Code complete. Redmond, Washington, USA: Microsoft Press. Porter, A.A., Siy, H.P., Toman, C.A. & Votta, L.G. 1997, "An experiment to assess the cost-benefits of code inspections in large scale software development", Software Engineering, IEEE Transactions on, vol. 23, no. 6, pp. 329-346. Pincus Jon 2002 “PREfix, PREfast, and other tools and technologies“ presentation at ICSM 2002 MontrealRoush, W. 2003, "Writing Software Right", MIT Technology Review, vol. 106, no. 3, pp. 26-28 Shull, F., et al. 2002. What we have learned about fighting defects. , 249-258., Software Metrics, 2002.Proceedings.Eighth IEEE Symposium on SEI, ClearnRoom, http://www.sei.cmu.edu/str/descriptions/cleanroom_body.htmlSethi, R. 1996, Programming Languages - Concepts and Constructs, Addison & Wesley. Sommerville, I. 2001, Software Engineering, Addison-Wesley / Pearson, Reading, MA, USA.

static code analysis and code review - soberit-software business

Documents