introduction to software testing (paul degrandis) [reading assignment: chapter 15, pp. 231-252 and...
Post on 25-Dec-2015
223 Views
Preview:
TRANSCRIPT
Introduction to Software Testing
(Paul deGrandis)
[Reading assignment: Chapter 15, pp. 231-252 and notes by Paul deGrandis]
Overview
• Terms and Definitions
• Tool Overview
• Static Checkers
• Unit Testing
• Coverage and Profiling
• Memory Analysis
• Bug Reporting and Tracking
• Continuous Integration
NOTE
•All the material and examples from here on are generated using a single makefile.
•Log into tux, then:wget http://www.pauldee.org/vandvmakefile
mv vandvmakefile Makefile •Export your classpath:
export CLASSPATH=`pwd`/junit/junit-4.4.jar:`pwd`/emma/emma.jar:.
•To make and download all tools and examples:make
Tool Overview
•Verification• Static Checking
• Unit Tests for code correctness
• Memory and performance specifications
•Validation• Unit Tests for behavior and requirements
Static Checkers
•Does not execute the code
•Finds suspicious code or security vulnerabilities
•Tools
•C - SPLINT,GCC Warnings
•Java - FindBugs, PMD
Cyclone - a safe C
GNU Compiler Collection
FindBugs - Find Bugs in Java
Static Checkers - CDemo
• Installation of SPLINT• Download
wget http://www.splint.org/downloads/splint-3.1.2.src.tgz
• Untartar -xzvf splint-3.1.2.src.tgz
• Make (during make, splint checks itself)cd splint-3.1.2/./configuremake
Static Checkers - CDemo
•Running SPLINT
• Simple and Basicsplint -preproc +weak +show-summary +stats ../src/*.c
• Dependable Softwaresplint -preproc +checks +show-summary +stats ../src/*.c
Static Checkers - JavaDemo
• Installation of FindBugs• Download
wget http://prdownloads.sourceforge.net/findbugs/findbugs-1.2.1.tar.gz
• Untartar -xvzf findbugs-1.2.1.tar.gz cd findbugs-1.2.1
Unit Tests
• Validate a unit of code; smallest testable part
• Executes the code in a sandboxed environment
• Testing mostly for functional requirements• can also test some non-functional requirements
• Many approaches and schools of thought• History Based, Risk Based, Data Path, DOE
Unit Tests Frameworks-CScreenshots
Check Unit Test Framework
http://check.sourceforge.net
Demo
Unit Tests Frameworks-C
• Installation of Check
•Writing a Check unit test
•Compiling your test(s)
•Running your test(s)
http://check.sourceforge.net/doc/check.html/index.html
Demo
Unit Tests Frameworks-C
• Installation of Check• Download
wget http://downloads.sourceforge.net/check/check-0.9.5.tar.gz
• Untartar -xvzf check-0.9.5.tar.gz
• Make (notice the GCC warnings passed)cd check-0.9.5./configuremake
Demo
Unit Tests Frameworks-C
•Writing a Check unit test
• Write your test fixtures
• Write your test(s)
• Write your test suite
• Write your main method (execute your suite)
Demo
Unit Tests Frameworks-C
•Writing a Check unit test
• Write your main method (execute your suite)
Demo
Unit Tests Frameworks-Java
• Installation of Junit
•Writing a Junit unit test
•Compiling your test(s)• Just use javac
•Running your test(s)• just use java
http://junit.sourceforge.net/ (look at the cookbook)
Demo
Unit Tests Frameworks-Java
• Installation of Junit• Download
wget http://downloads.sourceforge.net/junit/junit-4.4.jar
• Add the jar to your classpathexport CLASSPATH=$CLASSPATH:`pwd`/junit-4.4.jar:.
Demo
Unit Tests Frameworks-Java
•Writing a Junit unit test
•Write your test fixtures
•Write your test(s)
•Write your test suite
•Write your main method (execute your suite)
Code Coverage
•Degree to how much code was tested (E.g., How confident are we in our tests and code)
•Criteria:• Statement - Has each line been executed
• Condition - Every evaluation (if/else, try/catch, switch, loop)
• Path - Every route of code
Code Coverage - JavaScreenshots
EMMAhttp://emma.sourceforge.net/intro.html
DemoCode Coverage - Java
• Installation of Emma
• Downloadwget http://downloads.sourceforge.net/emma/emma-2.0.5312-lib.zip
• Unzipunzip emma-2.0.5312-lib.zip
• Instrument Your Codejava -cp emma.jar:./sample:../junit/junit-4.4.jar emma instr -outdir ./sampleout -cp ./sample/ SimpleTest
• Run Your Code and Generate a reportjava SimpleTestjava -cp emma.jar emma report -r html -in sampleout/coverage.em,sampleout/coverage.ec
Memory Analysis
•Sandboxes an application in a VM
•Uses:• Memory Leak - allocate memory, but don’t free
• Buffer Overflow - access memory outside of a fixed buffer
• Profiling - performance analysis
DemoMemory Analysis - C
• Installation of Valgrind• Download
wget http://downloads.sourceforge.net/check/check-0.9.5.tar.gz
• Untartar -xvjf valgrind-3.2.3.tar.bz2
• Make (notice the GCC warnings passed)cd valgrind-3.2.3./configuremake
DemoMemory Analysis - C
•Running of Valgrind• Compile with debugging enabled
gcc -g myprog.c
• Run Valgrindvalgrind --leak-check=yes myprog arg1 arg2
• Full tutorial and infoformationhttp://valgrind.org/docs/manual/quick-start.html#quick-start.intro
Issue and Bug Tracking
•Organizes and tracks issues, enhancements, features, and bugs in a software system
•Usually contains a database backend and a web frontend
•Generate report of project and team metrics and productivity
Issue and Bug Tracking
•Trac (http://trac.edgewall.org/)
•Roundup (http://roundup.sourceforge.net/)
•Tracker (http://tracker.rubyforge.org/)
Continuous Integration
•Committing changes early and often
•Building and testing at least nightly
• If code breaks, revert
• Integration issues discovered early as developers work with the latest code
top related