test driven development for embedded c, why debug? esc-411 … · 2011-09-23 · test driven...
TRANSCRIPT
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Test Driven Developmentfor Embedded C, Why Debug?
ESC-411Boston 2011
James W [email protected]
www.jamesgrenning.comwww.twitter.com/jwgrenning
You can find my book athttp://pragprog.com/titles/jgade/
1
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Questions I’ll Answer
• Why should I care about TDD?• What is TDD?• How is TDD adapted to embedded development? • How do I test code with dependencies?• Why are Unit Tests critical?• Why is Automation Critical?
2
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Don’t be Late, Don’t Have Bugs
MissedCustomer
ExpectationsLate Project
Poor Quality3
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
We Make Our Problems
Vague Requirements
MissedCustomer
ExpectationsLate ProjectUnrealistic Plan
Unplanned Activities
Unpredictable Activities
Poor Quality4
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Positive Feedback - Unstable System
Unrealistic Plan Late ProjectMissed
Customer Expectations
Schedule Pressure
Long Hours
Bugs Poor Quality
Vague Requirements
Unplanned Activities
Unpredictable Activities
5
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
This Work Flow is Designed to Allow Defects
6
Development
Test
Defects
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
7
Your program will have bugs. And they will surprise you when you find them.
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
The Physics of Debug Later Programming (DLP)
• As Td increases, Tfind increases dramatically• Tfix is usually short, but can increase with Td
8
Bug discoveryMistake made(bug injection)
Bug found Bug fixed
Td Tfind T fix
Time
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
A Bug’s Life
9From http://www.softwaretestinghelp.com/bug-life-cycle/
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Edsger DijkstraThose who want really reliable software will discover that they must find means of avoiding the majority of bugs to start with, and as a result, the programming process will become cheaper. If you want more effective programmers, you will discover that they should not waste their time debugging, they should not introduce the bugs to start with.
10
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Can we Realize Dijkstra’s Dream andPrevent Defects with
Test Driven Development?
11
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Development and Test Work TogetherPreventing Defects
12
Development
Test
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
13
A Complex system that works is invariably found to have evolved from a simple system that worked.
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
The Physics of Test Driven Development
• When Td approaches zero, Tfind approaches zero• In many cases, bugs are not around long enough to be considered
bugs.• See: http://www.renaissancesoftware.net/blog/archives/16
14
Mistake discovery
Mistake made
Root cause found
Mistake fixed
T d Tfind T fix
Time
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Zune 30G
• On December 31, 2008 this function went into an infinite loop.
• Ruining New Years Eve Parties around the world.
15
– It accepts the number of days since January 1, 1980 and figures out: day, date month and year.
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
16
Zune 30G Bug tracked to this Function
BOOL ConvertDays(UINT32 days, SYSTEMTIME* lpTime)
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
17
One the Got Awaystatic void SetYearAndDayOfYear(RtcTime* time){ int days = time->daysSince1980; int year = STARTING_YEAR; while (days > 365) { if (IsLeapYear(year)) { if (days > 366) { days -= 366; year += 1; } } else { days -= 365; year += 1; } }
time->dayOfYear = days; time->year = year;}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
18
This Test Could Have Prevented it
TEST(Rtc, check20081231){ days = daysSince1980(2008, 366); CHECK(ConvertDays(days, &time)); assertDate(WED, 2008, 12, 31);}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
What Tests to Write
• One Test Would Have Prevented Zune Bug• If we knew where the bugs are, we could just write
those tests (or fix the bug).• Bugs can be anywhere, so we have to write tests for
everything.
19
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
What is TDD?
20
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
TDD Micro Cycle
• Write a test• Watch it not compile• Make it compile, but fail• Make it pass• Refactor (clean up any mess)• Repeat until done
21
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
TDD State Machine in C/C++
22
Write the test
Make the test link
Make the test compile
Compilation error
Link error
New test failsMake the test pass
Refactor(Make it right)
All tests pass
All tests pass No more tests
Choose a test
Start
Compilation error
Link error
DONE!
Programming error
Compiles Clean
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Test Driving a Circular Buffer (FIFO)
23
Out-Index In-Index
23 7 66 12 99 16 90
Out-Index In-Index
42 -6 23 7 66 12 99 16 90
Out-Index In-Index
99
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Create a Test-List(avoid analysis paralysis)
24
Circular Buffer Tests
Initially Empty Transition to empty Transition to Full Transition from Full Put-Get FIFO Put to full Get from empty Filled but not wrapped around Wrap around
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Setup a Test Fixture
25
TEST_GROUP(CircularBuffer){ CircularBuffer* buffer;
void setup() { buffer = CircularBuffer_Create(); }
void teardown() { CircularBuffer_Destroy(buffer); }};
TEST(CircularBuffer, TestName){}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make a Partial Skeleton of the Production Code Header File
26
#ifndef D_CircularBuffer_H#define D_CircularBuffer_H
typedef struct CircularBuffer CircularBuffer;
CircularBuffer * CircularBuffer_Create();void CircularBuffer_Destroy(CircularBuffer *);
#endif // D_CircularBuffer_H
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make a Partial Skeleton of the Production Code Source File
27
#include "CircularBuffer.h"#include <stdlib.h>
struct CircularBuffer{ int dummy;} ;
CircularBuffer* CircularBuffer_Create(){ CircularBuffer* self = calloc(1, sizeof(CircularBuffer)); return self;}
void CircularBuffer_Destroy(CircularBuffer* self){ free(self);}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Choose a Test
28
Write the test
Make the test link
Make the test compile
Compilation error
Link error
New test failsMake the test pass
Refactor(Make it right)
All tests pass
All tests pass No more tests
Choose a test
Start
Compilation error
Link error
DONE!
Programming error
Compiles Clean
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Which Test?
29
Out-Index In-Index
41 59 14 33 7 31
In-Index Out-Index
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Write a Test
30
Write the test
Make the test link
Make the test compile
Compilation error
Link error
New test failsMake the test pass
Refactor(Make it right)
All tests pass
All tests pass No more tests
Choose a test
Start
Compilation error
Link error
DONE!
Programming error
Compiles Clean
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Start with a Test That is Easy to Get to Pass -- Designing the Interface
31
TEST_GROUP(CircularBuffer){ CircularBuffer* buffer;
void setup() { buffer = CircularBuffer_Create(); }
void teardown() { CircularBuffer_Destroy(buffer); }};
TEST(CircularBuffer, ShouldBeEmptyAfterCreate){ CHECK(CircularBuffer_IsEmpty(buffer));}
Write the test
Make the test link
Make the test compile
Compilation error
Link error
New test failsMake the test pass
Refactor(Make it right)
All tests pass
All tests pass No more tests
Choose a test
Start
Compilation error
Link error
DONE!
Programming error
Compiles Clean
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make the Test Compile
32
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make the Test Compile
33
#ifndef D_CircularBuffer_H#define D_CircularBuffer_H
typedef struct CircularBuffer CircularBuffer;
CircularBuffer * CircularBuffer_Create();void CircularBuffer_Destroy(CircularBuffer *);int CircularBuffer_IsEmpty(CircularBuffer *);
#endif // D_CircularBuffer_H
Write the test
Make the test link
Make the test compile
Compilation error
Link error
New test failsMake the test pass
Refactor(Make it right)
All tests pass
All tests pass No more tests
Choose a test
Start
Compilation error
Link error
DONE!
Programming error
Compiles Clean
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make the Test Link
34
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make the Test Linkand Intentionally Fail
35
#include "CircularBuffer.h"#include <stdlib.h>
struct CircularBuffer{ int dummy;} ;
... not all code shown ...
int CircularBuffer_IsEmpty(CircularBuffer* self){ return 0;}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Watch it Fail
36
CircularBufferTest.cpp:48: error: Failure in TEST(CircularBuffer, ShouldBeEmptyAfterCreate)
CHECK(CircularBuffer_IsEmpty(buffer)) failed
Write the test
Make the test link
Make the test compile
Compilation error
Link error
New test failsMake the test pass
Refactor(Make it right)
All tests pass
All tests pass No more tests
Choose a test
Start
Compilation error
Link error
DONE!
Programming error
Compiles Clean
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make the Test Pass
37
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Write no More Code than is Necessary to Pass the Current Test
• Hard code behavior or partial implementations are OK and often preferred.
38
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Make the Test Pass
39
#include "CircularBuffer.h"#include <stdlib.h>
struct CircularBuffer{ int dummy;} ;
... not all code shown ...
int CircularBuffer_IsEmpty(CircularBuffer* self){ return TRUE;}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Repeat until You Run Out of Tests
• The first tests drive the interface definition• Later tests complete the behavior• You will think of more tests as you go
– Write the new test– or Add it to the test list
40
Code without
new feature
Code withnew tested
feature
Half the way through, half the tests are passing and the code is
partially complete
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
But that is so Inefficient!So Slow!
Show us the fast way!
41
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
42
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Get good at being careful and you can go fast.
43
The Careful way is the Fast Way!
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Sustainability
44
Speculate Code Test DebugTraditional development
Time developers do not notice nor plan for
Test-driven development
Speculate Test and Code Debug
Feels slower
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
45
A Complex system that works is invariably found to have evolved from a simple system that worked.
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Adaptation for Embedded Software
What is scarce during embedded development?
46
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Hardware is Scarce!
• It does not exist.• It is being used by
someone else.• It has bugs of its
own.
47
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Minimize DoH!
Debug
on
Hardware48
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Use Your Development Systemfor a Test Bed
• Multi-targeted code.• Must beware of hardware and OS dependencies.• Object Oriented approach to Dependency
Management.• Avoid inefficiencies
– Waiting for hardware.– Waiting for restart.– Waiting for downloads.– Waiting for long compiles.– Debugging on the target.
49
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
But There are Risks with Development System Tests
• Architecture differences– Word size– Big-endian Little-endian– Alignment
• Compiler differences• Library differences• Execution differences
50
Write a TestMake it Pass
Refactor
Stage 1
Compile for TargetProcessor
Stage 2
Run Tests in the Eval Hardware
Stage 3
Run Testsin Target Hardware
Stage 4
AcceptanceTests
Stage 5
More Frequent Less Frequent
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
TDD Adaptation for Embedded Development
51
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
TDD Adaptation for Embedded Development
52
See : http://renaissancesoftware.net/files/articles/ProgressBeforeHardware.pdf
Write a TestMake it Pass
Refactor
Stage 1
Compile for TargetProcessor
Stage 2
Run Tests in the Eval Hardware
Stage 3
Run Testsin Target Hardware
Stage 4
AcceptanceTests
Stage 5
More Frequent Less Frequent
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Problems Found at Appropriate Stage
53
Stage Problems Likely to Find
1 Logic, design, modularity, interface, boundary conditions
2 Compiler compatibility (language features)Library compatibility (header files, prototypes)
3 Processor executions problems (bugs in compiler and standard libraries)Portability problems (word size, alignment, endian)
4 Ditto stage 3Hardware integration problemsMisunderstood hardware specifications
5 Ditto stage 4Misunderstood feature specification
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
How do I test Code with Dependencies?
54
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
A Module with Collaborators
• Every minute, the RTOS wakes up the Light Scheduler.
• If it is time for one of the lights to be controlled, the LightController is told to turn on/off the light.
55
Time Service
+ GetTime()+ SetPeriodicAlarm()
LightScheduler
+ScheduleTurnOn()+RemoveSchedule()+WakeUp()
Light Controller
+ On(id)+ Off(id)
Hardware RTOS
<<anonymous callback>>
Admin Console
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Program to Interfaces
• Separate interface and implementation as separate entities.
• This design has good separation of responsibilities
56
<<interface>>Time Service
+ GetTime()+ SetPeriodicAlarm()
LightScheduler
+ ScheduleTurnOn()+ RemoveSchedule()+WakeUp()
<<interface>>Light Controller
+ On(id)+ Off(id)
Model 42 Hardware RTOS
<<anonymous callback>>
Model 42Light Controller
RTOS Time Service
<<implements>> <<implements>>
Admin Console
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Testing a Module with Collaborators
• Use the real collaborators if you can.
• Use fakes when you must.
57
<<interface>>Time Service
+ GetTime()+ SetPeriodicAlarm()
LightScheduler
Test
LightScheduler
+ ScheduleTurnOn()+ RemoveSchedule()+wakeUp()
<<interface>>Light Controller
+ On(id)+ Off(id)
Light Controller Spy
FakeTime Service
<<implements>> <<implements>>
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Testing the Scheduler
58
TEST(LightScheduler, ScheduleOnTodayNotTimeYet){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(999);
LightScheduler_Wakeup();
LONGS_EQUAL(LIGHT_NA, LightControllerSpy_GetState(3));}
TEST(LightScheduler, ScheduleOnTodayItsTime){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(1000);
LightScheduler_Wakeup();
LONGS_EQUAL(LIGHT_ON, LightControllerSpy_GetState(3));}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Partial Skeleton Lets You Try the Design and Test Ideas Early
59
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Right Next to the Siliconwith
Mock Objects
60
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Message Flow for Flash Memory
Block Erase with Error
61
FlashDriver FlashDevice
IOWrite(CommandRegister, 0x40)
IOWrite(offset, data)
IORead(StatusRegister)
b7 == 0IORead(StatusRegister)
b7 == 1, other bits == 0
Repeats while b7 == 0
IORead(offset)
data
Flash_Program(offset, data)
FlashSuccess
b7 == 1
Start
Program CommandWrite 0x40 to 0x0
Start/Stop
Write data to address
Read status register
b3 == 0
b4 == 0
b1 == 0
YES
NO
YES
YES
YES
Vpp Error
Program Error
Protected Block Error
NO
NO
NO
Clear status Write 0xFF to 0x0
Wai
t for
read
y lo
op
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Flash Program Flow Chart
62
How do you test these errors in the target?
How do you test these errors in the target?
How Many Tests are Needed?
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Test List
63
Flash Memory Write - Test List Successful operationInvalid programming voltageProgram errorAttempt to program protected blockRead back failsDevice timeout during write operation
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Flash Driver Test Fixture
64
<<implementation>>IO
<<implements>>
+ Read(addr) : data+ Write(addr, data)
<<interface>IO
+ Expect_Read(addr, data)+ Expect_Write(addr, data)
MockIO<<hardware>>
+ Program(addr, data)+ ProtectBlock(block)+ EraseBlock(block)//etc
FlashDriver
FlashDriverTest
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Flash Driver Test
65
TEST(Flash, ProgramSucceedsReadyImmediately1){ int result = 0; MockIO_Expect_IOWrite(0, 0x40); MockIO_Expect_IOWrite(0x1000, 0xBEEF); MockIO_Expect_IORead(0, 1<<7); MockIO_Expect_IORead(0x1000, 0xBEEF);
result = Flash_Program(0x1000, 0xBEEF);
LONGS_EQUAL(0, result); MockIO_Assert_AllExpectationsMet();}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Mock Flash Read/Write
66
//MockIO.h#include "IOReadWrite.h"
void MockIO_Create(int maxExpectations);void MockIO_Destroy();void MockIO_Expect_IOWrite(ioAddress_t offset, uint16_t data);void MockIO_Expect_IORead(ioAddress_t offset, uint16_t data);void MockIO_Assert_AllExpectationsMet();
//IOReadWrite.h#include <stdint.h>
typedef uint32_t ioAddress_t;typedef uint16_t ioData_t;
ioData_t IORead(ioAddress_t offset );void IOWrite(ioAddress_t offset, ioData_t data);
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Mock Flash Read/Write
67
//Snipvoid MockIO_Create(int maxExpectations){ expectations = malloc(sizeof(Expectation)*maxExpectations); memset(expectations, 0, sizeof(expectations)); setExpectationCount = 0; getExpectationCount = 0; maxExpectationCount = maxExpectations; failureAlreadyReported = 0;}
//MockIO.ctypedef struct Expectation{ int kind; ioAddress_t addr; ioData_t value;} Expectation;
static Expectation* expectations = 0;static int setExpectationCount;static int getExpectationCount;static int maxExpectationCount;static int failureAlreadyReported = 0;
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Capturing Flash Read/Write Expectations
68
void MockIO_Expect_IOWrite(ioAddress_t addr, ioAddress_t value){ failWhenNoMoreRoomForExpectations(report_too_many_write_expectations); recordExpectation(FLASH_WRITE, addr, value);}
void MockIO_Expect_IORead(ioAddress_t addr, ioAddress_t value){ failWhenNoMoreRoomForExpectations(report_too_many_read_expectations); recordExpectation(FLASH_READ, addr, value);}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Mock Flash Read
69
ioData_t IORead(ioAddress_t addr){ setExpectedAndActual(addr, -1); failWhenNotInitialized(); failWhenNoUnusedExpectations(report_read_but_out_of_expectations); failWhen(expectationIsNot(FLASH_READ), report_expect_write_was_read); failWhen(expectedAddressIsNot(addr), report_read_wrong_address);
return expectations[getExpectationCount++].value;}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Mock Flash Write
70
void IOWrite(ioAddress_t addr, ioData_t value){ setExpectedAndActual(addr, value); failWhenNotInitialized(); failWhenNoUnusedExpectations(report_write_but_out_of_expectations); failWhen(expectationIsNot(FLASH_WRITE), report_expect_read_was_write); failWhen(expectedAddressIsNot(addr), report_write_does_not_match); failWhen(expectedDataIsNot(value), report_write_does_not_match); getExpectationCount++;}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
All Expectations Must Be Met
71
void MockIO_Assert_AllExpectationsMet(){ if (failureAlreadyReported) return; failWhenNotAllExpectationsUsed();}
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
72
Some of the Things Professional Developers Need to Know
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Unit Tests Are Critical
Though Not Enoung
73
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
System Level Tests Cannot be Thorough
• 1000 (or more) tests are needed to test this simple system
74
10 states
10 states
10 states
5 interactions
5 interactions
5 interactions
Tests
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
TDD unit tests can be thorough
• As few as 30 unit tests and 15 integration test when tested as units
75
10 states
10 states
10 states
5 interactions
5 interactions
5 interactions
Tests Tests
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
TDD is About Design
76
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Refactoring
77
Structured code transformation to prepare the code
for change
Key points:• Doesn’t adjust functionality• Small and disciplined• Well defined• Keeps code healthy
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Refactoring’s Three Critical Skills
78
Recognize what is wrong, fix it!Keep the code working the whole time!
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Modularity == Testability
79
Focus on tests first ensures modularity and other good design principles
It’s about design, stupid!
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
80
Design from the perspective of userather than implementation
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Tests Must Be Automated
81
test
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Assume Test Effort is Proportional to Development Effort
82
Effort
Iterations
dev
5
10
15
20
25
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
• 25% of all defects are introduced while changing and fixing code
• [R.B Grady, Software Process Improvement]
83
If a system is working, leave it alone. Don't change anything
test
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Manual Test is Unsustainable
84
Effort
Iterations
Unsustainable growth
dev
5
10
15
20
25
test
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Risk Accumulates in the Untested Code Gap
85
Effort
Iterations
Unsustainable growth
dev
5
10
15
20
25 Untested Code Gap
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Why Don’t We Just Test at the End, Save all that Retesting?
Requirements
Code
Design
Test
Time
NovemberSeptemberJulyMayMarch
Test and FixTest and Fix
Test and FixTest and Fix
Test and FixTest and Fix
Test and Fix
Test and FixTest and Fix
Test and FixTest and Fix
Test and FixTest and Fix
Test and FixTest and Fix
Test and FixTest and Fix
86
Test
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
FBI Case-File System - Never Delivered
87
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Denver Baggage SystemAlmost One Year Late
88
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Exercise
89
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
Why is it Worth Changing How You Program
• Defects kill predictability:–Cost of fixing is not predictable–When they materialize is not predictable
• Test-driven is predictable:–Working at a steady pace–Results in fewer bugs–More productive than “debug-later programming”
• Test-driven programmers rarely need the debugger
90
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
References and Resources• [XP] Kent Beck, Extreme Programming Explained, 1999• [REF] Martin Fowler. Refactoring. Improving the Design of Existing Code. 1999• [WELC] Michael Feathers, Working Effectively with Legacy Code• [TDD] Kent Beck, Test-Driven Development, 2003• [XUNIT] Gerard Meszaros, xUnit Testing Patterns, 2008• [PRAG] Andy Hunt, Dave Thomas, The Pragmatic Programmer• [SLAD] Craig Larman and Bas Voode, Scaling Lean & Agile Development• [POP] Mary Poppendieck and Tom Poppendieck, Implementing Lean Software Development:
From Concept to Cash, 2006• [AGILE] Robert C. Martin, Agile Software Development: Principles, Patterns, and Practices, 2002• [CLEAN] Robert C. Martin, Clean Code, 2008• [LESSONS] Cem Kaner, et. al. Lessons learned in Software Testing• [TD] Lasse Koskela, Test Driven, 2007
• Test harnesses– [CPPTEST] www.sourceforge.org, project CppUTest– [FITNESSE] www.fitnesse.org
• Groups– http://groups.yahoo.com/group/testdrivendevelopment– http://groups.yahoo.com/group/AgileEmbedded
91
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
See Embedded TDD and Related Blogs and Papers
http://www.renaissanceSoftware.net http://www.renaissancesoftware.net/blog/
• Embedded TDD• Zune Bug: Test Driven Bug Fix• Learning Tests are Free!
• TDD as a Design Rot Prevention System• Crashing Your Way to Great Legacy C Tests • TDD and the Big Framework Part• Bug Fixes and TDD• Physics of Test Driven Development• Tests vs. Short Term Cache Between Your Ears• Embedded Systems Conference FAQ • I miss constructors • Who says you can’t test drive a device driver?
92
• Why are You Still Using C? • Planing Poker• Agile Embedded Software Development (ESC)
• Launching Extreme Programming at a Process Intensive Company (IEEE)
• Test Driven Development for Embedded Software• Progress Before Hardware • Agile Times - Containing Progress Before
Hardware• Test-Driven Development for Embedded C++
Programmers
Test-Driven DevelopmentESC-411
Copyright © 2008-2011 James W. GrenningAll Rights Reserved. For use by training attendees.
http://pragprog.com/titles/jgade/
93