e methods & toolsquality methods & tools * summer 2001 * page 3 familiar with its use,...

e

METHODS & TOOLSGlobal knowledge source for software development professionals ISSN 1023-4918Summer 2001 (Volume 9 - number 2)

Bring Back the User-Friendly Paradigm!

In the 1990, the diffusion of the GUI-based operating systems (MacOS and Windows) has put anew perspective on the user interface aspect of software development. The buzzwords were thatinterfaces have to be "user-friendly" and "intuitive". These notions are sometimes vague andhave a marketing content. As a long term Windows user, I am still not at ease with the Macinterface, even if this product is regarded as being more "intuitive". Nonetheless, the conceptsdeveloped at Xerox-PARC are now widely accepted and shared. The GUI vocabulary (window,push button, drop down list, etc.) is used in all software development.

One of the benefits from this era was that, for a moment, GUI-based applications developed byexternal vendors and internal teams could have a common "look and feel", if the developerswere respecting the visual standard of the operating system. The domination period of theclient/server paradigm was however a relatively short one. The ascent of the Internet and thediffusion of Web-based applications completely changed the situation. Running in a browser,the application interface is independent of the client operating system. Based on HTML andavailable extensions (JavaScript, Flash, etc.), each application is able to regain the "freedom" tohave its own graphical standard. The different - and sometimes delirious - looks that are visibleon Internet symbolise how the "look" aspect overtook the user interface vision. However, is it inthe interest of the user (often a customer) to suffer two minutes Flash introductions or todownload heavy graphical files over a low-bandwidth modem connection? Is this the best wayto use the server and network resources? The situation reminds me the first days of wordprocessing software. Freed from the typewriters, people produced documents with ten differentfonts in four sizes, just because it was easy...

Let us remember that the first objective of the interface is to ease the usage of the application.Just look at heavy traffic sites like Yahoo for an example on how simplicity meets usefulness!

InsideAutomation Strategy Out of the Garage .................................................................................. page 2

HOORA, The Method for Requirements Analysis ................................................................. page 8

The Work Situation in Software Development ....................................................................... page 13

Quality

Methods & Tools * Summer 2001 *

Abstract

As a software engineer, I have been involvedin software quality and testing of variousforms of commercially developed softwarefor over seven years. Many of these softwareapplications ranging from database tierservers, LAN|Web communication, HRapplications, installers, and printer devicedrivers. In all of these software examples, Ideveloped various forms and levels ofautomation to increase the efficiency of thetesting effort with an equal variety ofsuccess. It has been my experience that mostof the success implementing automated testtools has primarily not resided in theautomation tools employed. Rather, it hasexisted more in the vision of the leadershipto make a paradigm shift from old methodsof testing software in order to actualize thevalue of automated test tools. Without anenvironment designed, accepted, anddeveloped in all stages of the test planningprocess the success of automation can beplagued with delays, feature changes, andresistance to change. The following testingstrategy was developed to assist my HPcolleagues and partners to recognize thenecessity to reduce the time and cost oftesting methods as it currently exists. I haveborrowed from the HP Rules of the Garageas a template in defining the paradigm shiftneeded to facilitating better, faster, andsmarter use of software testing resources.

Believe you can change the world

Past success can be an invigorating reminderof how good a job we have done; that thereis no need to alter the course of how we testour products, or whether we need toinvestigate alternatives to the way we haveperformed testing on past products. In thisview, past success can be a detriment tobeing able to change testing habits. To

believe that one can change the worldrequires that we are willing to make changeourselves. I think Carly Fiorino stated thisbest, “We must be willing to eat our own dogfood…”. This means you cannot simplyadvocate the benefits of automation. Onemust actually implement and use the toolswithin a process improvement cycle –plan|do|check|act (the Shewhart cycle)[Grady]. Automation development andtesting must be understood as having nodifference in application from that of thedevelopment practices involved in thesoftware being tested.

Work quickly, keep the tools unlocked,and work whenever

Although software testing is oftenconstrained to follow the productdevelopment release schedule, itnevertheless should make every effort tooptimize planning and arrange resources onits own well defined set of checkpoints.Setup time must be reduced to a minimum orcompleted prior to drop of the next coderelease. It is essential to develop processesthat enable testing as soon as release dropsoccur. This requires significant reduction insetup time associated with testing of the nextsoftware release cycle candidate. In mostcases, there needs to be a consistent testenvironment established having static testmachines in place at all times. In addition, atable of objectives and backup proceduresshould be prepared in advance toaccommodate dynamic testing where avariety of SW|HW|OS configurations arenecessary to complete the scope of plannedtesting. We cannot move quickly enough ifwe choose to wait until the next drop occurs.

Testing tools are only valuable if they getused. Particularly with complex automation,it is necessary for the tool users to become

Automation Strategy Out of the Garage

Mark Rossmiller, SWQA Software Design Engineer,Vancouver Printer Division, Hewlett Packard

Quality


familiar with its use, provide for theexpanded use of the tool, and mostimportantly, to provide usage feedback thatfurther enhances the value of the tool intesting. It can then be determined if thesupport of the tool for required testing hasmerit or if a suitable alternative is available.Some of the risks in practice that caninherently devalue tool and automationinvestment are:

1. Lack of a sponsorship authority thatchampions and continuously promotesuse of the tool(s).

2. Tools are implemented for testing that isineffective, or not as originally intended.

3. Support and involvement of partners arenot incorporated in the tool design.

4. Significant and sometimes un-communicated feature changes to theproduct being tested occur.

5. Unclear ownership, expected outcome,and training in the use of the tool.

6. Lack of motivation to use the tools;usage improvements thus cannot bedetermined rendering the tool obsolete or“locked”.

When testing resources become scarce,further improvements in efficiency can stillbe realized through automation that can berun unattended or after-hours. “Lights out”testing has been developed and available forseveral years at HP. VCD has severalautomation tools built with Visual Test onsite and leveraged from other sites. Thesetools have great value, have a history ofproven use, and should continue to evolve tomeet changing product testing demand.(Please find tool detail in Resources sectionof this document).

Know when to work alone and when towork together.

“Lasting productivity improvement mustcome from within and cannot be imposedfrom without.” [Bumbarger].If you want to work alone in an organization,propose radical change. Obviously, this is

not the time to work alone since progressivechange requires a receptive and collaborativeeffort. But the act of proposing changepresents many perceptual, habitual, cultural,emotional, and political obstacles from thosewho would be expected participants andsupporters. That is why these environmentalfactors need to be addressed or it is futile tointroduce the tools of change. Real, lastingproductivity improvement requires change.And change requires creativity andinnovation. The irony in productivityimprovement is that success is most unlikelyto succeed when imposed from without – butoften is forced to change when ineffectivefrom within.

Share tools, ideas. Trust your colleagues.

Results of using automation in softwaredriver testing showed a reduction in testingtime by half. There is also the inherentbenefit of consistency of execution –repeatability. Thousands of keystrokes orsettings can be repeated exactly without thevariation when done by hand. Automationreduces thrashing through defects that arenot properly characterized and/or hard toduplicate. This results in thousands ofreported defects over time remaining openand without a clear resolution. Automationfurther benefits the tester by providing timesavings when multiple SW|FW|HW|OSconfigurations are required. A multiple oftests can therefore be run concurrently andmultiply the effectiveness of the testers time.

Figure 1. Illustration of statistical average.

Quality


No politics, no bureaucracy

Organizations by the very nature of theirexistence are political and as they grow -become bureaucratic. Our organization is noexception. But this does not mean thatvisionary thinking cannot be used to improveor eliminate as many obstacles within theorganization to effect positive change.

The customer defines a job well done

From the standpoint of software quality, thecustomer as well as the next customerrepresents Argus project data, R&D partners,beta testing, outsource testing, and finalmedia to name a few. Automation hasindirectly provided improvements inservicing the next customer via automateddefect handling tools to report defects moreefficiently and earlier in the developmentcycle.

Automated defect submittals have, for thepast 2 years, contributed the largest share ofreported defects at a saving of more than 771hours in submittal costs. We should continueto support these systems and provide thetechnical resources and tools to maintain thereduced cost of defect handling.

Radical ideas are not bad ideas.

I found the following “commandments” byRobert Schuller highly insightful indetermining our organizational behaviortendency to resist radical ideas. If one canrelate to experiencing one of thesecommandments then we have taken the firststep in recognizing where to start improvinghow we approach testing:

1. Never reject a possibility because yousee something wrong with it.

2. Never reject a possibility because youwill not get the credit.

3. Never reject an idea because it isimpossible.

4. Never reject a possibility because yourmind is made up.

5. Never reject an idea because it is illegal.

6. Never reject an idea because you do nothave the money, manpower, muscle, ormonths to achieve it.

7. Never reject an idea because it willcreate conflict.

8. Never reject an idea because it is notyour way of doing things.

9. Never reject an idea because it might fail.

TOTAL AUTOMATED OUTSOURCE SUBMITTALS928 DEFECTS Y2K - CURRENT

Cost Savings ~ 277 hours

7

12

22

5

9

1

15

1011

15

8

1 12 2 1

5 6

1 114

8

12

16

4

30

12

35

1

6

11811

20

13

26

59

347

1

19

45

1316

12

28

42

8 9

364

11

2

10

3

13

32

16

28

1618

67

13

2

109

2

7

19

1310

1

107

2 1 1 11 1

11

16

21

36

6

21 23

74122

0

10

20

30

40

50

60

70

05.01.00

12.01.00

19.01.00

26.01.00

02.02.00

09.02.00

16.02.00

23.02.00

01.03.00

08.03.00

15.03.00

22.03.00

29.03.00

05.04.00

12.04.00

19.04.00

26.04.00

03.05.00

10.05.00

17.05.00

24.05.00

31.05.00

07.06.00

14.06.00

21.06.00

28.06.00

05.07.00

12.07.00

19.07.00

26.07.00

02.08.00

09.08.00

16.08.00

23.08.00

30.08.00

06.09.00

13.09.00

20.09.00

27.09.00

FY'00 Total = 928 submittalsCost Savings ~ 277 hours

FY'99 Total = 1,681 submittalsCost Savings ~ 504 hours

Figure 2. Efficiency improvements

Quality


10. Never reject an idea because it is sure tosucceed.

Demand change

Once again, the approach taken toautomating software testing is greatlyaffected by the environment in which it isexpected to perform. Currently, the softwarequality organization is entrenched in atechnician rich paradigm where testingmethods are geared toward manual, ad hocvalidation of the software driver andassociated printer products. This method isoften far down-stream from where theeventually discovered defect – actually wasintroduced in the development design.

(1) The point is that optimization(automation) should be introduced at themost optimal time in the development cycleto prevent defects from being introduced.

(2) To put automation and tools where theyare most effective requires the softwarequality organization to reclassify the rolesand responsibilities of its quality engineeringstaff. Tools engineering should operateclosest to our R&D partners in order toprovide automation at the most effectivetime when design defects can be filtered out.This is before driver release candidatesarrive in the labs for validation; rather thandown stream where canned testing produceslimited results.

(3) Software quality must aggressivelyevaluate the core competencies of its staffand actively anticipate the knowledge basenecessary to meet current and future testingrequirements for new and emergingtechnology. The organizations inability tofill requisitions for needed technical staff andFW engineers, for example, is an indicatorthat internally we have not adequatelyanticipated demand external to HP; or wehave not grown our own competenciesinternally to meet future need.

(4) Software quality does not employconsistent testing methods that help facilitateautomation, practical tools, or overall

efficiency improvement. Other side effects tothis inconsistency make valuation of cost ortime associated with testing nearlyimpossible. Features may be added, changed,or even removed in development, whichtends to ripple through the test plans andrelevant test cases. The way in which wecharacterize and manage our defects is notconsistent creating non-value added workactivity increasing the cost of testing.

Invent different ways of working

Here is an analogy to explain the situation ina software quality organization. There are anumber of people feverishly bailing waterfrom a leaky boat. The captain brings onmore people to help bail but none of themknows how to fix the leak in the boat. Theweight of the additional people only makesthe water rush into the boat faster and noweveryone is feverishly bailing out the boat.Meanwhile, the boat has no direction and isgoing nowhere. The best solution to thecaptain’s problem might be to bring peopleon board who can fix the leaks in the boatrather than bail water. Now the captain canstop the leaking, find a direction, and getunderway making progress toward a goal.

Our organization must develop a new way ofworking, by not only investigating but alsoimplementing new ways of testing ourproducts. Automated testing methods are oneof many ways to do that if properlysponsored and approached. Bothmanagement and testing personnel havebecome accustomed to doing things the sameway, with relative success, and continuallyfall back on inefficient testing methods weare familiar with but cannot make realprogress in improving. We have becomecomplacent and indifferent to changing themethods of testing with which we are mostfamiliar and comfortable. Therefore, whenthe work starts rushing in and people becomestressed, we all start bailing faster andgetting nowhere for the future.

Some common risks to the test automationeffort include management and teammember’s support fading after not seeing

Quality


immediate results, especially when resourcesare needed to test the current release.Demanding schedules will put pressure onthe test team, project management andfunding management to do what it takes toget the latest release out. The reality is thatthe next release usually has the sameconstraints and you will wish you had theautomated testing in place.

Make a contribution every day

I recommend our organization take a longlook at the core competencies of our staffand aggressively promote the learning andgrowth perspective of current softwaretesting personnel. Development plans shouldhave well defined learning expectations thatboth managers and personnel recognize as anecessary objective for the success of theindividual in the software qualityorganization. These learning expectationsshould carry weight in decidingopportunities for advancement, ranking,compensation, and overall employee jobsatisfaction. We also must maintain astandard of expectations on new personnelthat they have the skills to improve ourorganization now and in providing new ideasto increase testing efficiency for futuretechnology of our products. Any concessionsmade toward qualifications must be made upthrough continuing education opportunities.Otherwise, it is easy to see why peoplebecome resist to using sophisticated toolsand what seem to be complex processes theyhave done without just fine.

If it does not contribute, it doesn’t leavethe garage

Automating or simplifying unnecessary workonly makes it more efficient--it does notmake it necessary. Of one of our most recentprinter driver products, only 23% of thedefects reported resulted in developmentcode changes. Of that 23% resulting in codechanges, 21% of those were found by thedocumented testing cases or “canned tests”that are currently used to develop automatedtest scripts. Therefore, only 5% ofdocumented or canned test cases result in

code changes or product improvements.

Figure 3. Defects resulting in code changes

Tools and automation should be deployedwhere testing methods have the highestrepetition and/or result in the most return oninvestment – in the form of bona-fide defectresolution that result in productimprovements. Moreover, areas of defectprevention in the design phase, codeinspection, and validation of feature changesare the best area candidates fororganizational and product improvements insoftware testing.

Believe that together we can do anything

I believe we can accomplish a great dealmore than we currently do. How do we getparticipation? Here are some suggestions:

• Get management commitment to theautomation and tooling process

• Ensure management availabilitythroughout the process not just thebeginning and end

• Nurture proper communication channels:be direct, open, and honest aboutprogress

• Let people be involved in decisions thataffect them

• Demonstrate mutual respect by listeningto each other; listening includesempathizing

Quality


• Get those involved in the process to talkto each other

• Invest in training; we need knowledgeworkers

• Eliminate class distinctions andadversarial relationships; not SWQA +R&D = R&D

Attention to these requirements will createadvantages for the whole organization byimproving communication, spreadingknowledge, increasing collaborative attitude,and improving motivation and morale.

Invent

You cannot discover new oceans unless youhave the courage to lose sight of the shore.

From a financial and internal businessperspective, software quality’s bestcontribution to the bottom line is in reducingcost. This means focusing our resources andautomation tool effort at the most optimalpoint in the value chain. Tools that willprovide verification between R&D deliveryof the software driver and preceding plannedverification in software quality. Fortunately,efforts are underway to introduce automationwhere it is most effective. Baselineacceptance, code inspection, driver file andlibrary verification tools are excellentopportunities to reinvent the way we test anddeliver our software products.

The learning and growth perspective willrequire more knowledge workers. Toaccomplish this requires a regularassessment to increase our internal corecompetencies and to bring the technicalexpertise on board that offers efficiencyopportunities to the organization.

References & Resources

1. Grady, Robert B., Successful SoftwareProcess Improvement. Hewlett PackardCompany. Prentice Hall, New Jersey, 1997.

2. Fiorino, Carly. The Journey: Rules of theGarage. Hewlett Packard Company. PaloAlto, CA. 1999.

3. Degrace, Peter and Leslie Hulet Stahl.Wicked Problems, Righteous Solutions : ACatalogue of Modern Software EngineeringParadigms. Yourdon Press ComputingSeries, 1999.

4. Visual Test6 Bible by Thomas R. ArnoldIi, Thomas R., II Arnold.

5. Bumbarger, Wm. Bruce. OPERATIONFUNCTION ANALYSIS. Presentation,WSU Engineering Management, 1999.

6. Test Automation automated applicationcompatibility, HP Deskjet feature testing;authors: Thong Nguyen, Mark Rossmiller,VCD, Hewlett Packard Companyhttp://vcsweb2/ca/swqa/toolproc/testauto/testauto/ppframe.htm.

7. Host-Client Virtual Lab automatednetworked resource and remote allocatedtesting; authors: Norm Peterson, DavidWagner, AIO/HID/SWTC, Hewlett PackardCompanyhttp://vcsweb2/ca/swqa/toolproc/host-client/hostclient/ppframe.htm.

8. SWATH automated application,network compatibility testing, HP Laserjetfeature testing; author: Beverly Takeuchi,PLD, Hewlett Packard Companyhttp://laserjet.boi.hp.com/2PDF/OneClickProjectPlan.doc.

9. VT in Depth. In Depth Productions, Inc.http://www.vtindepth.com.

10. NetHost Network Distribution TestScript for host client test execution: author:Richard J. Wartnerhttp://members.aol.com/rwvtest/index.htm.

UML


The HOORA (Hierarchical Object-OrientedAnalysis) method is a pragmatic approach tomodeling your systems. One of the uniquefeatures of HOORA is that it not onlyexplains the UML (Unified ModelingLanguage) notation. It defines a sequence ofsteps (a process) that should be followedwhile modeling. It adds process knowledgeand guidelines to the UML notation.

One of the innovative features introduced byHOORA, are so-called automatic diagrams.A static model diagram shows one particularclass, and all the other classes that arestatically related with it. A dynamic modeldiagram shows one class and all the otherclasses that dynamically interact with it. Apackage summary diagram shows onepackage and all the other packages that itdepends on, statically and dynamically.

Such diagrams are recommended forcomplexity management and project follow-up. They are generated from the modelinformation. These diagrams allow tocombine the advantages of providingfreedom, with the rigorousness required totackle today’s applications.

A major add-on to the HOORA method is atool going beyond UML. This tool is calledHAT, HOORA Analysis Tool. HAT showsthe feasibility and the need for a toolsupporting the process, the modelingactivity, and not just the (UML) notation.

In this paper, we explain the HOORAprocess, and these automatic diagrams. HATscreenshots are used.

1. HOORA process overview

A complete overview of the HOORAprocess can be found in the HOORA Process

Manual [1] on http://www.hoora.org. Thispaper only highlights a few issues.

The goal of the HOORA model, at each levelof abstraction, is to derive a suitablecollection of 'things', so that, when theycollaborate (and when they know eachother), they solve the set of problemsapplicable at the particular level ofabstraction. Because of that, HOORAstresses the dynamics of the system. We donot just want to identify ‘candidate’ objects.We want to identify these objects that arewell suited to structure the problem. And themain criterion we use for that, is to considerthe dynamics.

How do we achieve this? We start with usecases to identify fairly big functional blocks.Then, we try to find the set of classes that weneed to support every scenario identifiedwithin each use case. As a result, we obtaindynamic models showing which classeswork together with other classes. Thedynamic model shows the classescollaborating with each other ('who do youtalk with?'). The static model is itscounterpart, showing the static relationships(‘who do you know?'). As in the real world,object A can only talk with object Bprovided object A ‘knows’ object B, orobject A ‘knows’ one or more intermediaries‘knowing’ object B in their turn. So, staticand dynamic relationships are interlinked.

Considering the dynamics is done byinvestigating scenarios and identifyingclasses that are suitable to capture thatparticular behavior. Because scenarios areexamples, it is important to have amechanism to summarize the decisions takenduring scenario modeling. This is where thefirst automatic diagram type pops up, thedynamic model.

HOORA, The Method for Requirements Analysis

Johan Galle, [email protected], http://www.hoora.orgE2S, Technologiepark 5, B-9052 Zwijnaarde, Belgium

UML


2. Dynamic Model

2.1 HOORA responsibility-driven nature

There are primarily two schools of thoughtas to the best way to identify appropriateobjects, data-driven versus responsibility-driven. We take the position that a

responsibility-driven approach facilitates amore logical, more seamless, more object-oriented process [2].

While identifying classes, we emphasizedynamics. We only introduce classesbecause we need them at a given level ofabstraction. At that level, we identify

Hoora process overview

UML


responsibilities and we allocate them to thenewly introduced classes. At that stage, wemay also identify operations for the classes.These operations implement (part of) theresponsibilities [3].

The major reason for HOORA to beresponsibility-driven, is that we feel that thecore characteristic of object-orientation isNOT inheritance, relationships or real-worldequivalence (although these characteristicsare important). We feel that the corecharacteristic is that a model allowsdescribing our system, and this modelconsists of the collaboration of a minimal setof objects. This defines ‘object’. An object iswhatever part helps us describing the systemas a collaboration of parts.

Among other reasons, this emphasis is due tothe need to consider real-time requirementsearly in the project phase. So, we need a wayof working which is very suitable fordescribing (and annotating) behavior.

HOORA uses scenarios as a key driver to themodeling process. The scenarios will ensurethat we identify exactly this set of classesthat we need in order to capture the system'sbehavior. Scenarios are there to ensure thatwe identify the objects that are required atthe particular level of abstraction that we areworking on. Other guidelines (e.g. 'underlinethe nouns') do not provide such a guarantee.

Of course, HOORA does not reject data-driven approaches. In practice, bothapproaches are used in parallel, and data

objects are still important candidate objects.But they are not the only source. Theproblem with data-driven approaches, is thatyou need additional guidelines (i.e. considerthe dynamics) to determine which classes /associations NOT to consider at a given levelof abstraction.

2.2 Dynamic model diagram

The dynamic model diagram is centeredaround a particular class. For this class, weconsider all sequence diagrams (representingscenario’s) where this class occurs. We thengenerate arrows capturing the dynamiccontrol flow to and from this class.

Figure 1 shows all the classes thatCommunication Manager interacts (‘talks’)with. It shows that Communication Managertalks with Transaction Manager, MerchantFixed Terminal, Customer Mobile Deviceand vice versa. This diagram is generatedbased on a set of sequence diagrams. Classeshave been identified based onresponsibilities. So, this diagram must beunderstandable from the point of view ofthese responsibilities. Because theresponsibilities have been chosen in aparticular way, some classes do not appear inthe Communication manager’s dynamicmodel.

Solely by reasoning about theseresponsibilities, we may identify errors orincompleteness in this diagram. Such errorsneed to be corrected on the correspondingsequence diagrams. This is part of the

Me rch an t fixed te rm in al

(f rom e -b an k c l i en t)

Cu s to m e r m o b ile de vice

(from e -b an k c l i en t)Tra ns a ction Ma na g er

Co m m u n ica tio n Ma n ag er

Figure 1. Dynamic Model Diagram

UML


HOORA support for complexitymanagement.

3. Static Model

3.1 Static model diagram

The static model receives less emphasis inthe HOORA process. Still, the classes asidentified (mainly by investigating thedynamics), have static relationships amongeach other. These relationships arerepresented on class diagrams. In the sameway as the dynamic model, we generate astatic model diagram, centered around oneparticular class, based on all class diagramswhere this class occurs.

Figure 2 shows all the classes thatCommunication Manager is statically relatedwith (‘knows’). Static relationships alsocontain multiplicity information (1..n, n..nrelationships).

3.2 Comparing dynamic and static models

There is a validation check to be done whencomparing the static and dynamic model.

We are now at a stage where we believe that,at some level of detail, we have entered allthe model information required. The goal of

this step is that we investigate what the linksare between one class A and other classes B,both statically (A knows other classes B) anddynamically (A talks with other classes B).We will check whether these dynamic andstatic links are consistent with each other (itshould be possible for A to select theappropriate B’s it wants to talk with, basedon the B’s that it knows).

Consider the static and dynamic model ofCommunication Manager. We need to checkthat not only there are one direct or acombination of indirect static links for everydynamic link. We also need to checkwhether the Communication manager is ableto select the particular instance it is said totalk with according to the underlyingsequence diagrams.

The goal of this activity is to identify areaswhere the model is still incomplete. Staticand dynamic model information is thenadded, until both diagrams are consistentwith each other. It is a common practice toignore some aspects during a particularphase in analysis. Still, it is important tohave a model that is at least consistent in thisrespect. We are not allowed to ignore aparticular aspect in one part of the model,but still to use it in another part.

Merchant fixed term inal

(from e-bank cl ie nt)

+receive proof( mercha nt id, customer id, amou nt, pro of code)+request payment( amount, merch ant id , customer id)

+wi thdraw money( merchant i d, amount)

Sys tem

Customer mobile device

(from e-bank cli ent)

+conne cted()+contact e-bank serve r()+log in()+switch off()

+switch on()+get customer signa ture( transaction id , amount, transaction info )+load money( customer id, amoun t)

Com munication Manager

+load money( customer id, amoun t)

+send pro of( merchan t id, customer id, a moun t, proo f code )+request payment( amount, me rchant i d, custome r id)+wi thd raw money( merchant i d, amount)+get customer signa ture( transaction id , amount, transaction info)

*

*

1..*

Figure 2. Static Model Diagram

UML


4. Package Model

Levels of abstraction are a major means todeal with complexity. In HOORA, we usepackages to represent them. A packageconsists of model elements. A package canbe decomposed further into packages. Thestatic and dynamic models show classesbeing associated with each other. Theseclasses can belong to different packages. Thepackage summary diagram allows viewingthe dependencies between packages(summarizing the static and dynamicrelationships).

The goal of such a diagram (Figure 3) is tocheck whether the dependencies betweenlevels of abstraction make sense (taking intoaccount the intended purpose of eachpackage), and to suggest changes to it.Classes can be moved from one package toanother, packages can be grouped or split,packages can be decomposed, … We aim toachieve minimal dependencies betweenpackages. Cyclic dependencies imply theimpossibility to separate packages.

The hierarchy rules in HOORA are stricterthan in UML. HOORA defines two factorsthat determine the distance between twoelements. We calculate the number of levelsyou need to go up the hierarchy and thenumber of levels you need to go down thehierarchy. The modeler, by investigatingthese numbers, has quantitative data showingthe need to refactor.

5. Conclusion

This paper shows one way of using UMLand HOORA. HOORA allows somevariations, e.g. state modeling in early phaseof use case modeling is possible as well, themoment when requirements traceabilityrelations are decided upon,etc. Still, HOORAis a lot stricter than UML. A majoradvantage of the HOORA method is that wedo not have a ‘blind’ phase of classidentification. We only introduce the classesthat we need, according to the collaborationsrequired in the scenarios. This method isdriven by the dynamics. The dynamic modeldictates the static model. The motivation forintroducing automatically generateddiagrams showing the static model, dynamicmodel and package summary, is tocontribute to complexity management (andthus the need to guarantee that all the linksbetween model elements are shown).

6. References

1. HOORA Process Manual,http://www.hoora.org.

2. D. Demarco, “Analysis and designmethodology”, Thoughtworks Inc.,Summer 95.

3. R. Wirfs-Brock, “Adding to YourConceptual Toolkit: What's ImportantAbout Responsibility-Driven Design?”,ROAD, July-August, 1994.

RadioSystemStation

Power Instruments

Figure 3. Package Summary Diagram

Peoplewarel


If one wants to improve software quality,one ought to analyse not only the qualityof the software products but also theproduction process itself. The worksituation of software developers has beenanalysed in an empirical andinterdisciplinary study; its major resultsare reported here.

Some of the important results are thatcommunication is required to a highextent, that there are negative stresseffects in terms of burnout and that userparticipation has negative effects onproject and process quality. Moreover,software developers are not as interestedin the functionality of new tools as theirusability; new methods like objectoriented design may lead to increaseddemands to communicate with each other.

The work situation of software developersas a factor of software quality

If one wants to improve software quality,one should not only look at the result of theproduction process but also at the productionprocess itself. Traditionally the quality of theproduct has been the focus of attention.Recently, this view has been complementedby taking into consideration the processquality as well. Moreover, the productionprocess of software is not only looked at asan aspect of total quality management and asa factor of the product quality but also as anissue in its own right.

This viewpoint implies that the participantsof the process become the targets of researchin the area of software engineering - thesoftware developers, the project leaders, andpartly those users who participate in theproduction process. The work situationduring the production process is of majorimportance for process and product quality.

This was the starting point of the IPAS-project (IPAS is a German acronym andstands for "interdisciplinary project on thework situation software development). In all29 software development projects from 19German and Swiss firms were studied. Themeans size of the projects was 10 members;the total sample comprised 200 softwaredevelopers. All of them filled out aquestionnaire and were interviewed forseveral hours.

The software produced by these projectscovered a broad application domain, forexample administration, banks, insurances.Military or academic projects were notincluded because they do not produce for afree market

Our framework for research

The basic framework for ourinterdisciplinary project is presented in Table1. The idea was to cross the differentperspectives of the disciplines workingtogether in this project (computer science,work psychology and industrial sociology).Our original framework was modified forthis presentation because we excluded thesociological part from this article - a columnon project management.

The rows of this framework consist ofcategories from work psychology that havebeen shown to be important to describe worksituations, the columns are related tosoftware-technical criteria that are importantin software engineering. These twodimensions lead to 15 problem areas thatwill be used to describe the results of ourstudy. This article presents an overview ofthe results.

The Work Situation in Software Development

Michael Frese, University of Giessen, Otto Behagel Str. 10F, 6300 Giessen, GermanyWolfgang Hesse, University of Marburg, Hans Meerwein Str., 3550 Marburg, Germany

Peoplewarel


Results and discussion

1. Coping with complexity

The necessity to cope with the complexity ofthe software development process is animportant problem for software developersand their managers. Most likely, complexityof this process will increase in the future.While new programs and tools rendercomfortable metaphors or graphical userinterfaces, the user of such programs andtools for the design process is confrontedwith a rapidly growing number of products,difficult decisions, incompatible interfaces,etc.

There are ever shorter innovation andproduction cycles. Once one has to know onetool, there is already a new version or a newproduct on the market. This leads to new

(learning) problems if these new tools areused. Thus, one result of our research is thatusability and easy learnability are moreimportant criteria for software designers thanbetter functionality (cf. 1.3 below).

There are two strategies to cope withcomplexity: one can either reducecomplexity or learn to live with it. In thesoftware projects studied, we found bothstrategies. On one hand, there was problemoriented modelling, a coherent procedurewith one method (e.g., with an objectoriented approach) or the use of CASE tools- all of these are attempts to reducecomplexity. On the other hand,psychological research showed thatcomplexity reduced stress and increasedmotivation in the software developers (cf.l.2)

Software technicalcategories /Psychologicalcategories

Relationship with user Development Methods and tools

Coping withcomplexity

Familiarising oneselfwith user applicationsareas

Appraisal of one's jobcomplexity

Use of tools anddevelopers'requests

Cooperation andcommunication

Difficulties with userparticipation

Communication asmajor part ofdevelopment

Influence of newmethods oncommunication

Job discretion Factors influencingjob discretion:standards, guidelines,change requests,flexibility

Correlation betweenjob discretion andburnout

Influence of jobdiscretion onacceptance of tools

Qualification Qualification ofsoftware developers inthe application area

What characteriseexcellent softwaredevelopers

Learning to usemethods and tools

Stressors Developer's stressthrough userparticipation

Stress and burnout:software developmentas challenge or stressfactor

Effects ofchanging methodsand tools

Framework for research

Peoplewarel


1.1 Qualification of software developers inthe application area

One important aspect of complexity in thework of software developers is the necessityto get to know new application areas and tofind solutions that are adapted to the specificuser needs.

Current tendencies aim at a more highlyintegrated development process ranging fromfunctional design to technical design andimplementation. The pressure to producesoftware more efficiently and with fewercosts will most probably increase in thefuture. This implies that reusability will be ahighly important topic. Reusability does notonly refer to software modules but also tofunction design, user models, and even toanalysis documents. This leads to newdemands on software developers to deal withcomplexities not only in their own technicalarea but also in the areas of functionaldesign, user modelling and analysis ofparticular application areas

1.2 Appraisal of job complexity

Since software development is a highlycomplex undertaking, one often draws theconclusion that its complexity should bereduced. This is probably true in certaincases. However, our results suggest that thisshould not be a general strategy. Workcomplexity is often something very positivefor the software developers. For example,software designers with highly complex jobsshow less burnout. They are also prouder oftheir work and their team is prouder thansoftware designers with jobs of littlecomplexity. Thus, a reduction of complexitymay have counterproductive effects.

It is useful to differentiate betweencomplexity of tools on one hand andcomplexity of tasks and activities on theother hand. Tool complexity is very oftenseen as something negative, task and jobcomplexity are positive aspects of the workplace. Tools should, therefore, not reduce theoverall complexity of the job; they shouldonly contribute to a reduction of routine

activities or lead to a reduction of "detours".Examples for detours are activities that arenot directly linked to the work tasks, e.g.mounting a new tool. This allows thesoftware developers to concentrate theintellectual challenges in work.

1.3 Use of tools and developers' requests

Various parts of our research were concernedwith tool use: which tools were used fordoing which tasks, what did the softwaredevelopers want from tools, effects of tools,etc.. The results show that there is a largediscrepancy between theory and practice.While the CASE market offers tools withsophisticated graphical user interfaces, manysoftware developers still have to deal withVSAM, CICS or line editors.

A comparison between what softwaredevelopers want and what they have showsthat typical tasks like coding, editing, anddebugging are adequately supported by tools.On the other hand, the software developerswould like to have better tools in the areas ofdocumentation, graphical support (especiallyanalysis and specification tools), testsupport, generators and project management.

In general, the developers report to be wellsupported by the tools available. However,the importance of usability was higher thanthe functionality of the tools. Forty per centsaw "easy to learn" and "easy to use" as themost important characteristics of a tool.Functional aspects like "without errors, "welladjusted to task", "easy to combine withother tools" or "performance" were onlymentioned by 5% to 8% as important. Thewants and requests of software developersare relatively modest. Only those designerswho had got to know other tools in otherprojects or in their studies presentedsuggestions for improvement. The more theyhave used tools the more they thought thattheir tools are flexible enough to changethem.

Tools may be important but they are clearlynot more important than motivational factorsand social support by co-workers and

Peoplewarel


supervisors.

2. Communication and cooperation

2.1 Difficulties with user participation

Presently, there are many arguments for ahigher degree of user participation in thedesign process, e.g., through prototypingOften, only the advantages of such anapproach are presented and potentialdisadvantages are not discussed. We alsoexpected that user participation would leadto positive consequences. However, the dataprove the opposite.

We found that the efficiency of the softwareprocess, the changeability of the product, thequality of the product, time efficiency, etc.were all reduced in projects with high userparticipation. In other words, the users seemto disturb the development process. Thereare more change requests in the projects withuser participation. The innovativeness ofthese projects is also seen to be lower by thesoftware developers. These results are notthe opinion of just a few softwaredevelopers. They are supported by themajority of the software designers in theprojects with high user participation.

Our results do not mean, however, that userparticipation is harmful per se. But it doesshow that there may be more dangers anddifficulties in user participation than what isknown at this point. User participation canlead to frictions, to additional tasks anddemands. These problems have to beinvestigated and adequate strategies have tobe developed to deal with them. At the veryleast, one has to take into account that theseproblems can appear with user participationand rapid prototyping. In order to cope withthese matters a certain amount of time andbudget should be allocated when userparticipation is planned.

In addition, software developers' orientationshave to be changed. As long as userparticipation is not defined to be a centraltask for software developers, userparticipation will just lead to disagreeable

add-on demands. Only after a neworientation is introduced can softwaredesigners look positively at change requestsby the users.

2.2 Communication as a major part ofsoftware development

It is now generally known that the solitaryprogrammer is a rather exceptional case.Nevertheless, many trainings and universitycurricula do not sufficiently take intoconsideration the fact that programmers haveto work in teams (at least not in Germany).Interestingly, programming is only a verysmall part of software development (about10% of the time) while all the socialactivities, e.g. exchange of information,consulting, team discussions, etc., make up amuch larger percentage of the developers'work day (about 40% of the work tasks is insome way socially oriented).

Can we conclude that software developmentis a social profession? This is certainly notthe case. But the social component is ofutmost importance not only for team leadersbut also for the team members. Mostdevelopers' tasks have a social component,e.g. specification, design, testing, systemsupport, technical support, review sessions,coordination, adjustment of interfaces, etc.

All this requires much social competencies.The developers have to react flexibly whendifferent communication requirementsappear. These requirements may mean thatone has to moderate a group, to structureideas and to support other people or tocriticise them. Conflicts have to be copedwith and different communication strategieshave to be followed in discussions,depending upon the other's knowledge andprerequisites. Not all of these competenciesare acquired through normal academictraining courses.

It is of particular importance not to present ageneral sort of social competence trainingbut to offer specific courses integratingtechnical skills in the field of computerscience with communication skills.

Peoplewarel


2.3 The influence of new methods oncommunication

New methods and tools may considerablyaffect cooperation and communication in ateam. For example, Parnas principle ofinformation hiding or the data abstractionmethod may lead to a bureaucratisation insoftware development and a reduction ofcommunication. In contrast, a change to anobject oriented method may lead to theopposite result.

The following remarks are not based onstatistical analyses because there were oddlya few projects in our sample that used objectoriented design methods. However, it wasour impression, that there is particularlystrong need for communication andcooperation in such projects. This isprobably due to different software structures.While the data abstraction methodexclusively connects different modulesthrough import/export relationships, this isnot the case in object oriented design whereinheritance relationships play an equallyimportant role.

The data abstraction method allows a designby developers who work relativelyindependently from each other. Whiledivision of labour is necessary in objectoriented design as well, the variousdevelopers have to communicate a lot, e.g.on features inherited from each other.Usually, a purely formalised communication,e.g. the exchange of formal specifications, isnot sufficient because intensive and frequenttalks about the implications of variousdecisions in the design process are necessaryin object oriented design.

3. Job discretion

Job discretion implies that a person haspossibilities to change things at a workplace. For example, if job procedures are notrigidly prescribed, if software developershave some influence on what kinds of toolsare used, if they can decide how they aredoing their job and if they can decide at whattime they do certain tasks, their job

discretion is high.

There are essentially two managementapproaches to job discretion. One impliesthat job discretion will only lead to areduction of productivity and thatmanagement should organise work in such away that there is little job discretion. This isthe Tayloristic approach. The other approachis based on modern work psychology and itimplies that the people at work will only befully motivated to work if they can use theirabilities and skills in their work. This impliesthat job discretion should be high.

3.1 Correlates of job discretion

Different software designers have differentdegrees of job discretion. One issue is theflexibility and the changeability of theprograms. If there is a high degree ofchangeability of the program, one can, e.g.,deal more quickly and adequately withchange requests.

In this connection, it is of interest to knowwho is most often responsible for changes -is it the contractor of is it the softwaredevelopers themselves. If it is the former,there is little scope of action (and hence littlejob discretion), if it is the software designers,change requests are themselves the result ofa high degree of job discretion. Our resultswere surprising: apparently, a large part ofthe change requests stems from the softwaredevelopers. About 40% of the changerequests were due to internal sources.

Management or other developers in the teamwere responsible for 14% of the changerequests. However, the most important causeof change requests was the development ofnew solutions by the individual softwaredeveloper him- or herself (25%). Jobdiscretion is used here to develop andimprove one's work and one's solutions.

No wonder that job discretion is seen as apositive factor in work psychology thatcontributes to higher product and processquality.

Peoplewarel


3.2 Correlation between job discretionand burnout

Job discretion is not only an important factorinfluencing performance, it is also related toburnout. The higher the job discretion is, thelower is the burnout for software developers.If there is a high degree of rigid division oflabour, this usually reduces job discretion.Moreover, there is a relationship between jobdiscretion and work complexity. If there islittle complexity, there is also little that canbe decided in work.

Since burnout probably has an impact onproductivity - be it through absenteeism orthrough little motivation for work - a highdegree of job discretion is an importantproductive factor.

3.3 The influence of job discretion onacceptance of tools

One of the issues in our research project wasto study the acceptance of tools by thesoftware developers. In general, the toolswere quite well received. The softwaredevelopers think most highly of simulationtools, low level editors and projectmanagement tools. The tools that areperceived to be less good are projectlibraries. This may be due to the fact thatthese tools can only be used adequately witha high degree of experience.

There was also an interesting interaction ofjob discretion and the perceived usefulnessof the tool. Perceived usefulness of the toolwas a general perception by the softwaredevelopers of how good the tool wasadjusted to the tasks and how usable it was.In general, usefulness of the tool alsoimplied that stress effects were minimised.This fact was mitigated by job discretion,however. Under conditions of high jobdiscretion, usefulness of the tool was relatedto reduced stress effects. If the job discretionwas low, there was little relationship tostress effects.

Thus, the usefulness of the tool seems tohave a positive impact only if the general

conditions in the job allow the person a lot offreedom. If this freedom is low, theusefulness of the tool does not help thesoftware developers to cope with the stresseffects.

4. Qualification

4.1. Qualification of software developersin the application area

Software developers have to learn newthings in every project. In most cases, thishas to be done without formal training. Ofparticular importance is the knowledgeacquired about the area of application. Togain this knowledge the software developershould have the following competencies:• to learn a different conceptual system in

which he or she has to be the translator,• to be able to pose clear and sensitive

questions,• to separate plausible from implausible

hypotheses and essential from unessentialinformation,

• to discover structures which may be soroutinised in the area of application thatpeople will not be able to verbalise themany more,

• to distance oneself from one's ownegocentrism and to put oneself into theshoes of somebody else; this implies thatone has to be able to distance oneselffrom one's own discipline and to take theperspective of what the users have beensaying,

• to develop criteria to find out whether onehas been understood or whether or notone has understood what the users havebeen saying,

• to make abstract concepts concrete, i.e. tofind examples for abstract examples in thearea of application.

Moreover, the knowledge in the area ofapplication has to be reconciled with therequirements of the project. This means, forexample, that the software developer has toanalyse requirements of the application areaand whether they can be realised technically.

Peoplewarel


Thus, software developers need to showconsiderable qualifications. At the sametime, they rarely receive formal training formany of these skills. This implies that theyhave to learn strategies to learn bythemselves. These strategies may be one ofthe most important factors of softwaredevelopers' success in their job - again it issomething that has received little attention.

4.2 What characterises excellent softwaredevelopers?

In order to find out which qualifications andcompetencies software developers need to dotheir job well, we have to identify excellentsoftware developers in the projects studied.

This was done with a procedure used quiteoften in work psychology: peer nominations.All people were asked to identify the bestsoftware developer in their team. Thosepeople identified by two of their colleagueswere taken to be excellent softwaredevelopers. Moreover, the softwaredevelopers were asked what characteristicsmake them excellent. What characterisesthese excellent in comparison to averagesoftware developers?

The three most important characteristicswere: technical competence, ability to workin team, work style. Technical competenceimplies a high degree of professionalexperiences, an ability to recognise and solveproblems quickly and a high degree ofknowledge about the particular project (e.g.knowledge to whom one can address whichquestions, which information can be receivedwhere, the structure of the system to bedeveloped, knowledge of previous errors andwrong decisions and why are things done theway they are done).

The ability to work in teams implies that it isfun to work with this particular person, thatthis person can communicate effectively andthat he or she is willing to work togetherwith others. A good work style ischaracterised by using systematic proceduresat work, by being independent in work andby being team oriented.

It is interesting to note that the excellentsoftware developers did not have a higherjob tenure but they had more intensiveexperiences. They had participated in moreprojects and they knew more programminglanguages. They are asked to participate inmore meetings and they are asked morequestions by the other software developers.Thus they have to communicate more (andare apparently better able to communicate)than the average software developers.

Again, we find the importance ofcommunication skills. The excellentsoftware developers are the natural leaders oftheir teams. Sometimes they are just theinformal leaders, but they are also oftenmade to become supervisors in due time.

4.3 Learning to use methods and tools

The qualification of software developers forthe use of methods and tools is an importantprerequisite to do the job well. Our studiesshowed that learning while doing the jobprevails rather than formal and systematictraining. In about 40% of the cases, the useof methods and tools was learned on the job;the software developers received someformal training in only about 16% of thecases.

However, there are clear differences betweenthe academically trained computer scientistsand the non-academic software developers.

The computer scientists had someknowledge on methods and tools in 41% ofthe cases; this was the case for only 26% ofthe software developers who were notacademically trained.

This reinforces the importance of self-discovery learning in software development.Those who have learned how to learn will beat an advantage. To support this "learning tolearn", one would have to teach learningtechniques, self-organisation, etc..

Peoplewarel


5. Stressors

5.1 Developers' stress through userparticipation

There is a clear correlation between userparticipation in a project and the number andintensity of stressors impinging on thesoftware developers.

In projects with user participation, there aremore stress factors than in projects withoutuser participation. At first sight, this resultmay be surprising. However, it makes sensethat software developers, who have to workunder high time pressure anyhow, will havemore stress if their normal work isinterrupted and additional user requirementshave to be met on top of their normal work.Again, this shows that user participation isnot as unproblematic as it seems.

5.2 Stress and burnout: softwaredevelopment as challenge or stress factor

From the standpoint of work psychology, thework places of software developers are ingeneral quite good. The software developershave challenging tasks, usually of highsignificance and importance. These tasks arecomplex enough and it is possible to interactsocially in the job. Nevertheless, there arestress factors which may contribute topsychological impairment even in this sectorthat may seem to be privileged: too muchwork, frequent interruption, unclear or lateinformation, career pressure and pressure tooutperform.

Many software developers and projectleaders checked in the questionnaire thatthey have to deal with an enormousworkload and that their work is interruptedagain and again. They have to work overtimefrequently and cannot compensate this withadditional leisure time.

Most overtime has to be done by projectleaders and user delegates in the softwareteam. The latter have a particular highworkload. At the same time, they have towork both for the project and in their normal

duties. An added burden is the pressure fromtheir constituency (the users to be) and thepressure to be part of the software team.

The burnout syndrome is of particularimportance for software developers. Burnoutimplies that a person feels exhausted and isnot able to identify with his or her work anymore and that he or she has the feeling, thatone's ideal, ideas and one's potential cannotbe realised.

One has to assume that burnout maycontribute to loss of motivation at work, topsychosomatic problems and to a higherdegree of absenteeism. Thus, burnoutprobably does not only reduce motivationbut also productivity. A finding supportingthis claim is that excellent softwaredevelopers show less burnout than theaverage ones.

Burnout has a clear-cut relationship to thestressors of software development: thehigher the stress factors at work, the higherthe burnout. Moreover, those people withhigher job discretion and higher complexityof work also show less burnout. Similareffects appear with group climate: teams thatare democratic, more open for critique andless authoritarian also show less burnoutamong their members.

5.3 Effects of changing methods and tools

The high degree of innovation in the area ofmethods and tools can be one stress factor aswell. While tools are supposed to support thework of software developers, they oftenproduce the opposite effect. This may be themajor reason why software developerattaches the most importance to the criterion"learnability" when they judge tools (c.f.1.3).

6. Conclusion

Our project has dealt with an area that hasnot been frequently or systematicallyresearched. From a psychologicalperspective some surprising findings werethe high degree of communication

Peoplewarel


requirements in the job of softwaredevelopers, the importance of burnout as themajor effect of stress at work, the cohesivepicture of excellent software developers andthe negative impact of user participation onproduct and process quality. Some of theseresults have been reported by other as well.The most troubling finding is certainly thatuser participation may have a negative effectin the design process.

From the standpoint of computer science,important findings were that softwaredevelopers are less interested in thefunctionality of new tools but rather in howeasily they can be learned. Moreover, thewell-known gap between softwaredevelopers and users appeared again in ourresearch.

New methods, e.g. object oriented design,may lead to a higher necessity tocommunicate well. The question of thecomplexity of the tools versus thecomplexity of the job is certainly somethingto be considered in the application anddevelopment of new tools.

Applying new software engineeringtechniques always implies that the worksituation of software developers is changedfor the better or for worse. Thus, the changeof the jobs of the software developers is animportant factor to be considered in the areaof software engineering.

This article was originally published in theJuly 1995 paper edition of Methods & Tools

Classified Advertisement

Add Value by implementing Aardvark Bug Tracking SoftwareConsolidate all of your project information in one place andallow your development and testing process to thrive! Aardvarkbug and defect Tracking software is the leading browser basedbug tracking system, find out how you can concentrate on thetasks that add value at www.red-gate.com/bug_tracking.htm orcontact [email protected] for more information.

Advertising for a new Web development tool? Looking to recruitsoftware developers? Promoting a conference or a book?Organising software development related training? This space iswaiting for you at the price of US $ 20 each line. Reach morethan 15'000 web-savvy software developers and project managersworldwide with a classified advertisement in Methods & Tools.Without counting those that download the issue without beingregistered! To advertise in this section or to place a page orWeb site ad simply send an e-mail to [email protected]

METHODS & TOOLS is published by Martinig & Associates, Rue des Marronniers 25,CH-1800 Vevey, Switzerland Tel. +41 21 922 13 00 Fax +41 21 921 23 53 www.martinig.chEditor: Franco Martinig; e-mail [email protected] ISSN 1023-4918Free subscription: www.martinig.ch/mt/index.htmlThe content of this publication cannot be reproduced without prior written consent of the publisherCopyright 2001, Martinig & Associates

http://www.red-gate.com/bug_tracking.htm

e methods & toolsquality methods & tools * summer 2001 * page 3 familiar with its use,...

Documents