building software for experimental workyccheng/papers/building... · 2005/7/15 6 mapping problem...

2005/7/15 1

Building Object-Oriented Software for Experimental Work

台北科技大學

資訊工程系

鄭有進副教授

http://www.ntut.edu.tw/~yccheng

2005/7/15 2

Outline

Experimental work and problem solvingCharacterizing software development for experimental work

requirements, expectations, and realitiesA software-centric view of experimental work

The best practicesA case study

RANSAC Curve detectionSyncFree

Conclusion

2005/7/15 3

1. Experimental work and problem solving

2005/7/15 4

Experimental work, problem solving, and software development

Experimental work involves advanced problem solvingIn many research areas, problem solving is carried out through computer programs,

E.g., signal processing, communication, computer vision, multimedia and etc., A comprehensive system can be complex

2005/7/15 5

Phases for problem solving [Skowronski 2004]

PreparationGather information about problemCarry out experiments and test possible solutions

IncubationLet the problem sit and work subconsciously

IlluminationThe moment when the solution’s central idea appears in a flash of light

VerificationFlash of insight expands into a solution, which is then tested against reality

2005/7/15 6

Mapping problem solving phases to software development (I)

Preparation research; building software A software/framework built based on known literature.The software/framework must be robust/efficient enough for implementing well-defined tests of ideas.Should allow you to ask and answer “what if”The purpose is to generate observations about the known methods (esp. those with rooms for improvement)

2005/7/15 7

Mapping problem solving phases to software development (II)

Incubation pre-production use of software

Not much parallel here, but the software should readily help exploration

2005/7/15 8

Mapping problem solving phases to software development (III)

Illumination implementation and test of the central idea

The “eureka!” idea should be implemented and tested for their correctness.

2005/7/15 9

Mapping problem solving phases to software development (IV)

Verification software for the proposed; Comparison with existing methods

Well-defined targets of comparison to claim improvementFairness in input and comparison category

2005/7/15 10

Software work in research phases

verification

illumination

incubation

preparation

Amount of software work

phase of Research

2005/7/15 11

2. Characterizing software development for experimental work

2005/7/15 12

Expectations/Requirements

In addition to being functionally correct, software for experimental work should

Productivity: Deliver results good enough for publishing, usually before its completion. Extensibility: Support exploration of new ideas, it must be easy to add (and remove) functionalities.Be explorative and observable: Support online analysis and drill-downsMaintainability and reusability: Accumulate well without constantly rebuilding

2005/7/15 13

Realities (I)

Research maybe well-planned, but software development for research work is usually under-plannedSoftware work is usually delivered in a water-fall fashion

Problems are discovered too late and remedies can be costly

Frequent re-implementation

2005/7/15 14

Realities (II)

Development period is extendedDevelopment team is unstable

People moving in and out (esp. during summer).New staff with unpredictable skill level

Programs are brittlePrograms are insufficiently (really, barely) tested

2005/7/15 15

3. A software-centric view of experimental work

2005/7/15 16

A software-centric view of experimental work

process

domain skillssoftware

toolspeople

2005/7/15 17

A number of best practices

1. Sizing the gap2. Object-orientation3. Adopt a process4. Frameworks and design patterns5. Exceptions

2005/7/15 18

1. Sizing the gap: bridging features to resources [OOMD 2005]

What are the desired features?What are the available resources?

Libraries, framework, environmentsCan the desired features be constructed directly with the available resources?

If so, just do itIf not, add a layer of abstraction recur

2005/7/15 19

How large is the gap?

Narrowly viewed, the gap is smallRecall that Fortran and C used to be (and still are) good enough for building programs for experimental work

But the gap will be larger if you have more expectations:

The “ilities”: Observability, controllability, extensibility, maintainability, reusability and productivityThis is especially so if we treat experimental work as a comprehensive software rather than as a program

2005/7/15 20

Gap bridging by building around an existing tool (I)

Center your system around a software tool (e.g., MatLab, Khoros, etc.)

Pros:Good support at API, class library, framework and component levelsPowerful UI metaphors for modeling your experimentsProductivity increases once mastered

2005/7/15 21

Gap bridging by building around an existing tool (II)

Cons:Architectural impact: tool put constraints on how a function is implementedcustomization can be difficultTool use (UI and API) involves a learning curve and can be non-trivialPerformance problems.

2005/7/15 22

2. Object-orientation

Procedure-orientation:We view work to be done mainly as a procedureImplicit in the process are entities upon which the procedure is carries out.

Entities captured in data structuresProcedures specified by algorithms

2005/7/15 23

Adage:

Algorithms + Data Structures = Programs

Niklaus Wirth

2005/7/15 24

Object-orientation

We still view the work to be done as a procedure.But participants (entities) in the procedure are made explicit.The participants collaborate to get the procedure done.Object-orientation captures the static structure of the application domain

More resilient to changeSuitable for evolutionary work

2005/7/15 25

Object-orientation

• Encapsulation, messages and methods, inheritance, polymorphism.Reification: promote to object where you expect variation

Example: design patterns strategy and state

2005/7/15 26

Form and Function (I)

Software has two aspects: form and function

UseCase

Architecture

function

1..*

form

1

BaselineArchitecture SupportArchitecture

1 *

2005/7/15 27

Form and Function (II)

In mainstream OO processes (e.g., RUP and XP), function and form grow together.

Architecture baseline takes shape early as primary use cases are built.

In the typical experimental work software development, function gets most of the attention.

Since the perceived gap is usually smallForm is non-existing, or is just whatever the outcome of the aggregation of functionalities.

2005/7/15 28

The adage revised for object-orientation:

Objects + Responsibilities + Collaborations = programs

[Algorithms + Data Structures = responsibilities through Operations

We still do it, but at a lower-level and delayed in the process]

2005/7/15 29

3. Adopt a software development process

Regardless of the available resources, if the gap is large, your task will be non-trivial.

you need a software development process to guide your work and to address requirements, expectations and realities simultaneouslyTo help new staff in their transition phase in joining the development teamTo systematically accumulate the artifacts

But be light on the artifacts

2005/7/15 30

Process centric view

ContinuousDelivery, on Budget and schedule

people

skills

peopleprocess

2005/7/15 31

Which process?Agile methods:XPCrystalJAD…

RUPOMTFusion…

Heavy weightLight weightMany artifactsCold communicationLarge team

Few artifactsHot communicationSmall team

2005/7/15 32

Proven process elements (best practices in processes)

Use cases (requirement capture)Increments (staging strategy)Iterations (rework scheduling strategy)Patterns (design and implementation)Unit tests (implementation)Continual integration Review (all stages)

2005/7/15 33

A simple process and its workflow

1. Write use cases2. For each use case

3. Extract domain model4. Evolve into design model5. Implement and do unit tests6. Integrate7. Test use case8. Review

2005/7/15 34

Workflow elements, roles and responsibilities

Researcher/staffUse case tests

Researcher/staffReview

StaffIntegration and tests

StaffImplementation and unit tests

StaffDesign modeling

Researcher/staffDomain modeling

ResearcherWriting use cases

2005/7/15 35

3.1 Use case modeling

Use case: a sequence of steps describing a user-system interaction in which a well-defined service is tendered.Use case captures functional requirement for a userUse cases are high-level, domain-oriented descriptions written in a way understood by project stakeholders

2005/7/15 36

Use case modeling (I)

As a starting point in system modeling.All the other artifacts are derived from the use cases.

As a guide to testing the system built.The system built must reflect the process presented in a use case.Useful in function/acceptance tests

2005/7/15 37

Use case modeling (I)

Write just enough detail to keep the use case interesting

Too much detail reverts the development to a structured process

You don’t need to have all use cases in place to start building system.

However, the first use case should not be a trivial use case

2005/7/15 38

JCIS use case

2005/7/15 39

3.2 Increments and iterations (I)

A use case is roughly an increment: it is not the whole thing, but it delivers functionality by coherent chunks.Several iterations may be executed on an increment

To remove bugs, add functionality, and improve quality

2005/7/15 40

Increments and iterations (II)

Aim for a working system (at least covering the present use case) at the end of an increment.

Early feedback to users can guide late-stage development

Iterations should be time-boxed: e.g., 2-4 weeks on small projects.

2005/7/15 41

Increments and iterations (III)

Incremental strategy ensures that transition from domain model to design model is feasible

Increment delivers functionality in coherent chunksFirst increment should shape the architecture baselineLater increments built on top on early increments

2005/7/15 42

Iteration vs. waterfall (I)

requirement

Effort/result

AnalysisDesign

Implementation

IntegrationTesting

time

2005/7/15 43

Iteration vs. waterfall (II)

R A D I TIT

R A D I TIT

R A D I TIT

Effort/result

…

… timeIteration 2 Iteration nIteration 1

2005/7/15 44

3.3 Domain Modeling (I)

Make domain model explicitExtract the static conceptual domain structure.

concentrate on the conceptsfind out how concepts are related

Capturing the static structure is one of the reason why OO models are more reusable

Static structure is less likely to change radically, as compared to functionalities.

2005/7/15 45

Domain Modeling (II)

Use case-based method for domain modeling:

1. Scan for nouns in the use cases for concepts2. Scan for stative verbs in the use case for

associations On the other hand, use whatever you feel most comfortable (CRC, semantic nets, conceptual diagram, header files, etc.) to get domain modeling going

2005/7/15 46

Domain Modeling (III)

The concepts should have a consistent medium-to-coarse granularity level

Fine grained concepts bug you down (too much detail)

Avoid application classes, or classes due to realization platform

E.g., GUI classes and database

2005/7/15 47

3.4 Design modeling (I)

Models the domain as a set of software classes.

A design model is also a valid domain model, but it is both domain- and application-dependent, which may be not easy to obtain off-hand.

Coarse-grained concepts are usually realized by a set of cooperating classes

Each coarse-grained concept becomes a micro-controller delegating work to its internal classesKnowledge of design patterns and architecture styles are extremely helpful

2005/7/15 48

Design modeling (II)

Created by finding what a class must do to realize the operations being requested on the conceptual entities.

Responsibilities that are implied in the conceptual model must be made explicit in the design modelDiscover the responsibilities in the order of their occurrence

Responsibility assignment is the most important activity in OO design

2005/7/15 49

3.5 Implementation, unit tests and Continuous integration

Write tests for every operation of an object (more detail on unit tests later)

Logical unit tests (LUT)Integration unit tests (IUT)Functional unit tests (FUT)

Write characterization unit test (CUT) for unfamiliar APIsTests should be accumulative and automatic

Use a unit test framework such as CppUnit or JUnit

2005/7/15 50

Testing can be an embarrassment-saver

“Requirements for quality [of research and experimental software] may be quite low, and the process may be a little better than debugging and a few regression tests. Nonetheless, the risks of embarrassments from failed public demos and withdrawn papers suggest a greater investment.”

Stuart FeldmanQueue, Feb 2005

2005/7/15 51

3.6 Review

Peer reviews of requirements, analysis, design, and codeProcess observation and improvement

Development speedLearning “what you don’t know you don’t know”

Holes in your layer to bridge the gapArtifacts accumulation and synchronization

Use cases, domain model, design model, key interaction, tests

2005/7/15 52

Why design at all

Design aims to provide a solution for the functional aspectEqually important, if not more, it provides a solution for non-functional aspects.

The “-ilities”

2005/7/15 53

4. Design Patterns

"Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice."

Christopher AlexanderThe Timeless Way

2005/7/15 54

Four most essential elements of a pattern [GoF]

Pattern namea handle used to describe a design problem, its solutions, and consequences in a word or two.

Problem describes situations to apply the patternSolution describes the elements that make up the design, their relationships, responsibilities, and collaborations.Consequences are the results and trade-offs of applying the pattern

critical for evaluating design alternatives and for understanding the costs and benefits of applying the pattern

2005/7/15 55

Example use of patterns

Reification of behaviors for extensibility and variation examples:

Strategy [algorithm as object]State [state as object]Iterator [pointer as object]

2005/7/15 56

Strategy pattern

before

2005/7/15 57

Strategy pattern

after

2005/7/15 58

Pattern Space [GoF]

2005/7/15 59

Pattern purposes

Creational patternsthe process of object creation

Structural patternsthe composition of classes or objects

Behavioral patternsthe ways in which classes or objects interact and distribute responsibility

2005/7/15 60

Pattern Scope

ClassClass patterns deal with relationships between classes and their subclassesThese relationships are established through inheritance, so they are static—fixed at compile-time.

ObjectObject patterns deal with object relationships, which can be changed at run-time and are more dynamic.

2005/7/15 61

Frameworks [GoF, ch 1]

A framework is a set of cooperating classes that make up a reusable design for a specific class of software.You customize a framework to a particular application by creating

application-specific subclasses of abstract classes from the framework.

2005/7/15 62

Framework and architecture

The framework dictates the architectureof your application. It will define

the overall structure, its partitioning into classes and objects, the key responsibilities thereof, how the classes and objects collaborate, and the thread of control.

2005/7/15 63

Framework and design

A framework predefines these design parameters so that you, the application designer/implementer, can concentrate on the specifics of your application. The framework captures the design decisions that are common to its application domain. Frameworks thus emphasize design reuseover code reuse,

though a framework will usually include concrete subclasses you can put to work immediately.

2005/7/15 64

Inversion of control

Reuse on this level leads to an inversion of control between the application and the software on which it's based.

When you use a toolkit (or a conventional subroutine library for that matter), you write the main body of the application and call the code you want to reuse. When you use a framework, you reuse the main body and write the code it calls. (The Hollywood principle.)

2005/7/15 65

Consequences

Not only can you build applications faster as a result, but the applications have similar structures. They are easier to maintain, and they seem more consistent to their users. On the other hand, you lose some creative freedom, since many design decisions have been made for you.

2005/7/15 66

If applications are hard to design, and toolkits are harder, then frameworks are hardest of all.

A framework designer gambles that one architecture will work for all applications in the domain. Any substantive change to the framework's design would reduce its benefits considerably, since the framework's main contribution to an application is the architecture it defines. Therefore it's imperative to design the framework to be as flexible and extensible as possible.

2005/7/15 67

Framework and design patterns

A framework that addresses them using design patterns is far more likely to achieve high levels of design and code reuse than one that doesn't. Mature frameworks usually incorporate several design patterns. The patterns help make the framework's architecture suitable to many different applications without redesign.

2005/7/15 68

Framework examples

Graphical editors for different domains like artistic drawing, music composition, and mechanical CAD [VL90, Joh92]. Compilers for different programming languages and target machines [JML92]. Financial modeling applications [BE93]Document-View framework

2005/7/15 69

Patterns and Frameworks

Design patterns are more abstract than frameworks.Design patterns are smaller architectural elements than frameworks.Design patterns are less specialized than frameworks.

2005/7/15 70

A Learning Strategy: Contrast

Pick an example from Motivation or samples codeDesign it from the first principles, or simply just you-know-howRead the patternRedo the example according to the patternCompare the two implementations

2005/7/15 71

5. Exceptions and exception handling

public void methodName( ) throws Exception3// throws{

try{ // try // invoke operations that throw exceptions…

}catch( Exception1 e1 ) { … }// catch catch( Exception2 e2 ) { … }// catch finally { … } // final

}

2005/7/15 72

Exception and exception handling for better robustness

Exceptions are the way to let an application know that an abnormal condition has occurred and attention is required from the client code.

Replaces error code, which are subject to encoding and interpretation (not a first class entity)

Robustness is one of the major design issues to be considered when developing an application, especially large-scale ones.

Java mandates exception handling to really promote robustness

2005/7/15 73

Exception handling

Exceptions are caught by the corresponding exception handlers along the method invocation stack.

If the calling method can not handle the exception, it throws the exception up to its calling method repeatedly. If an exception can’t be caught by any method along the method invocation stack, the thread that throws the exception will be failed.

2005/7/15 74

Designing for exception handling in implementation

Requires knowledge of the application domain to adequately handling exceptions:

Whether an exception is at domain layer (domain classes) or infrastructure layer (Java API) is encoded in the type of the exception.[domain classes should not throw API exceptions]All in all, programmer is usually ill-informed of how an exception should be handled.

Further, lots of different exceptions are possible in large scale application.

Designing handlers for all exceptions at the implementation stage can be daunting.

2005/7/15 75

Consequences

Consequently, it is not uncommon to see bypassing exceptions: e.g., catching an exception and do nothing (to bypass Java EHM)wrong handling: logic for remedying the exceptional situation is ill-informed or not available.

This decreases the robustness of the application which the Java exception handling mechanism seeks to achieves in the first place.

2005/7/15 76

Checked and unchecked exceptions

package java.lang Throwable

ErrorException

RunTimeException Non-RunTimeException

Any SubClassAny SubClass

Any SubClass

checked exception

unchecked exception

2005/7/15 77

Strategies for exception handling (I) [COAD]

Use checked exceptions for recoverable situationsUse unchecked for programming logic problemsReserve the use of java.lang.Error for underlying virtual machine problemsAlways throw a subclass of java.lang.Exception or java.lang.RuntimeException.

2005/7/15 78

Strategies for exception handling (II) [COAD]

Create exception classes to represent what went wrong, not where.Use the same parameterized message template for all instances of an exception classDo not catch and ignore exceptionsDo not catch checked exceptions and throw an unchecked exception

2005/7/15 79

Strategies for exception handling (III) [COAD]

Catch unchecked exceptions and throw a checked exceptionDo not propagate exceptions from one architectural layer to anotherCreate abstract superclasses for related sets of exceptionsConsider declaring that a method throws a more generic exception

2005/7/15 80

Architecture and design level problems

ModularityException evolutionFunction evolution

Reuse of exception handling logicReadability

2005/7/15 81

Process Context (I)

1. Choose the use case covering core functionality as the first increment.

2. Do at least two iterations on the first increment. First iteration build the core functions; second iteration and on improves architecture and usability.

Produce result before completing the entire program

strategiesExpectation/ requirement /reality

2005/7/15 82

Process Context (II)

1. Practice “tester’s mentality”: play adversary of the classes/methods by actively thinking how to break them. Do the same to the use case.

2. Write tests that run automatically using a testing framework such as CppUnit.

“Correctness”

strategiesExpectation/ requirement /reality

2005/7/15 83

Process Context (III)

1. Choose the use case covering core functionality as the first increment.

2. Architecture styles and design patterns

3. Add iterations (within this increment or later) to expose the architecture.

4. To not to deviate from the architecture in later increments

To support exploration of new ideas; easy to add (and remove) functionalities.

strategiesExpectation/ requirement /reality

2005/7/15 84

Process Context (IV)

1. Create self-documenting code with tests

2. Choose a minimal set of artifacts

3. Mandate that artifacts are in place at the end of each increment

4. Sync artifacts

communication/staff stability

strategiesExpectation/ requirement/reality

2005/7/15 85

4. Case studies

Curve DetectionDevelopment of SyncFree: a

NSC open source project

2005/7/15 86

Case study: curve detection

RANSAC curve detection: the input is a binary edge map from which curves (circles and ellipses) are extracted.Detection performance (missing rate/false positive rate/time/space) must be compared with existing RANSAC and randomized Hough methods Improvement to sampling/transform might be found from analyzing false positivesPre-run a long experiment

2005/7/15 87

Use cases

Run circle detection on a binary edge map

Core steps and variants Compare two circle detection methods

Identical resource constraintBest effort comparison

Play back the detection processAnalyze false positives

2005/7/15 88

Use case for first incrementUse case: Run circle detection on a binary edge map 1. This use case begins when user request circle detection done on a binary edge

map. 2. User prepares a binary edge map. 3. User prepares detection profile. 4. Circle detector reads the profile to set up the detection, including sampling

strategy, transform, and 5. Detector carries out the detection. Grader is notified when detection is done.. 6. Grader reports detection results base on reference keys in detection profile. 7. Grader reports performance characteristics, including missing rate, false positive

rate, and computing time.

2005/7/15 89

Domain model: first increment

2005/7/15 90

Review use case and domain model

Once use case and domain model are ready (first draft), call for a review Purpose of the review: validation and learningIn the review, staff learn how conceptual model might interact to satisfy the use caseAnother method for discovering concept CRC (Class/Responsibility/Collaboration)

2005/7/15 91

Design model: sampler

Responsibility: take random samples from the edge mapVariation: pre-segmented edge map

2005/7/15 92

Sampler as a filter controller

2005/7/15 93

Increment 1 design model

2005/7/15 94

Iterations

Two additional iterations are addedTo expose the filter architectureTo expose the MVC architecture for Sampler-Transform-Testing Criteria

Improve the usabilityMultiple tries on randomly generated imagesFurther refine detection profileSummary of detection results

2005/7/15 95

Increment 2 and on

Build on existing MVC+filter architectureCompare two methods

Two methods identical except in the intended component (e.g., sampling method or transformation method)

Template method (GoF)Flow modeled with a template methodVariations hooked to the template method at the extension points

2005/7/15 96

Concentrate on the Domain

Add GUI only if domain layer is relatively completeAvoid implementing service layer (e.g., distribution, persistence framework)

They are technically challengingCould easily distract your effort (domain)Find or buy service layer class libraries/framework

2005/7/15 97

Language and tools

Language: gnu C++“Ide”: Emacs, EclipseVersion control: cvs + winCvs, EclipseUnit testing: CppUnitUML tool: dia, visioOS: Linux RedHat 6.2, 7.0, 9.0

2005/7/15 98

SyncFree: 一個開放原始碼個人資料同步軟體之開發

2005/7/15 99

本次報告我們將介紹SyncFree的開發、設計與使用

開發動機與設計重點開發流程與環境

使用案例說明

2005/7/15 100

開發動機與設計重點

2005/7/15 101

在不同場合操作不同電腦的使用者，經常發生弄不清楚某個最新版本的檔案位於何處的問題

家中

辦公室 出差

2005/7/15 102

解決個人資料同步問題的方法很多，我們希望提供一個方便、有彈性且容易使用的方案

資料操作與管理方法

(單一資料或多份資料)資料複製媒介

資料同步機制 (自動或人工)資料同步工具

2005/7/15 103

資料同步工具軟體大致可區分為下列四類, SycnFree 1.0 屬於Type II & Type III

2005/7/15 104

免費、開放原始碼、容易使用、跨平台、跨網際網路、擴充性為 SyncFree的設計目標

容易使用

(wizard 介面與 console 模式)擴充性

(支援多種同步資料的方法)跨網際網路

(支援HTTP, FTP)

2005/7/15 105

開發流程與環境

2005/7/15 106

我們在開發 SyncFree時實施持續整合(continuous integration)

2005/7/15 107

持續整合系統利用StatCVS產生之 SyncFree各個模組所佔比例報表

2005/7/15 108

StatCVS產生之 SyncFree模組異動紀錄

2005/7/15 109

StatCVS產生之詳細程式碼異動紀錄

2005/7/15 110

持續整合系統利用JUnit 產生之單元測試結果畫面

2005/7/15 111

持續整合系統利用JDepend產生之程式相依性報表

2005/7/15 112

使用案例說明

2005/7/15 113

SyncFree住家使用環境應用範例

2005/7/15 114

SyncFree辦公室使用環境應用範例

2005/7/15 115

您可至台北科技大學資訊工程系網站下載 SyncFree，並歡迎您將使用心得告訴我們

我們使用Java技術,開發了一個免費、開放原始碼、容易使用、跨平台、跨網際網路的個人資料同步軟體

在開發 SyncFree的過程中，我們套用了設計樣式、簡單設計與持續整合等技術

Question?

2005/7/15 116

套用Strategy與ObjectFactory樣式以支援多種資料同步演算法

2005/7/15 117

同樣地，我們套用Strategy與ObjectFactory樣式以支援多種資料傳輸協定

2005/7/15 118

2005/7/15 119

5. Conclusion

2005/7/15 120

Summary

The process improves on the following aspects of the project:

Productivity: continual deliveriesQuality: through unit and acceptance testingCommunication: in review sessionsDocumentation: self documenting code, tests, and additional artifactsStaff transition

2005/7/15 121

Further reading

TestingS. McConnel, Code complete, Chapter 25 Microsoft press. 1993.E. Gamma and K. Beck, “JUnit cookbook”.

ReactoringM. Fowler, Refactoring, Addison-Wesley. 1999.

PatternsE. Gamma et al., Design Patterns, Addison-Wesley. 1994.F. Buschmann et al., Pattern-Oriented Software Architecture, John Wiley. 1996.

2005/7/15 122

M. Grand, Design Patterns in Java, 2 volumes, John Wiley, 1998.

Development processC. Larman, Applying UML and Patterns, Prentice Hall, 1998.Blaha and Rumbaugh, Object-oriented Modeling and Design, 2nd ed, Prentice Hall, 2005A. Cockburn, Agile Software Development, Addison-Wesley, 2002

2005/7/15 123

C++/STLS. Lippman, C++ Primer 3rd ed., Addison-Wesley, 1998 U. Breymann, Designing Components with the C++ STL, Addison Wesley, 1998.S. Meyers, Effective C++, 2nd Ed. Addison-Wesley, 1998. S. Meyers, More Effective C++, Addison-Wesley, 1996.

2005/7/15 124

[Skowronski 2004] V. Skowronski, “Do agile methods marginalize problem solvers?” IEEE Computer, Oct 2004. pp 120.[Stelting 2005] S Stelting, Robust Java, Prentice Hall, 2005.