sept. 9, 2009 seminar at beijing university 1 deriving and analysing the quality of software...

Sept. 9, 2009

Seminar at Beijing University

1

Deriving and Analysing the Quality of Software

Architectural Designs

Hong ZhuDepartment of Computing, Oxford Brookes University,

Oxford OX33 1HX, UKEmail: [email protected]

mailto:[email protected]

Sep

t. 9,

200

9

2Seminar at Beijing University

OutlineOverview of existing work and open

problemsOur approach

Graphic model of software qualityDerivation of software quality model from designAnalysis of quality models

ConclusionComparison with related worksFuture work

Sep

t. 9,

200

9


The Notion of Quality Quality, in particular software quality, is an elusive

concept and varying from people to people. Garvin’s definitions of quality

Transcendent view: Quality is universally recognisable. It is related to a comparison of features and characteristics of products.

Product-based view: Quality is a precise and measurable variable. Differences in quality reflect the differences in quantities of some product attributes.

User-based view: Quality is the fitness of intended uses.Manufacturing-based view: Quality is conformation to

the specifications.Value-based view: A quality product is one that

provides performance at an acceptable price or conformance at an acceptable cost.

Sep

t. 9,

200

9


Software Quality Models Models about software quality in terms of

The factors that affect software quality• Attributes directly indicate the quality of he system,

e.g. reliability, correctness, user friendliness, etc. • Attributes indirectly related to quality, e.g. internal

complexity, etc. The interrelations between the factors

• Causality models: Causal relationship, in terms of stereo-type relations

• Quantitative models: Quantitative relations expressed as numerical functions, Numerical values of the basic attributes are given, e.g.

through using metrics/measurementsThe overall quality in an attribute on a more high level of

abstraction is calculated numerically

Sep

t. 9,

200

9


Hierarchical Models of SQA set of quality related properties organised

into a hierarchical structureE.g. decompose quality into a number of quality

attributes and attributes into a number of factors, and then a number of measures, etc.

Positive causal relationship is representedExamples:

McCall model (1977)Boehm model (1978)ISO model (1992)Bansiya-Davis model of OO designs (2002), etc.

Sep

t. 9,

200

9


McCall’s Model

Product operations

Product transition

Product

revision

Maintainability: Can I fix it? Flexibility:Can I change it?Testability:Can I test it?

Portability: Will I be able to use it on another machine?

Reusability:Will I be able to reuse some of the software?

Interoperatability:Will I be able to interface it with another machine / software?

Correctness: Does it do what I want? Reliability: Does it do it accurately all the time?Efficiency: Will it run on my machine as well as it can?Integrity: Is it secure?Usability: Can I run it?

Sep

t. 9,

200

9


ISO 9126 Model

Software quality

ReliabilityMaturity

Fault toleranceRecoverability

Portability

AdaptabilityInstallability ConformanceReplaceability

Maintainability

AnalysabilityChangeability

StabilityTestability

EfficiencyTime behaviour

Resource behaviour

UsabilityUnderstandability

Learnability Operability

Functionality

SuitabilityAccuracy

InteroperabilityCompliance

Security

Sep

t. 9,

200

9


Relational Models of SQ A quality model consists of:

A number of quality attributes and factors, etc. A set of stereo types of relationships between them,

normally include • Positive relation:

High on one aspect of quality implies inevitably high on the other

• Negative relation: High on one aspect of quality implies inevitably low on the other

• Neutral relation: High or low on one aspect of quality does not imply on the other aspect the quality is high or low.

Examples:Perry model (1991)Gillies model (1992, 1997)

Sep

t. 9,

200

9


Perry’s ModelCorrectness

Reliability

Efficiency

Integrity

Usability

Maintainability

Testability

Flexibility

Portability

Reusability

Interoperability

CR

E

I

U

M

T

F

P

R

I

Direct

Inverse

Neutral

Sep

t. 9,

200

9


Roles in Software DevelopmentUnderstanding software quality

To measure an existing system’s quality, To predict a system’s quality during

developmentTo analyse quality problems during development

and/or in operation of a systems

Guidelines to development activitiesTo elicit users’ requirements on software quality To perform quality assurance activities in

software development and maintenance

Sep

t. 9,

200

9


Key Features of Existing SQ Models

Feature Pros Cons

Universal Applicable to all software systems

Not address system specific issues

Simplicity Stereo types of relationships between quality attributes

Incapable of dealing with complicated relationships between quality attributes

Black-box Treat software systems as black-boxes

Provide little help to the designers to consider system’s structure

Empirical Developed-based on empirical knowledge from experiences

Cannot be validated or verified of its correctness

Sep

t. 9,

200

9


Our Approach Representing software quality models in a diagrammatic

notation (Zhang, Zhu, et al. 2002) to improve expressiveness of quality models to represent

complicated relationships between factors to associate quality features with the structure of the system

Deriving software quality models from architectural designs (Zhang, et al. 2002, Zhu, 2005) to systematically derive quality models from software architectural

designs by employing extended hazard analysis concepts and techniques

Automating the analysis of quality models to help software designers (Zhang, et al, 2006a, 2006b) to perform various analysis tasks based on graphic quality models

by developing and using automated software tools

Sep

t. 9,

200

9


Graphic Notation for Modelling SQ

Subject[+|-] PropertyPhenomenon

Node: representing quality carrying properties and quality attributes and observable phenomena

Link without annotation: representing logic relations between the nodes.

Link with Annotation: providing additional information of the rationale of the relation between nodes.

[+ | - ]

Annotation of Reasons

[+ | - ]

Interface is not displayed properly

Client side- Compatibility

Font size too small to be illegible.

System- Usability

Difficult to operate

Node A Node BLink

Sep

t. 9,

200

9


Example: Web-based ApplicationsHTML files

+ Well StructuredLarge size

HTML files+ Navigability

Small number of hyperlinks

System+ UsabilityEasy to find required info

Web Server- Responsiveness

Long response time

HTML files- Correctness

Contains broken links

Online Help- AvailabilityNot available

Client side- Compatibility

Font size too small to be illegible when displayed

user’s platform

System- Usability

Cannot find required info

Server side- Performance

Execution speed is slow

Server side+ Load

Highly demanded

Files are considered as

unavailable when time-out

Simpler hyperlink network usually easier

to navigate

Less nodes means less links

Need long time to transmit the files

Web page cannot be displayed

properly

Unable to obtain files through hyperlinks

Unable to get help when experiencing difficulty

Sep

t. 9,

200

9


Deriving Quality Models: The HASARD MethodsHASARD: Hazard Analysis of Software ARchitectural DesignInput: A software architectural model Output: A quality model in graphic notationProcess:

Hazard identificationA hazard identification method is applied to identify all quality sensitive observable phenomena of the components, connectors, the system, etc.

Cause-consequence analysisThe causal relationships between the identified hazards are recognized

Model assemblingThe information obtained in the previous steps are assembled together and represented in the graphical notation

Quality concern recognitionThe quality carrying properties/quality attributes that a phenomenon demonstrates are recognized according to the nature of the phenomenon

Sep

t. 9,

200

9


Hazard IdentificationExtended Hazard and Operability Studies

(HAZOP)The method relies on determining answers to

questions of what-if nature. By applying a set of guide words systematically

to develop a collection of what-if questions. They are applied to the attributes of various

components and connectors of the system being studied.

If a deviation from the normal working of the component is credible, the behaviour of the component is considered as a possible hazard.

Sep

t. 9,

200

9


Guide Words for Software Hazard Identification

Guide word

Applicable attribute

Interpretations

No Date/control signals

No data / control signal exchanged; No data / control signals produced/received.

Component property/ function

The component / connector does not have the designed property / function.

Component / connector

The system does not contain the component / connector.

More Quantitative parameters

The value of the parameter is too large.

Less Quantitative parameters

The value of the parameter is too small.

Sep

t. 9,

200

9


As well as

Event or activity Redundant data sent, or event/activity occurred in addition to the intended ones.

Component / connector

Other components / connectors are added in addition to the intended ones.

Part of Structured data Only a part of the data produced, stored or received.

Structure events Only a part of the events happened.

Reverse Direction of flow The information flow in the opposite direction.

Event The opposite event happened.

Other than

Data/control signals, quantitative/qualitative parameters

Incorrect data / control signals produced;The parameter has a value different from the design.

Component/system’s function or property

The component has a functionality / property different from the designed.

Component/ connector Other kind of component / connector is contained.

Sep

t. 9,

200

9


Early Periodical events The event happened earlier than expected.

Late Periodical events The event happened later than expected.

Before Temporal orders The event happened in the order earlier than designed.

After Temporal orders The event happened in the order later than designed.

Sep

t. 9,

200

9


Example: Internet ConnectionGuide word

Hazard/Failure mode Causes Consequences

No The internet connection passes no messages between the client and server.

Physically disconnected; Traffic jam; Software failure; Network server down.

Client cannot communicate with the server.

More More messages are delivered than the clients and the server send out, e.g. duplicated messages.

Hacker’s attack; Heavy traffic on the Internet caused resending packages.

System clash; Overload on the server or the client.

Less Fewer messages are delivered than the server and the clients send out, i.e. lost messages.

Discontinued Internet connection; Heavy traffic on the Internet; Software failure.

Incomplete transactions; System clash; Damage the integrity of the data and program on the server and/or client.

Sep

t. 9,

200

9


As well as

Messages are delivered to other destinations in addition to the designated receiver.

Hacker’s attack; Software failure.

Leak of sensitive information.

Part of Only a part of the packets of a message is delivered to the destination client or server.

Discontinued Internet connection; Heavy traffic on the Internet; Software failure.

Software failure; Production of incorrect computation results if incompleteness is not detected.

Other than

A message not from the client (or the server) is passed to the server (or client).

Hacker’s attack; Other software system’s failure

System failure; Damage the integrity of the data and the program.

Other than

The message is in a different format.

The client or the server is modified; Fault in the software.

System failure.

Sep

t. 9,

200

9


Cause-Consequence Analysis It aims at understanding the causal relationships

between hazards. It can be performed backward or forward, or a

combination of both. Forward analysis

• Search for the potential effects, i.e. consequences, of a hazard until the consequence is terminal.

• A hazard or failure mode is terminal: if it does not affect any other component of the system or If It does not cause any other hazards/failures. if we are not interested in its further consequences.

Backward analysis • Search for the causes of a hazard until the hazard is

primitive. • A hazard or failure mode is primitive:

if its causes cannot be further identified without additional knowledge about the system

if we are not interested in its causes

Sep

t. 9,

200

9


Results of Cause-Consequence Analysis

Sep

t. 9,

200

9


Constructing Graphic Quality Model Input:

The records of the cause-consequence analysis Output:

A partially completed graphical representation of quality model

The transformation:Each record of the hazard or failure mode becomes a

node with the component and phenomenon as specified in the record.

Each row in the record becomes a link from the node that represents the cause to the node that represents the consequence.

The explanation column of the row forms the reason of the link.

Sep

t. 9,

200

9


Example:

Sep

t. 9,

200

9


Identification of Quality Concerns

For each node in the diagram Identify the quality attribute or quality carrying property

that the phenomenon demonstrates• Comparing the observable phenomenon with the

definitions of a set of quality attributes and quality-carrying properties

• Recognising a new attribute or property, if no existing one matches.

Fill the identified quality attribute into the slot of the node Examples,

‘a hyperlink is broken’ demonstrates the quality attribute correctness of the HTML file.

‘Server is down’ is related to the availability of the server.

Sep

t. 9,

200

9


Analysis of Graphic Quality ModelsContribution factors

What are the factors that affect a specific quality issue that the user (designer) is interested in?

HTML filesWell Structured

Large size


Long response time



Server side- Load balance

Some servers have high demand, but some not

Need long time to transmit the

files

Example: Factors of server’s responsiveness

Sep

t. 9,

200

9


Impacts of design decisions What are the consequences

of a design decision?

Simpler hyperlink network usually

easier to navigate

HTML files+ Navigability

Small number of hyperlinks


Long response time

System- Usability

Cannot find required information

System+ Usability

Easy to find required information

Less nodes means less links


Files are considered as unavailable when

time-out

HTML files+ Well Structured

Large size

Example: The impacts of HTML files are of large sizes

Sep

t. 9,

200

9


Quality risks When a negative quality property is present, where does the risk come

from? And what are the implications of the problem?

Example: (partial model)The risk of long response time in a web-based application: its causes and implications

HTML files+ Well Structured

Large size


Long response time

System- Usability

Cannot find required info



Server side- Load balance

Some servers have high demand, but some not

Files are considered as

unavailable when time-out


Sep

t. 9,

200

9


Configure is easy

Overhead due to using middleware

System

Scalability

Adding more resources is easy

System

Performance

Response time is short

System

Flexibility

Use J2EE technology

-

More processing

power

Relationships between quality issues How two quality issues are interrelated?

Example:Relationships between flexibility and performance

Sep

t. 9,

200

9


Trade-off points:When a design decision has positive impact on one quality attribute but negative impact on another quality attribute, a trade-off must be made to balance between the two. Where are the points in a complicated design that need to make trade-offs? Or simply, where are the trade-off points?

It is hard to reuse

Reduce the interaction between EJB components

system- Economy

More development time

EJB componetstructureness

Large size

System performance

Shorter response time

Example:

EJB components’ size is a trade-off point in systems implemented using EJB.

Sep

t. 9,

200

9


Automation of Quality Analysis SQUARE: Software QUality and ARchitecture modeling Environment.

Tools to support software architectural modelling Tools to support the derivation of software quality models from architectural

design Graph algorithms are designed and implemented to automate the analysis

of software quality;

Hazard Analysis

Tools

Quality Analysis

Tools

Quality Model Editor

Architecture Model Editor

SA Model

Quality Model

Quality Analysis Results

Model Repository

Project Manager

Overall structure of SQUARE toolkits

Sep

t. 9,

200

9


Screen Snapshot of SQUARE toolkits

Sep

t. 9,

200

9


Case Study The subject:

A real e-commerce system of online trading of medicine. The system is operated by the local medicine trading

regulation authority who supplies medicines to all state-owned hospitals in the province.

Its main functions:• customer relationship management, • product catalogue management, • online trade management, • online auction of medicine supplies, • online information advertisement, • a search engine for medicine information, and so on.

The system is implemented in J2EEThe system has been in operation for more than one

year at the time of case study

Sep

t. 9,

200

9


Process of The Case StudyStep 1: Construct the architectural modelIt is a reverse engineering process:

Review of the design documents• The system’s design document consists of four main parts:

(a) a simple and schematic J2EE architecture showing that the system uses J2EE as implementation technology; (b) interface design, which consists of a lot of HTML files; (c) database design, which consists of about 63 database tables; (d) a simple UML class diagram that contains a dozen of classes and shows the logical view of the system.

Review of parts of the source code. • When design information was not well documented, parts of the

source code was reviewed Construction of architectural model Confirmation of architectural model by some of the chief developers

of the system• The draft architectural model was reviewed and corrected • The final version of the model was approved of its accuracy

Sep

t. 9,

200

9


Architecture of CRM Subsystem

Sep

t. 9,

200

9


Step 2: Construction of Quality Model

Employed the HASARD method and the SQUARE toolkits

The quality model was reviews and corrected in several iterations with the collaboration with the developers

The final version was confirmed for its accuracy and correctness by the developers

The final result: 48 nodes 60 links between the nodes.

Sep

t. 9,

200

9


Step 3: Analysis of the Quality ModelThe quality model was analysed using the SQUARE

analysis tools Identification of quality risks and quality trade-off points Derivation of the impacts of design decision and the contribution

factors on certain quality attributes The results were feed back to the developers

A workshop was run to validate the outcomes • All our findings were consistent with what has been observed in

the development and operation of the system. • Some of the phenomena observed in the operation of the

system were first time satisfactorily explained A number of specific suggestions on the improvement of the

system’s architecture were made• Some were taken by the development team in the development

of the new release of the system. • Some would result in major changes of system’s architecture

and regrettably cannot be implemented within the budget of the new releases.

Sep

t. 9,

200

9


Example 1: Causes of usability problemThe quality problem concerned:

‘Users often cannot find desired information on the website’Analysis goal:

Find the factors that may cause the problem Method:

Select the node that contains ‘user’ as the entity and “Cannot find info” as its phenomenon

Use the tool to find the contribution factors to this nodeResult:

The tool generated a sub-diagram that contains 25 nodes out of the 48 nodes in the quality model.

Interpretation of the result: Most components of the system affect usability of the system Usability is a very sensitive quality issue in the design

The outcome: Details are provided by the tool about what affect usability Useful direction to enhance usability

Sep

t. 9,

200

9


Example 2: Contribution Factors to Server’s AvailabilityProblem to be analysed:

What are the factors that affect the server’s availability?Method:

Select the node that contains ‘server’ as entity and ‘availability’ as property.

Use the tool to find the contribution factors.Result:

A sub-diagram is generated, that shows that the factors include hardware reliability, software reliability, power supply, system security, and maintenance.

Outcome: Suggested measures to improve availability To prevent hackers from attacking the server To ensure a reliable power supply To use reliable hardware to run the server To use fault tolerance to avoid software crashes on invalid input To provide maintenance tools to enable online maintenance To reduce the time that the system has to be shut down for

maintenance

Sep

t. 9,

200

9


Quality factors that affect server’s availability

Sep

t. 9,

200

9


Example 3: Identify the quality trade-off points Problem to be analysed:

What are the trade-off points in the design? Method:

Select a node and to find the consequences of the node using the tool Identify whether is leads to both positive result and negative result

It is hard to reuse

Reduce the interaction between EJB beans

system- Economy

More development time

EJB BeansGranularity

Large size

System performance

Shorter response time

Examples: • The size of

HTML files is a trade-off point. (See diagram on slide 25)

• The granularity of the session beans.

Sep

t. 9,

200

9


Comparison with related works Scenario-based analysis of software architectures

SAAM, ATAM, etc.: • Builds no quality model, • not automated

Our approach: • More systematic to derive quality model from design, • Can be automated for analysis once a model is constructed, • Models can be relatively easily validated

Hazard analysis of software system HAZOP, FMEA, etc.:

• Focused on safety, • Systematic, • But not interrelationships between quality attributes

Our approach: • More general rather than just for safety, • Focuses on the relationships between attributes

Sep

t. 9,

200

9


Future work

Combining with scenario-based methods to construct graphic quality models to represent results of scenario analysis

Analysis other quality features, e.g.Process-oriented quality features: how does a design

affect the software development process? Such as reusability, modifiability, portability, etc.

More case studiesExisting case study is on e-Commerce web-based

applicationsWork in progress: real-time and embedded systems

Sep

t. 9,

200

9


References

H. Zhu, Y. Zhang, Q. Huo and S. Greenwood, Application of Hazard Analysis to Software Quality Modelling, in Proc. of COMPSAC’02, pp. 139-144, IEEE CS, 2002.

H. Zhu, Software Design Methodology: From Principles to Architectural Styles, Elsevier, 2005.

Q. Zhang, J. Wu, and H. Zhu, Tool Support to Model-based Quality Analysis of Software Architectures, Proc. Of COMPSAC’06, IEEE CS, 2006.

Q. Zhang and H. Zhu, Automated Analysis of Software Designs with Graphic Quality Models, WSEAS Transactions on Computers, Vol. 5, Issue 10, Oct. 2006, ISSN 1109-2750, pp2232-2237.

Sep

t. 9,

200

9


Acknowledgement

The work reported in this talk is based on the research collaborated with Dr. Yanlong Zhang of Manchester Metropolitan University, UK, Dr. Sue Greenwood of Oxford Brookes University, UK, Ms. Qian Zhang and Mr. Jian Wu of National University of Defence Technology, China.

sept. 9, 2009 seminar at beijing university 1 deriving and analysing the quality of software...

Documents

notion of quality quality

quality product

software quality attributes

aspect of quality

quality problems

overall quality

particular software

number of quality attributes