quality analysis of industrial systems. laquso experience serguei roubtsov

Quality analysis of industrial systems.

LaQuSo experience

Serguei Roubtsov

/ LaQuSo / Mathematics & Computer Science PAGE 219-04-23

LaQuSo: Laboratory for Quality Software

• 9 employees + master students and students-assistants (HG 5.91)

• industrial projects

• research projects

• own research in software quality assessment and tooling


Analysis approach

Focus on maintainability:

• Static code analysis, architecture assessment, code reviews, tooling

• Based on quantifiable measures: software metrics

• Provides an answer to an analysis question; for this purpose metrics:• should reflect quality criteria, thresholds

• Visualisation


What code is hard to maintain?

• Hard to understand• not documented• cluttered or inconsistently used/developed • too large

• Difficult to modify• duplicated • intertwined• non-extendable• non-portable

• Difficult to test / analyse • too complex

What to analyze?


Architecture:• dependencies (layering)• dependency cycles • code external duplication• dead code• system documentation

Code base:• code size and complexity• duplication metrics• potential bugs• documentation• adherence to standards


Our Tooling

Software Quality Analysis and Visualisation Toolset

Abstractness and Instability

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Abstractness (A)

Insta

bil

ity (

I)

core.mo

del.base

core.mo

del

core.action.configu

ration.defaults.helpe

r

core.service

core.action.configu

ration.salesTax

.helper

core.action.contra

cts.meterreads.helpe

r

core.action.credita

pp.applica

tion.helpe

r

core.action.credita

pp.creditdecisio

n.bureauS

umm

ary

core.integration.bu

sinesslaye

r.creditA

pplica

tion.creditD

ecision.b

ureaus.pojo

core.action.credita

pp.creditdecisio

n.rac

core.action.credita

pp.creditdecisio

n.scoring

core.action.credita

pp.helper

core.action.credita

pp.importe

dCre

ditAp

ps

core.integration.bu

sinesslaye

r.SU

SI

core.action.pricing

.application.helper

core.action.quote.vdo

core.client.w

eb.form

core.criteria

core.hcriteriaA

PI

core.integration.bu

sinesslaye

r.address

Valida

tion

core.integration.bu

sinesslaye

r.contract.expo

rt

core.integration.bu

sinesslaye

r.creditA

pplica

tion.creditD

ecision.b

ureaus

core.integration.bu

sinesslaye

r.creditA

pplica

tion.creditD

ecision.ra

cs.pojo

core.integration.bu

sinesslaye

r.creditA

pplica

tion.creditD

ecision.s

coring.pojo

core.integration.bu

sinesslaye

r.creditA

pplica

tion.docum

ents.pojo

core.integration.bu

sinesslaye

r.creditA

pplica

tion.merlin

core.integration.bu

sinesslaye

r.genie

core.integration.bu

sinesslaye

r.mo

odys.custom

erSearc

h

core.integration.m

arshalling.casto

r

core.mo

del.impl

core.mo

del.impl.ba

se

core.pricing.factory

core.rule.serv

ice

core.schedule.job.burea

us

core.service.access

core.service.partner

core.service.workflow

core.util.excel

core.util.validation

core.versioning

test.serv

ice.select

AV

Repository

(2)

(3)

(4)

(1)

Name, Class Count, Abstract Class Count, Ca, Ce, A, I, D, V:bsh,0,0,1,0,0,0,1,1com.caucho.burlap.client,0,0,1,0,0,0,1,1com.caucho.burlap.io,0,0,1,0,0,0,1,1com.caucho.burlap.server,0,0,1,0,0,0,1,1com.caucho.hessian.client,0,0,1,0,0,0,1,1com.caucho.hessian.io,0,0,1,0,0,0,1,1com.caucho.hessian.server,0,0,1,0,0,0,1,1com.ibatis.common.util,0,0,1,0,0,0,1,1oracle.toplink.essentials.sessions,0,0,1,0,0,0,1,1oracle.toplink.exceptions,0,0,2,0,0,0,1,1oracle.toplink.expressions,0,0,1,0,0,0,1,1oracle.toplink.internal.databaseaccess,0,0,1,0,0,0,1,1oracle.toplink.jndi,0,0,1,0,0,0,1,1oracle.toplink.logging,0,0,1,0,0,0,1,1oracle.toplink.publicinterface,0,0,2,0,0,0,1,1oracle.toplink.queryframework,0,0,1,0,0,0,1,1oracle.toplink.sessionbroker,0,0,1,0,0,0,1,1oracle.toplink.sessions,0,0,2,0,0,0,1,1oracle.toplink.threetier,0,0,1,0,0,0,1,1oracle.toplink.tools.sessionconfiguration,0,0,1,0,0,0,1,1oracle.toplink.tools.sessionmanagement,0,0,1,0,0,0,1,1org.aopalliance.aop,0,0,9,0,0,0,1,1org.aopalliance.intercept,0,0,24,0,0,0,1,1org.apache.axis.encoding.ser,0,0,1,0,0,0,1,1org.apache.catalina.loader,0,0,1,0,0,0,1,1

org.aspectj.weaver,0,0,2,0,0,0,1,1org.aspectj.weaver.ast,0,0,1,0,0,0,1,1org.aspectj.weaver.bcel,0,0,1,0,0,0,1,1org.aspectj.weaver.internal.tools,0,0,1,0,0,0,1,1org.aspectj.weaver.loadtime,0,0,1,0,0,0,1,1

org.quartz.spi,0,0,1,0,0,0,1,1org.quartz.utils,0,0,1,0,0,0,1,1

org.quartz.xml,0,0,1,0,0,0,1,1org.springframework.aop,24,20,17,6,0,83,0,26,0,09,1org.springframework.aop.aspectj,39,7,3,24,0,18,0,89,0,07,1org.springframework.aop.aspectj.annotation,27,3,0,19,0,11,1,0,11,1org.springframework.aop.aspectj.autoproxy,3,0,1,8,0,0,89,0,11,1org.springframework.aop.config,17,3,1,15,0,18,0,94,0,11,1org.springframework.aop.framework,37,9,22,18,0,24,0,45,0,31,1org.springframework.jdbc.core,53,20,6,20,0,38,0,77,0,15,1org.springframework.jdbc.core.metadata,22,2,1,10,0,09,0,91,0,1org.springframework.jdbc.core.namedparam,10,4,3,12,0,4,0,8,0,2,1org.springframework.jdbc.core.simple,17,6,0,12,0,35,1,0,35,1org.springframework.jdbc.core.support,8,5,2,14,0,62,0,88,0,5,1org.springframework.jdbc.datasource,27,7,13,14,0,26,0,52,0,22,1org.springframework.jdbc.datasource.lookup,8,2,2,13,0,25,0,87,0,12,1org.springframework.jdbc.object,14,8,0,12,0,57,1,0,57,1org.springframework.jdbc.support,15,5,12,16,0,33,0,57,0,1,1org.springframework.jdbc.support.incrementer,15,4,0,8,0,27,1,0,27,1org.springframework.jdbc.support.lob,18,5,5,12,0,28,0,71,0,02,1org.springframework.jdbc.support.nativejdbc,10,2,2,7,0,2,0,78,0,02,1org.springframework.jdbc.support.rowset,4,2,2,6,0,5,0,75,0,25,1org.springframework.jdbc.support.xml,7,6,0,7,0,86,1,0,86,1org.springframework.web.servlet.view.xslt,4,2,0,17,0,5,1,0,5,1org.springframework.web.struts,16,5,0,22,0,31,1,0,31,1org.springframework.web.util,24,6,26,15,0,25,0,37,0,38,1org.w3c.dom,0,0,12,0,0,0,1,1org.xml.sax,0,0,3,0,0,0,1,1


SQuAVisiT

• Flexible• Plug-in architecture

• Languages• C, Cobol, Java, JavaScript, PL/SQL, Delphi, C#

• Analysis tools (third party and our own)• dependency extractors, duplication detectors, error

detectors, metrics calculators, parsers, code style checkers

• Visualization tools• MetricsView, GraphViz, ExTraVis , MatrixZoom, SolidSX,

visualization modules of third-party tools


Real life industrial systems

They are often:• Heterogeneous (C/Assembler, Cobol/PL SQL,

Java/Object mapping to SQL…)

• Incomplete (some code is in libraries and third-party components)

• Not compilable and executable within analysis environment ( ‘weird’ OS, proprietary development environment, …)


Expert system

Industrial case: Insurance company’s expert system

• What kind of system do we have?• Heterogeneous: JavaScript, PL/SQL, C++, Java,

Cobol• Medium size: 300 KLOC• 15 years old• Scarce documentation• Oracle DB

• Problem reported• Maintenance (dis)continuation decision


Dependencies Model: Matrix View

• (Almost) layered: good design

• BUT data layer is accessed from several layers

• Layers affected by calls from top layer are visible (red squares)

Data layer


Dependencies Model: Extravis

• Green ‘bubbles’: controversial coding approach

• Parameters as names f(1,3) -> f_1_3

• Absence of dedicated data access layer is confirmed


Code duplication

• Code is polluted with duplication: restructuring would improve maintainability but may change the architecture

CCFinder/Gemini

(Toshihiro Kamiya)


Summary

• Layered architecture• System is well-structured but• JavaScript + two-tier architecture could cause serious

maintenance problems in the future• Code polluted by duplication

• Low impact if no major changes are expected

• Analysis advice:• Short term

− Refactor and maintain for limited amount of time (3-5 years)

− Develop overview documentation• Long term

− Migrate to three-tier architecture


Industrial case: Embedded System

• What kind of code do we have?• Component system with compile-time binding via

make files

• C with embedded Assembler

• Complete

• Medium size: 150 KLOC

• Developer’s assumption• Layered architecture

• Problems reported• Extensive change. Is architectural purity still

preserved?


Dependencies Structure

• system is poorly layered

• unexpected cyclic dependencies exist between components


Summary

• Layered architecture• System is poorly-structured (indications of decay) but

• Code is of good quality, well documented• The system is NOT large or complex

• Analysis advice:• Reengineer affected parts according to the presumed

layered architecture


Industrial case: Pension fund

• What kind of system do we have?• Homogeneous: Cobol• Large: 1.7 MLOC• 17 years old• Oracle DB

• Problem reported• Need to migrate due to discontinuation of

support


Dead code?

• “Empty spaces” in the visualization

• 1216 modules not called by other modules• Dead code?• Other (sub)systems?

• 651 are dead• Confirmed by the developers


Results of Analysis: effort

Halstead metrics

Time to understand (T) is proportional to Halstead Effort:

T = E / 18 /3600Time to implement, hours

0

50

100

150

200

250

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Time to understand, hours


Summary

• Architecture • System structure is preserved but intertwined in some

places• Dead code is widely spread

• Code is polluted by duplication but

• Percentage of weak (large, complex) parts is low

• Analysis advice:• Short term:

− Refactor weak parts, eliminate dead code and maintain for limited amount of time

• Long term:− Redevelop on a modern platform


Expert system

Industrial case: Insurance company’s front-end

• What kind of system do we have?• Technical data

− Homogeneous: Java

− Large: 750 KLOC

− Oracle DB

− J2EE application (Spring Framework, Hibernate)

• Recently developed by a third party

• No documentation

• Problem reported• Purchase decision


Code not available? What can we do?

1) Install locally.

2) Perform measurements.

4) Analyse, interpret.

3) Tune

Customer


Understandability: Documentation

• Comments percentage (LOCs counter tool)• Average: 43%

• Ranging from 0% to 1500%. Why?− large repeated header blocks of comments

− commented out code

• Javadoc (CheckStyle tool)• 85260 violations

• Missing or malformed declarationsDocumentation generation is impossible, orDocumentation quality is compromised

Documentation quality should be reassessed!

Dn – Distance from the main sequence

Abstractness = #AbstrClasses/#Classes

Instability = #Out/(#Out+#In)

1

10

Dn = | Abstractness + Instability – 1 |/LaQuSo SET / W&I

main sequence

zone of pain

zone of uselessness

[R.Martin 1994]

Average Dn

1.00

0.00

Benchmarksopen source

0.186 our System

/LaQuSo SET / W&I

What about distributions?

PAGE 28

Dn threshold value

% of packages beyond threshold

our System

an average open source system

/LaQuSo SET / W&I


What have we seen?

Understandable not documented

Poor Javadoc. Commented out code suspected.

Modification abstractness instability balance

Good

Effort Time to understand: 1.87 man-years

Annual maintenance effort (first year): 7.7 man-years

Architecture is good Documentation should be reassessed


Conclusions

• Approach comprising analysis and visualization

• Supported by SQuAVisiT, a flexible tool allowing• To address different maintainability aspects• To combine different analysis and visualization

techniques

• Confirmed by analysis of several middle-size to large systems (150 KLOC – 1.7 MLOC)

Future Work

• Flexible SQuAVisiT data structure:How to store data (dependencies, metrics) about historically

and hierarchically different software artifacts?

• (Fully-featured) parsers/fact extractors for C++, C#, Delphi,…

• Improved dependency analysis:Analysis of dynamic bindings, injected dependencies,

dependencies via global and static variables,…

• More metrics to retrieve:e.g. Dn-based analysis for different languages,

analysis/benchmarking of distributions of different metrics / LaQuSo / Mathematics & Computer Science PAGE 3119-04-23

(Your?)

quality analysis of industrial systems. laquso experience serguei roubtsov

Documents

quality software

tooling slide

static code analysis

code size

code reviews

software quality assessment

software metrics

complex slide