measuring the structural quality of software€¦ · cwe-252 unchecked return parameter of control...
TRANSCRIPT
2/16/2017
1
Measuring the Structural Quality of Software
Measuring the Structural Quality of Software
Paul C. BentzDirector of Government and Industry Programs, CISQ
What is CISQ?
OMG Special Interest Group
CISQ is chartered to define automatable measures of software size and quality that can be measured in the source code, and promote them to become Approved Specifications of the OMG®
CISQ is chartered to define automatable measures of software size and quality that can be measured in the source code, and promote them to become Approved Specifications of the OMG®
OMG Special Interest Group
CISQ is chartered to define automatable measures of software size and quality that can be measured in the source code, and promote them to become Approved Specifications of the OMG®
CISQ Sponsors
CISQ
Co-founders
IT Executives
Technical Experts
Copyright © 2016 CISQ. All rights reserved.
2/16/2017
2
CISQExec
Forum
CISQ/OMG Standards Process
Automated Function Points
Reliability
PerformanceEfficiency
Security
Maintainability
Approved MeasureSpecifications
ISO Fasttrack
Deployment Workshops
OMG
Copyright © 2016 CISQ. All rights reserved.
• OMG Supported Specification for Automated Function Points
• Mirrors IFPUG counting guidelines, but automatable
• Specification developed by international team led by David Herron of David Consulting Group
Automated Function Points
2/16/2017
3
Content of CISQ Measures
CISQ Quality Characteristic Measures Example architectural and
coding violations composing the measures
Security 22 violations(Top 25 CWEs)
• SQL injection• Cross-site scripting• Buffer overflow
Reliability 29 violations• Empty exception block• Unreleased resources• Circular dependency
Performance Efficiency
15 violations• Expensive loop operation• Un-indexed data access• Unreleased memory
Maintainability 20 violations• Excessive coupling• Dead code• Hard-coded literals
Copyright © 2016 CISQ. All rights reserved.
� CWE-22 Path Traversal Improper Input Neutralization� CWE-78 OS Command Injection Improper Input Neutraliza tion
� CWE-79 Cross-site Scripting Improper Input Neutraliza tion
� CWE-89 SQL Injection Improper Input Neutralization� CWE-120 Buffer Copy without Checking Size of Input
� CWE-129 Array Index Improper Input Neutralization
� CWE-134 Format String Improper Input Neutralization� CWE-252 Unchecked Return Parameter of Control Elemen t Accessing Resource
� CWE-327 Broken or Risky Cryptographic Algorithm Usag e
� CWE-396 Declaration of Catch for Generic Exception� CWE-397 Declaration of Throws for Generic Exception
� CWE-434 File Upload Improper Input Neutralization
� CWE-456 Storable and Member Data Element Missing Ini tialization� CWE-606 Unchecked Input for Loop Condition
� CWE-667 Shared Resource Improper Locking
� CWE-672 Expired or Released Resource Usage� CWE-681 Numeric Types Incorrect Conversion
� CWE-706 Name or Reference Resolution Improper Input Neutralization
� CWE-772 Missing Release of Resource after Effective Lifetime� CWE-789 Uncontrolled Memory Allocation
� CWE-798 Hard-Coded Credentials Usage for Remote Auth entication
� CWE-835 Loop with Unreachable Exit Condition ('Infin ite Loop')
Common
Weakness
Enumeration
cwe.mitre.org
Robert MartinMITRE
The 22 CWEs in the Security Measure
Copyright © 2016 CISQ. All rights reserved.
2/16/2017
4
Confidential
Mul
ti-la
ngua
ge,
mul
ti-la
yer
Arc
hite
ctur
e
EJB
PL/SQL
Oracle
SQL Server
DB2
T/SQL
Hibernate
Spring
Struts.NET
COBOL
IMS
Messaging
Sybase
� Integration quality� Architectural compliance� Risk propagation� Application security � Resiliency checks� Transaction integrity � Function point,� Effort estimation
� Data access control� SDK versioning� Calibration across
technologies� IT organization level
System Level
• Code style & layout • Expression complexity• Code documentation• Class or program design• Basic coding standards• Developer level
Unit Level1
Java
Java
JavaJava
Web Services • Single language/technology layer
• Intra-technology architecture• Intra-layer dependencies• Inter-program invocation• Security vulnerabilities• Development team level
Technology Level
JSP ASP.NETAPIs
Technology Stack
CISQ Measures the Technology Stack
2
3
Copyright © 2016 CISQ. All rights reserved.
Multi-language, Multi-layer Architecture
EJB
PL/SQL
Oracle
SQL Server
DB2
T/SQL
Hibernate
Spring
Struts.NET
COBOL
IMS
Messaging
Sybase
JavaJava
Java
Web Services
JSP ASP.NETAPIs
Technology Stack
Entry
Filtering,Authentication
Access
Analyzing transactions and data flows across languages and layers
Challenges in the Technology Stack
Copyright © 2016 CISQ. All rights reserved.
2/16/2017
5
How Do CISQ Measures Relate to ISO?
• ISO 25000 series replaces ISO/IEC 9126 (Parts 1-4)• ISO 25010 defines quality characteristics and sub-c haracteristics• CISQ conforms to ISO 25010 quality characteristic definitions• ISO 25023 defines measures, but not at the source c ode level• CISQ supplements ISO 25023 with source code level measures
CISQ defined automatable measures for quality characteristics highlighted in blue
Copyright © 2016 CISQ. All rights reserved.
CISQ in Service Level Agreements
Copyright © 2016 CISQ. All rights reserved.
Monitor and Track Product Quality against Targets i n Service Level Agreements
Monitor and Manage Service Provider Performance Ove r Time
Automated Function
Points245
628
931
86
1047
659
2/16/2017
6
App Certification Using CISQ
CISQ measuresCISQ-conformant
technology CISQ-
conformance assessment
Technology vendors
used in
CISQ service process
CISQ-conformant service process
CISQ-conformance assessment
Service providers
to provide
ApplicationCertification
Security XσReliability XσPerformance XσMaintainability Xσ
� CISQ/OMG− only assess conformance− do not certify applications− program initiates 2016
� Service providers− use CISQ-conformant technology− in a CISQ-conformant service process− to provide application certifications
Copyright © 2016 CISQ. All rights reserved. 11
Copyright © 2016 CISQ. All rights reserved. 12
Join CISQ ! www.it-cisq.org
2/16/2017
7
CISQ-Standards in der Praxis für Test & Entwicklung
Steffen Ritter | CAST: Leader in Software Analysis and Measurement
CAST Application Intelligence Platform
Unit Leader Team Cost Date
1 MODEL
49+LANGUAGES
12+DATABASES
2 ANALYZE 3 MEASURE
ANALYSIS ENGINE
SOURCE CODE DEMOGRAPHICS
Analyses Driven Industry
Standards
• Single-technology
• Cross-technology
• Cross-language
• Cross-component
• Dataflow Analysis
• Transactional analysis
Automated
meta-model
Reverse engineered
through the
interactions of
components
through
heterogeneous
layers.
Curated Quality Assessment Model
Identified
violations are
evaluated for
risk
Violations are
categorized in
the context of
Health Factors
RESEARCH HOTSPOTS EXPLORE ARCHITECTURE
AED ADP & AAC
SEE THE BIG PICTURE
SYNTHESIS & ANALYTICS
INPUT PROCESSING
Automated
completeness
check
Code extraction
& formatting
Standards-Based Sizing
Rooted in
Industry
Standards
Application size
is measured in
Function Points
2/16/2017
8
Beispiel: Automatisierte Transaktionserkennung
Frontend HTML 5
Client-side Java Script
Java Code
Java Persistence API (JPA)
Datenbank
Wo Rauch ist… Brennpunkte objektiv priorisieren
Ausfallrisiko vs. Wichtigkeit der Anwendung für das Unternehmen
Applikations-EbeneObjektiver Portfolio-Gesamtüberblick
Heatmap aller Komponenten nach Risiken, Performance und Qualität
2/16/2017
9
CAST-Dashboard für Test Manager & Architekten
(1) Kritische Verstöße schnell identifizieren
(2) Risikoreichste Komponenten erkennen
(3) Regelverstöße verstehen
(4) Priorisieren und Aktionsplan entwickeln
(1) Kritische Verstöße schnell identifizieren
(2) Risikoreichste Komponenten erkennen
(3) Regelverstöße verstehen
(4) Priorisieren und Aktionsplan entwickeln
11
4433
22
Deep Dive auf Code-Level zu konkreten Transaktionen
2/16/2017
10
CAST-Dashboard für CIOs und IT-Management
CAST: 25 Jahre Erfahrung & 150 Millionen F&E
Hunderte internationale Konzerne bauen weltweit auf CAST
Nahezu alle großen ADM-Dienstleister nutzen CAST intern
Management-Beratungen nutzen und empfehlen CAST
Globale Systemintegratoren bieten Services auf Basis von CAST an
CGI
Kontakt: Steffen Ritter | Director Enterprise Sales
Herzog-Wilhelm-Str. 26, 80331 München, Tel: +49 89 215 89 441, E-Mail: [email protected]