gum-ho choe, korea laboratory accreditation scheme · pdf filekorea laboratory accreditation...
Post on 17-Mar-2018
233 Views
Preview:
TRANSCRIPT
Jan. 14, 2013
Gum-Ho Choe,
Korea Laboratory Accreditation Scheme (KOLAS)
Korean Agency for Technology & Standards (KATS)
Ministry of Knowledge and Economy(MKE)
Hong Kong ITC/HKAS Seminar
·
I. Nationally Recognized SW Program
II. Results of Survey on APLAC’s ABs
III. Accreditation Program in ABs
IV. Future Work of SW Testing
• Two Track System of Software Testing in Korea
• 1st ; Nationally Recognized Testing & Certification
• 2nd ; KOLAS Accredited Testing Laboratories
Process
Certification
ISO/IEC 15504(SPICE) CMMi SP(Software Process)
System
Certification
ISO 9001
TL9000
Product
Certification
SW Product Certification - GS Certification - Conformance Certification : RFID, Home Network etc.
Nationally Recognized SW Testing & Certification
SW product (include process doc.)
- CC Certification - SIL Certification (IEC61508) - V&V etc
1. System Certification
ISO 9001 certification
- to evaluate the provider’s quality system and admit quality guaranty capacity - to present 20 requirements for quality guaranty and test establishing quality system available to each customer.
System Certification
TL 9000 certification - TL 9000 is a quality management practice designed by the QuEST Forum. - It was created to focus on supply chain directives for the international telecommunications industry
- CB: NIPA(National IT Industry Promotion Agency)
- Process improvement approach that provides organizations with the essential elements of effective processes that ultimately improve their performance. - Can be used to guide process improvement across a project, a division, or an entire organization - SP is domestic certification in Korea based on ISO/IEC 15504. : for middle and small companies in Korea * IT Companies should be certificated to bid to public institutes.
: SP, CMMi, SPICE
Process Certification
SP Mark
2. Process Certification
Level 1
Level 2
Level 3 Process measured
and control & process
improvement
Process characterized for
project & organization
Processes poorly
controlled and reactive
Process improve
project level
Process improve
organization level
Maturity Levels(SP)
Product
Certification
GS(Good Software) Certification
CC(Common Criteria) Certification
SIL(Safety Integrity Level) Certification
V&V(Validation & Verification)
- Medical Area, Nuclear Area etc
Product Certification
Product Evaluation - except product development process - evaluate only the product itself
- GS certification
Evaluation of product including process documents
- overall lifecycle evaluation such as requirements specification, code and Execution file, so on. • Security certification : CC(Common Criteria) • Safety certification : SIL(Safety Integrity Level)
ISO/IEC 29119 IEEE 829 ISTQB Syllabus …
TMMi/TPI
Requirements Analysis
System Design
Architecture Design
Module Design
Coding
Unit testing
Integration Testing
System testing
Acceptance Testing
Software Testing and Related Standards
Measurement of Internal Quality
ISO/IEC 9126-1, ISO/IEC 25022
Measurement of External Quality
ISO/IEC 9126-2, ISO/IEC 25023
Measurement of Quality in Use
ISO/IEC 9126-3, ISO/IEC 25024
ISO/IEC 14598 series
Requirements Analysis
System Design
Architecture Design
Module Design
Coding
Unit testing
Integration Testing
System testing
Acceptance Testing
Software Measurement and Related Standards
SW Testing vs. Measurements
S/W Test
finding defects
providing confidence in the
product
providing insight in quality and
risks
Measurement
Identifying the Risks to System
providing quality in the product
using the test result
SW Measurements Methods on GS Certification
Informal
Risk Analysis
ISO 9126/25000 series
Quality Analysis
Failure Mode and Effect Analysis
Start with the classic quality risk categories
Start with six main quality characteristics
Start with categories, characteristics, or subsystems
Functionality, states and transactions, capacity and volume, data quality, error handling and recovery, performance, standards and localization, usability, etc
Functionality, Reliability, Usability, Efficiency, Maintainability, Portability(FRUEMP), then decompose into key subcharacteristics for your system
Key stakeholders list possible failure modes, predict their effects on system, user, society, etc., assign severity, priority, and likelihood, then calculate risk priority number(RPN)
Set priority for testing each quality risk with key stakeholders
Set priority for testing each subcharacteristic with key stakeholders
Stakeholders use RPN to guide appropriate depth and breadth for testing
GS Certification
기능성 신뢰성 사용성 효율성 유지보수성 이식성
적합성 정확성 상호운용성
보안성 준수성
시간반응성 자원효율성
준수성
Overall Quality Model
12 17 25 22 15 10
주특성
부특성
Requirement Basis Related Doc. note
System
Requirements
RFP 2-7D0-J222-001
56 M/W Technology
Selection Criteria
Technical
Documents
(Company)
SW Function User Manual 138
Quality
Requirements
ISO/IEC 9126-2, 25051
Evaluation Module for Enterprise S/W
62
Total Requirements are 286.
Example- Inchon Airport Information System
16
Focusing on Industries of SW Testing
Information
Technology
Convergence
Automobile
Aerospace
Telecommunication
Construction
Healthcare Defense
Shipbuilding
Machinery
Testing & Certification Area of IT Convergence
Conformance IEC 61850 etc
Sector Standards
Reliability
IEC 60068 etc
Safety ISO/IEC 26262 IEC 61508 etc
Interoperability Sector Standards etc
Security ISO/IEC 15408
Sector Standards etc
Quality/Performance IEC62278, ISO/IEC25000
Sector Standards etc
Product/ System
ABs ACLA
SS A2LA BoA HKAS IANZ IAJAPAN KOLAS NATA SAC SCC SM CNAS
1. Testing
Labs o(2) o(11) X X X o(6) o(9) o(15) X X o o(80)
2. Guides o o o o o o o
3. SR ? NA NN o o o o o
4. Experts,
etc NN NA o NA o NA o o o
1. Accreditation service,
2. ISO/IEC 17025 + Supplement requirement(SR) are enough
3. Necessity of comprehensive SR
4. Experts; Workshop, Training, etc
o : Yes, X : No, ? : Hard to answer, NA : No Answer, NN : Do Not Need, * NVLAP (34), NABL(1)
* SM wants to take training course by leading ABs such as A2LA, NVLAP, NATA, etc., and hold
workshop
. Circulated through 34 ABs : 12 ABs answered ( 35 % : answering rate)
Current Status
KOLAS has accredited 9 testing laboratories for SW
Accredited Laboratories
No. Lab. Name Accreditation
Date
Accreditation Scope
(Sub Major Discipline)
KT 005 Korea Testing
Certification
June 10, 2008 Embedded SW for Smartcard
KT 009 Korea Testing
Laboratory
September 30,
2010
SQuaRE, COTS, Common Criteria
for IT Security Evaluation
KT 519 Financial Security
Agency
March 19, 2012 ISO/IEC 25051
ISO/IEC 9126-2
Accredited Laboratories
No. Lab. Name Accreditation Date Accreditation Scope
(Sub Major Discipline)
KT 167 Telecommunications
Technology
Association
May 14, 2009 Common Criteria for IT Security
Evaluation
KT 327 Korea System
Assurance
June 27, 2007 Common Criteria for IT Security
Evaluation
KT 402 Korea Security
Evaluation Laboratory
Co. Ltd.
April 24, 2009 Common Criteria for IT Security
Evaluation
KT 448 ICT Korea Ltd. April 26, 2010 ID Card; ISO/IEC 7816-3, 10373-3,
ISO/IEC14443-2~-4
KT 463 KISA November 16, 2010 ISO/IEC 24709-1(IT-Conformance
testing for the biometric application
programming interface
KT 122 SGS Korea Aug. 23, 2012 IEC 61508-3 (Functional Safety)
ISO 26262-6 (Safety Management
System)
25
• CC Certification
• CC Consulting
• Smart Grid and
Industrial Network
Security Test
Security Evaluation
Team
• GS Certification
• SW KOLAS(ISO/IEC
17025)
Testing Service
• Process Certification
(SPICE, CMMi)
SW Evaluation
Team
IT System Evaluation
Team
• IEC61508 (Safety) SIL
certification
• Mission-Critical SW
performance and
Reliability Test
• SW V&V etc
Major Work of KTL (Korea Testing Laboratory) Recognized & Accredited
by MKE & KOLAS, Respectively
KT 009
Information Technology Security Testing
Common Criteria Testing ; 7 labs
Cryptographic and Security Testing ; 21 labs (foreign 11 EA)
Healthcare Information Technology Testing ; 5 labs
U.S. A. NIST NVLAP
Information and Communications Technology
NATA's Information and Communications Technology field
provides accreditation for a diverse range of software and
hardware testing in such areas as gaming systems,
information security evaluations and healthcare software.
* 15 labs accredited
1. Focusing on Product Quality and Security
2. Extending to Safety Management System &
Healthcare Information Technology
Extending to Safety Management System (SMS)
Based on ISO 26262, Technical Regulations, etc.
Current Status and Future Work
Based on CC, ISO/IEC 9126, 25000, and IEC 61508, etc
Embedded SW in fields of IT convergence
Sep. 12, 2011
Gum-Ho Choe, Convenor
Working Group on Accreditation of Software Testing,
APLAC Technical Committee
Reference 1
Dec. 3, 2012
Gum-Ho Choe, Convenor
Working Group on Accreditation of Software Testing,
APLAC Technical Committee
Reference 2
top related