semantic based storage qos management methodology
DESCRIPTION
Semantic Based Storage QoS Management Methodology. Renata Słota, Darin Nikolow, Jacek Kitowski. Institute of Computer Science AGH-UST, Krakow, Poland ACK Cyfronet AGH, Krakow, Poland. Research supported by MNiSW grant nr N N516 405535. Outline. Introduction - PowerPoint PPT PresentationTRANSCRIPT
KUKDM’2011, Zakopane
Semantic Based Storage QoS Management Methodology
Renata Słota, Darin Nikolow, Jacek KitowskiInstitute of Computer Science AGH-UST, Krakow, Poland
ACK Cyfronet AGH, Krakow, Poland
Research supported by MNiSW grant nr N N516 405535
KUKDM’2011, Zakopane
Outline
Introduction
Common mass storage system model and ontology
Storage system performance monitoring and estimation
Use cases
Summary
KUKDM’2011, Zakopane
Introduction There are applications (called data intensive applications), which:
– use data storage systems intensively,
– Have constantly growing demands concerning capacity and storage system efficiency.
Example of applications dealing with huge amounts of data
– Scientific applications (simulations, out-of-core computations, HEP experiments),
– Backup & restore, archiving, disaster recovery.
The performance of the data intensive applications depend on the performance of the underlying storage system
Applications running in distributed environments need some QoS (Quality of Service) concerning data access
KUKDM’2011, Zakopane
Introduction (cont.) Storage QoS metrics include
– data access latency
– data access bandwidth
– storage space
– data availability
Diversity of storage systems
Hierarchical Storage Management (HSM) systems
Disk arrays
Tiered storage
Problem of efficient storage performance utilization while respecting the storage QoS constrains
Storage performance monitoring and data transfer scheduling are necessary
KUKDM’2011, Zakopane
Goal of research
The subject of this research is the development of semantic-based storage management methodology allowing to achieve QoS for storage performance metrics.
As part of this research the following has been done:
• Development of common mass storage system model (C2SM) and relevant ontology
• Development of storage performance monitoring sensors and estimators
• Implementation of two use cases demonstarting our approach
The research has been done within the OntoStor project http://www.icsr.agh.edu.pl/ontostor
KUKDM’2011, Zakopane
C2SM
Is a common mass storage system model, which can be used to describe the state of a storage system
Consists of
set of well defined storage performance related parameters
Algorithms specifying the storage system behaviour
• Is based on the Common Information Model – CIM
• Is used in our methodology for unifying the description of storage performance parameters of heterogenious storage systems
KUKDM’2011, Zakopane
C2SM class diagram
KUKDM’2011, Zakopane
OntoStor ontology
• Developed based on C2SM
• Has been created semi-automatically using the ‘cim2owl’ tool, which has been developed at the DCS AGH
• The ontology is used in our methodology to find storage resources using semantic queries
•
KUKDM’2011, Zakopane
OntoStor ontology diagram
KUKDM’2011, Zakopane
Storage System Performance Monitoring
• Two components have been defined in our methodology: – sensors – obtaining storage
performance parameters,• Storage system dependent
– estimators – estimating the future storage performance based on the data from the sensors.
• Simulational, rule-based, statistical
• Three types of systems supported – local disk, disk array, HSM system
Testbed
KUKDM’2011, Zakopane
OntoStor Portal
KUKDM’2011, Zakopane
Disk array monitoring
KUKDM’2011, Zakopane
Monitoring of loaded disk array
KUKDM’2011, Zakopane
Estimation test
KUKDM’2011, Zakopane
Use cases
Two use cases have been implemented using the proposed methodology
Data access optimization with replication
Finding the best location for newly created replica for write access
Selecting the best existing replica for read access Components used:
Sensors, estimators, monitoring system, C2SM SLA (Service Level Agreement) monitoring
Components used: Sensors, monitoring system, OntoStor ontlogy, QoS metrics
ontology, C2SM
KUKDM’2011, Zakopane
SLA monitoring
1. QoS metric limits
2. How to obtain values of QoS metrics
3. How to monitor performance parameters
4. Description of parameters
KUKDM’2011, Zakopane
Summary
• Semantic-based storage management methodology based on monitoring and estimation of storage performance has been presented
• Common mass storage model and its relevant ontology has been proposed
• A set of sensors and estimators has been implemented
• Two use cases have been implemented using the proposed methodology
• The proposed methodology has been used in FiVO/QStorMan.