kliment ppt gi2011_testing_remote_final
DESCRIPTION
GI2011-X-border-SDI/GDI-Symposium, 23..5.2011: Bad Schandau (SAX) [ 11. Sächsisches GIS-Forum ] 24.5.2011: Decin (CZE) [ 1. Bohemian#Saxonian GIS-Forum ]TRANSCRIPT
TESTING OF SDI COMPONENTS – A FUNDAMENTAL INTEROPERABILITY
ELEMENT WITHIN INSPIRE AND NATIONAL SDI’S
T. Kliment, M. Tuchyňa, M. Kliment
This presentation has been transmitted remotely as TELE lecture from Pallanza (IT) thanks to “BizBiz-Tool”
Presentation outlines
• Introduction• Basic description of involved organizations within testing
o Slovak Environmental Agency (SEA)o Slovak University of Technology (SUT)o Slovak University of Agriculture (SUA)o Proposed tasks related to SDI components testing
• SDI component to be provided by SEA to INSPIRE&NSDI• Testing methodology• Testing environment proposal• Testing tools • Pilot testing of the SEA SDI components • Testing results summary• Conclusions and future work
Introduction
• European and National SDI should provide data discovery, evaluation and consumption via central point – Geoportal
• Network services are engines to serve data and metadata from repository to central SDI Geoportal
• Network service behaviour is defined by standards& specifications (ISO, OGC) and specified by regulations (INSPIRE)
• Before the NS connection to geoportal testing has to be against above requirements has to be done to ensure interoperability
• Voluntary collaboration established under the auspices of SDI PTB for such purposes
• Similar activities are nowadays problematic due to missing such experience in mandated organization
• Therefore such type of collaboration should increase among all organizations involved in SDI establishment
Basic description of involved organization
• Slovak Environmental Agencyo Public sector bodyo Coordinating SDI development within environmental domain o Contributing to national and international SDI development
implementation via:Metadata system for spatial and non spatial data Network services (discovery, view, download,
transformation, spatial data services Consolidated spatial data repository (Central Geografical
System) Client applications (Environmental geoportal, domain
specific web map clients)o Providing technical support for INSPIRE transposition and NSDI
implementationo LMO & expert representation in IOC TF within INSPIRE
Basic description of involved organization
• Slovak University of Technology - Geodetic departmentso Academic sector organizationo Has a study program geoinformatics in master degree levelo Has defended 2 PhD thesis and has 4 in processing phase that
are related to GIS/SDI domain - data quality, data modelling, web services, GIS usage in specific domains (archaeology, floods, geodesy)
o Performs education and research within GIS/SDIo Implements web map clients, web map services and testing
toolso Performed initial testing of discovery and view services against
INSPIRE requirements presented at conferences (GIS Ostrava, EnviroIForum,
GI2010) SDIC & expert representation in IOC TF within INSPIRE
Basic description of involved organization
• Slovak University of Agriculture – Dept. of Landscape Planning and Ground Designo Academic sector organizationo Has a study program Ground design and GIS in master degree
levelo Dept. is a producer of huge amount of data about land as a
results from ground design projectso Currently the pilot project is related to design and
implementation of IS about hydro physical properties of soils in Slovakia based on SDI principles:o Metadata and discovery services o View and download serviceso Processing services for spatial analyses
SDI components provided by SEA to INSPIRE/NSDI
• SEA covers data themes from all 3 INSPIRE annexes• Annex I – Hydrography, Protected sites• Annex II – Land Cover • Annex III – Biogeo regions, Habitates&Biotopes, Species
distribution
Network service type
Annex I Annex II Annex III
Discovery service YES
(terra catalog CSW 2.0.2)
YES(terra catalog
CSW 2.0.2)NOT YET
View serviceYES
(ArcGIS Server WMS 1.3)
YES(ArcGIS Server
WMS 1.3)
YES(ArcGIS Server
WMS 1.3)
Download service YES
(ArcGIS Server WFS 1.1)
YES(ArcGIS Server
WFS 1.1)
YES(ArcGIS Server
WFS 1.1)
Maps served by view service (WMS) Data served by download service (WFS)
Metadata for datasets&services served by discovery service (CSW)
Testing methodology
Testing Coverage
Testing scope
Testing temporalextent
Testing scenarios
Testingperformance
Testing reporting
INSPIRE, ISO, OGC
Complex/partial testing model
Short/long term
Conceptual/applicationdesign, testing tool
Service interface/quality/content
Report template
Testing environment proposals act SUTSEAT_testing
SDI components
tester SDI components
prov ider
testingEnv ironment
publishNetworkServ ice
displayResults
Communicate
SetupTestingScenarios
deployTests
reportResults
SelectTestingScenario
sendReportdownloadReport
«invokes»
«invokes»
«invokes»
•Users – SDI component tester, SDI component provider
•Use cases – PublishNS, Define/ConfigureTestingScenario,
SelectTestingScenario, PerformTest, ReportResults, ViewResults
SendResults, SaveResults, Communicate,
Testing tools
• Webtest tool - web application for web service testingo Developed by testers at SUT with Java and JSP technologieso Web GUIo Single, multiple, simultaneous GET and POST requestso Measures times: Between RQ and the first byte of the RS download Between RQ and the last byte of the RS download
o Counts presence of predefined stringo Configuration based on XML files Service endpoint definition Testing scenario definition (request to service)
o Provides results in the tabular formo Does not provide statistics for long term testing yeto Online accessible on: http://geo.vm.stuba.sk:8080/webtest/
Testing tools - webtest
testcase.xml results
Testing tools
• MDValidator - desktop application for batch metadata validation
o Developed by testers at SUT as Java application
o Desktop GUI
o Performs batch validation of MD as XML files from local dir
o Invokes online REST Web INSPIRE validator service
o Provides results in XML/HTML form
o Does not validate against ISO gmd schema yet
o Useful after batch MD transformation (xslt)`
Testing realization - discovery service
• Testing coverage - INSPIRE requirementso Search criteria - 25 queryableso Operations - 4 operations with predefined parameterso Quality of Service - 3 parameters
• Testing scopeo Tested 3 operations (DiscoverMD, GetDSMD, PublishMD) and
23 queryableso Estimated 2 quality parameters (Performance, Capacity), MD
• Temporal extento Short term testing - all predefined testing scenarios launched
few times• Testing scenarios + tools
o One for queryables+performance+DiscoverMD operationo One for capacity as combination of operationso One for Publish MD operation+performanceo webtest+MDValidator
Testing realization - view services
• Testing coverage - INSPIRE requirementso Operations - 3 operations with predefined parameterso Quality of Service - 3 parameters
• Testing extento Tested 2 operations (GetMap, GetVSMD)o Estimated 2 quality parameters (Performance, Capacity)
• Temporal extento Short term testing - all predefined testing scenarios launched
once• Testing scenarios + tool
o One for GetMap operation+performance estimationo One for capacity as combination of operationso webtesto
Testing realization - download services
• Testing coverage - INSPIRE requirementso Operations - 4 operations + 2 more for spatial object accesso Quality of Service - 3 parameters
• Testing extento Tested 3 operations (GetSDS, GetDSMD, DescribeSDS)o Tested 2 parameters of QoS
• Temporal extento Short term testing - all predefined testing scenarios launched
once• Testing scenarios + tool
o One for GetSpatialDataSet+performance&capacity estimationo One for DescribeSpatialDS and GetDSMD+performanceo webtest
Testing results summaryReport template structure proposals 1/2
Tested service Discovery
service View
serviceDownload
service
Interface (operations¶mters)
Results
GetDSMetadata SupportedParameters no tested
SupportedParameters no tested
SupportedParameters no tested
DiscoverMetadata SupportedParameters no tested
PublishMetadata SupportedParameters no tested
LinkService No tested No tested No tested
GetMap SupportedParameters no tested
GetSpatialDataSet SupportedParameters no tested
DescribeSpatiaDataSet SupportedParameters no tested
GetSpatialObjectNo tested
DescribeSpatialObjectNo tested
Testing results summaryReport template structure proposals 2/2
Tested service Discovery
service View
serviceDownload
service
Quality of service Results
PerformanceSatisfied
(115 requests sent
115 responses < 3s)
Satisfied on 90%(30 requests sent 27
responses < 5s)
Satisfied(30/10 requests sent
30/10 responses <
10/30s)
Capacity Satisfied(30 simultaneousrequests sent 30
responses < 3s)
Satisfied on 70%(20 simultaneous requests sent 14
responses < 5s)
Satisfied(10 simultaneous requests sent 10
responses < 30s)
Availability No tested No tested No tested
Other criteria Results
Search criteria Supported23/25 criteria tested
Search Criteria for the Get Spatial Object Operation
No tested
Content of the service Results
Metadata modelsNo tested
Data modelsNo tested
Conclusions
• Initial short-term testing has brought first and quite positive results• Collaboration between public and academic sector is important• Testing methodology needs high knowledge and understanding of
all requirements defined in related regulations• Testing scenarios preparation is important and consumes much
time• Testing results should be easily interpreted and understandable• Where possible and appropriate various levels of compliancy can
be introduced
Future work
• Discussions, proposals, suggestions on Testing methodology• Testing scenarios extensions
o For all required operations (Link NS)o RQ + RS parameters validation against INSPIRE specific
constraints (e.g. NS INSPIRE MD within GetNSMD response, Language parameter, Layer MD, Spatial Dataset MD)
o Long term testing scenarios for accurate service quality parameters estimation
o documentation of individual scenarios (conceptual level)• Testing report template
o Discussions and decisions on the form and content of the reports - (e.g. tabular form with information as Date, Test description, Test execution, Results, Pass/Fail definition, comments ....)
Future work
• Testing environment extensionso Functions for reporting exports, statistics calculations, plotso Testing series implementation (e.g. INSPIRE discovery service
testing scenario, ...)o Results storage in form of database to avoid loss of results in long term
• Testing of local spatial data and services compliancy against INSPIRE Annex II+III data specificationso Feasibility testing (Annex III)o Fitness for purpose testing
• Efforts to promote testing & validation (awareness rising)• Efforts to formalise testing & validation (proposal for
establishemnt of common testing platform allowing sharing tools, materials, methodologies, experience and expertice related to SDI components testing)
Thank you very much!
Contact info:SUT in Bratislava, Dept. of Theoretical GeodesySEA in Banská Bystrica, Dept. of Environmental informaticsSUA in Nitra, Dept. of Landscape Planning and Ground Design
[email protected]@gmail.com