cobweb - infrastructure and platform for environmental crowd sensing and big data
TRANSCRIPT
COBWEB
EnviroInfo & ICT4S 2015,
Workshop - Infrastructures and Platforms
for Environmental Crowd Sensing and
Big Data,
Copenhagen,
9th Sept, 2015
Chris [email protected]
Sta
Introduction to COBWEB
• Research Project: Funded under the European Commission’s Framework Programme 7
• Started Nov 2012 for 4 years (Month 35 of 48)
• First demonstrator completed and tested during the 2015 field season
• Why?
– GPS enabled, internet connected mobile devices now ubiquitous
– Lots of potential, eg, can citizen sourced environmental data be useful for decision making?
Citizen Observatory Web
• Generic crowdsourcing infrastructure
– A toolkit which can be downloaded and used
in multiple scenarios
• Data which supports policy
• Address data quality issues
• Open standards
Project Partners
UNESCO World Network of Biosphere Reserves
Sites of excellence to foster
harmonious integration of people and
nature for sustainable development
through participation, knowledge
sharing, poverty reduction and human
well-being improvements, cultural
values and society's ability to cope
with change, thus contributing to the
Millennium Development Goals
COBWEB Biosphere Reserves
Biosffer Dyfi Biosphere
Mount Olympus
Gorge of Samaria
Wadden See & Hallig islands
Dyfi Biosphere Reserve #1
www.biosfferdyfi.org.uk
Dyfi Biosphere Reserve #2
Contains Ordnance Survey data © Crown copyright and database right (2013)
©C
ou
ntr
ysid
e C
ou
ncil
fo
r W
ale
s. A
ll r
igh
ts r
eserved
.
Co-design
Co-design – Snowdonia National Park
• Japanese Knotweed - Fallopia japonica
• Giant Knotweed – Fallopia sachalinensis
• Hybrid knotweed – Fallopia x bohemica
Co-design – RSPB
• Domenlas Saltmarsh
– Survey vegetation
– Use quadrats
– Soil salinity levels
– Weather conditions
– Look at reversion process
• Covert Coch Peatbog
– Survey vegetation
– Use quadrats
– Soil moisture
– Grazing history
Janetbaxter.co.uk
Co-Design - Penparcau Community Forum
Co-Design - Penparcau Community Forum
• Temperature
• Humidity
• Wind speed
• Wind direction
• Pressure
• Precipitation
In situ sensors
Usability Testing
Not just apps…
A number of demonstrator mobile phone
applications
– Exactly what, deliberately left open and
subject to discussion with community
3 pilot case study areas:
1. Validating earth
observation products
2. Biological monitoring
3. Flooding
COBWEB Framework
Key components at different TRL’s
• Conflation
• QA workflow editor
• QA WPS/services
• Sensor networks
• GeoNetwork/Portal
• Middleware
• Authoring tool/Survey designer
• Apps
• User management and privacy
• Access control
• Authentication
The app itself
Key features
• Capture information
– Images
– Audio
– Text
– Location
• High quality background maps
• Saved maps for use “offline”
• Custom data collection forms
• Manual location correction
View COBWEB Portal
Join COBWEB Portal
Customise your own app for your survey
Customise your own app for your survey
Customise your own app for your survey
Citizen captures data on their phone
Approach to QA
• Quality Assurance: complex problem and often use case specific:
– Internal quality (metrics)
– External quality (fit for purpose)
• A generic system that can be easily customised to fit new use cases, based on standards
• Understanding data quality is likely to require a combination of approaches and tests, the system is designed to enable this
• As quality requirements are use case specific, the quality control tests are configurable to utilise a wide variety of datasets and parameters
Classifying quality: Seven pillarsPillar Example Test Notes
Pillar 1 – Location Based services
Assessment of spatial accuracy – estimate from a mobile device and number of satellites
Tests often carried out on the mobile device
Pillar 2 – Cleaning Removal of junk data via an attribute text check
Very lightweight, can flag or remove malicious entries
Pillar 3 – Automatic validation
Analysis whether an image is blurry
Higher level testing, often used to assess ranges
Pillar 4 – Comparison with authoritative data
Use of a set of boundary polygons to check whether an observation is in or out
Wide variety of tests that involve comparison with what it known
Pillar 5 – Model based validation
Running a flood model Can be complex, and may also include question based modeling
Pillar 6 – Big/Linked data Querying Twitter via a hashtag for similar phenomena
Tapping into large databases such as sensor records and social media
Pillar 7 – Semanticharmonisation
Rationalisation of entries via an ontology
Attempts to recognisemultiple entries of the same observation
Web Processing Service
• OGC Standard
• Web facing
• Public
• Holds spatial and non-spatial processes
• Processes are suited to chaining
• BPEL is the traditional method of chaining
– However, removed from WPS 2.0
BPMN Workflow Engine
• JBOSS JBPM – A Java based BPM workflow engine
• Orchestrates processes in a given order with defined inputs
• Chains results from one process to the next
• Can be executed remotely via REST
• Graphical interface utilises BPMN2.0 – a recently ratified OMG standard
• Unlike BPEL this interface is standardised
Current questions
• What is the Post Quality database. How to manage different schemas and different data sources?• RDF to handle the fuzzy view of crowdsourced
data?• How is client side QA being managed?
• GeoPackage + watching service?• Client side vs server side
• Security• Can we protect certain datasets/processes and
make them available to those who have the correct credentials?
• Many others!!
For more information on QA – see:
• Relevant sessions at the OGC Technical Committee in Nottingham next week
• Architecture Implementation Pilot – 8 deep dive videos:
http://www.ogcnetwork.net/node/1969
Service Provider (SP) Identity Provider (IdP)Discovery Service (DS)
“GEOSS user” Single-Sign-On
Trust Gateway (TG) to OpenID
Google OpenId
COBWEB/GEOSS AIP-6 Federation
NASA Ames
Secure Dimensions
CUAHSI*Catapult
University of Edinburgh
Kst. GDI.DE
*: Consortium of Universities for the Advancement of Hydrologic Science
EarthServer (FP7) project
MEEO
Remainder of project…emphasis shifting
• Rolling out software
• Getting greater buy-in
• Sustainability
• More focus on research agenda
• Greater focus on Greece, Germany
• Additional Biosphere Reserves
Thank you
• Website: http://cobwebproject.eu/
• Mailing list: http://eepurl.com/Ita05 or use the QR code
• Follow us: @cobwebfp7
• And look out for members of the COBWEB team at events.