environmentally optimized irrigation (enorasis) with istsos: project overview and next challanges
TRANSCRIPT
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
ISTSOS in support of ENORASIS for optimizing irrigation
This project has received funding from European Union’s Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No 282949
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
ENORASIS
The project ENORASIS is a FP7-ENV project (Grant Agreement 282.949). The project duration is three years (January 2012 – December 2014).
Objective Develop an integrated decision support system for environmentally optimized and thus sustainable irrigation management for farmers and water management organizations based on advanced technologies and models.
2 DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Partners
DACD / IST / ISTSOS in support of ENORASIS
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
The idea
• New EU policies in the field of water and agricultural policies introduce the principle of “the user pays" in irrigation water management – WFD (Water Framework Directive) – CSS (Cross Compliance Scheme) of the EU CAP
(Common Agricultural Policy).
• New tools and technologies allow the usages of
granular information on the present and future state of the system: – High resolution weather prediction models – Low cost sensors
• New devices and spread of the Internet allow easy
access to information for decision-making more conscious
4 DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Desktop and Mobile Application
16 July 2014
5
WHEN, WHERE, HOW MUCH IRRIGATE Users can access the ENORASIS system mainly by a web browser. The web platform is used by all the system actors (e.g. farmers, watering authorities, consultants and administrators) to perform their daily tasks. Most of the functionalities offered by the web application are also provided by the mobile application, developed for Android smartphones.
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Pilots results
16 July 2014
6
• Validation and assessment of ENORASIS solution against specified Key Performance Indicators (KPIs).
• The pilots are Implemented to cover several (8) crops types, multiple (4) geographical areas of different climate characteristics, and different (3) operational settings.
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Plots results (potatos, results from IUNG Poland)
7
Yield of potato t/ha bulbs affected by diseases [%]
Water consumption per unit yield Income in depending of production system
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Sensor data Subsystem: Data Fusion
9
FIELD 1
FIELD N
FIELD 2
Sensor
Gateway / Router
Coordinator / Seller
ENORASIS SYSTEM
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
The software
Open Source Software (GPL v2) Entering Incubation:
10
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
istSOS one enabler of the IOT
11
The Internet of Things (IoT) is the collection of billions of end devices intelligently connected and interoperating with servers and services.
Combine “Things” like Lego blocks to create applications, services, values….
How to chose the blocks? what are they? how do they fits together? We need well known language: standards for machine – machine interaction!
SOS is an OGC standard to: • register new sensor (THING) to the Web, • storing new observations • retrieving desired data
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
The Sensor Observation Service standard
12
From http://www.opengeospatial.org/standards/sos “The Sensor Observation Service standard is applicable to use cases in which sensor data needs to be managed in an interoperable way. This standard defines a Web service interface which allows querying observations, sensor metadata, as well as representations of observed features. Further, this standard defines means to register new sensors and to remove existing ones. Also, it defines operations to insert new sensor observations. This standard defines this functionality in a binding independent way; two bindings are specified in this document: a KVP binding and a SOAP binding (in version 2.0).”
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
The istSOS software
14
mod_wsgi
isodate psycopg2 pytz
Database
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Easy installation and initialization and management using GUI (WebAdmin Interface)
15
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Various output format using GetObservation request
16
XML – CSV - JSON
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Aggregation on-the-fly with GetObservation request
17
AVG, SUM, MAX, MIN Extra parameters: aggregateInterval = PT1H & aggregateFunction = AVG
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Real Time Quality Check
Configuration of quality index on the observed property level or sensor specific. Automatically data quality check assigning quality index on each observation based on specific constraints
18
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Raw data Assignment of quality indices during insertion:
Quality index example
19
Raw data
Meaningful data (Leve 1)
Statistically correct (Level 1 + 2)
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Time-Zone support
eventTime 2013-01-01T01:00:00+01 / 2013-01-07T01:00:00+01
eventTime 2013-01-01T02:00:00+02 / 2013-01-07T02:00:00+02
20th September 2013
20
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Acquisition Job Scheduler (New)
With Advanced Python Scheduler (APScheduler, embedded in istSOS) you can schedule sensor data to be inserted at times of your choosing
16 July 2014
21
Sensor File Transfer of Raw Data Server
Scheduler Data acquisition istSOS
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Virtual procedures (EVAPOTRANSPIRATION)
22
GRABOW
Real Sensor
Air Temperature Relative humidity
Wind velocity Solar radiation
V_GRABOW
Virtual procedure
Reference evapotranspiration FAO56 Calculation
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Status Page (New)
16 July 2014
23
Sensor status at a glance taking in account acquisition delay and inserting errors
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
istSOS Alert & Notification Service (New)
16 July 2014
24
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
JSON RESTful API
RESTful API for accessing and managing istSOS: • Get service status, test database connection, initialize new servces • Get, insert, update and delete:
– sensors, – offerings, – observed properties, – quality indexes, – unit of measures, – service metadata, – observations, – fetaures of interest, – supported epsg, – etc…
25
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
WALib supporting Sensors Mapping and Charting Observations
26
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
SERVICE STATISTICS: ENORASIS SOS
2xx Success
5xx Server Error
3xx Redirection
4xx Client Error
France
Serbia
Turkey
Poland
Italy
France 2709547 1.26 GiB Serbia 1466967 0.92 GiB Turkey 464619 275.16 MiB Poland 161054 111.57 MiB Italy 24416 23.17 MiB
TOTAL 4’826’60
3 2.6 GiB
ONE WEEK USAGE (4 plots) ONE WEEK ERRORS TYPE # %
2xx Success 2’413’666 99.9671 5xx Server Error 785 0.0325 3xx Redirection 7 0.0003 4xx Client Error 2 0.0001
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
SERVICE STATISTICS: Hydro-Meteo SOS
ONE MONTH USAGE
ONE MONTH OVERALL ERRORS
BANDWIDTH REQUESTS ISTSOS RAW 7.2 GiB 1'753'187 ALL ISTSOS 20.06 GiB 5'130'320
Procedures
Real Sensors 152 Virtual Procedures 42
5128307 99.96% 2xx Success 5128307 99.96% — 200 - OK: The request sent by the client was successful
1940.2 0.04% 5xx Server Error
1599.9 0.03% — 502 - Bad Gateway: Received an invalid response from the upstream
322.9 0.01% — 503 - Service Unavailable: The server is currently unavailable 17.4 0.00% — 500 - Internal Server Error
66 0.00% 3xx Redirection 66 0.00% — 301 - Moved Permanently: Resource has permanently moved
6.9 0.00% 4xx Client Error
6.5 0.00% — 404 - Document Not Found: Requested resource could not be found
0.3 0.00% — 400 - Bad Request: The syntax of the request is invalid 0.1 0.00% — 405 - Method Not Allowed: Request method not supported
Observations 41'930'452
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Big data issue
• Deploying the system worldwide we can immagine million of sensors with bilion of centralized data to be served and ingested
• We need a scalable and adaptable and fast: a solution for big data !
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
istSOS 3: enorasis edition
• Implemented and currently under testing
• Components: – Asyncronous programming – Non blocking services – NoSQL database (scalable, fast, cloud oriented)
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Roadmap
istSOS lib: • Became an OSGeo project (incubation) • Evaluate and test istSOS3 Web Admin: • Multiple observed properties on the same charts • User authentication and autorization data access • Raw data loading from the web interface (CSV, XLS…) • Integrated map viewer Others ideas: • Android client API Library • One click installer integrated with web server • Extend support for SOS version 2.0 with INSPIRE requirements
20th September 2013
31
DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation
Thank you Any questions?
Source Code & Download https://code.google.com/p/istsos/
Mailing list:
http://groups.google.com/group/istsos [email protected]
Home page https://geoservice.ist.supsi.ch/projects/istsos
http://www.enorasis.eu
http://geomatica-supsi.blogspot.ch/
http://geoservice.ist.supsi.ch/+geomatics