rflex wfs api reference 13.04 -...

FerryScope D1.2

Rflex WFS API Reference Version 1.0 13.04.2015

Kervinen, M.1, S.G.H.Simis1,2, V.Keto1,

M.Boettcher3 1Finish Environment Institute (SYKE) 2Plymouth Marine Laboratory (PML) 3Brockmann Consult Gmbh (BC)

BONUS Innovation 2012 Grant identifier: BONUS call2012inno-38

FerryScope D1.2 Rflex WFS API Reference Finish Environment Institute

ii v1.0, 31.03.2015

Change log

Version Date Revised by Change Authors

0.1 07.08.2014 - First internal release Kervinen, Simis, Keto

0.2 30.09.2014 - Introduction and data description

Corrections to ship codes

Simis

0.3 07.10.2014 - Finished edits to document format, added table of contents, introduction, data fields description, and data filtering procedures, references.

Content complete

Simis

0.4 31.03.2015 - Format with Deliverable template Boettcher

0.5 09.04.2015 - General project summaries Simis

1.0.1 09.04.2015 - Added WFS client information Boettcher

1.0.2 10.04.2015 - Added description of new combined algaline+ferrybox feature type

Kervinen

1.0.3 13.04.2015 - Added field descriptions of new feature type Simis

1.0.5 16.04.2015 Boettcher D1.2 release for BONUS upload -

This report may be cited as:

Kervinen, M., S.G.H.Simis, V.Keto, M. Boettcher. 2015. Rflex WFS API Reference. FerryScope D1.2, Version 1.0

FerryScope: Bridging the divide between satellite and shipborne sensing for Baltic Sea water quality assessment

Consortium:

Brockmann Consult GmbH (BC), Germany

Finish Environment Institute (SYKE), Finland

Estonian Marine Institute (EMI), Estonia

Grant identifier:

BONUS call2012inno-38

Finish Environment Institute FerryScope D1.2 Rflex WFS API Reference

v1.0, 31.03.2015 iii

Table of Contents

1 Introduction ................................................................................................................................. - 1 -

1.1 Purpose and scope .............................................................................................................. - 1 -

1.2 References ........................................................................................................................... - 1 -

1.3 Terms and acronyms ........................................................................................................... - 1 -

1.4 Document overview ............................................................................................................ - 1 -

2 Overview ...................................................................................................................................... - 2 -

2.1 The FerryScope project ....................................................................................................... - 2 -

2.2 Rflex hyperspectral reflectance measurements.................................................................. - 2 -

3 The Rflex data service .................................................................................................................. - 3 -

3.1 WFS Characteristics ............................................................................................................. - 3 -

3.2 Data served by the WFS ...................................................................................................... - 4 -

3.3 Ship codes ............................................................................................................................ - 4 -

4 Graphical user interface to Rflex data ......................................................................................... - 5 -

4.1 Interactive WFS query tool .................................................................................................. - 5 -

4.2 Interactive WMS service ...................................................................................................... - 5 -

5 Machine-to-machine interface .................................................................................................... - 6 -

5.1 Retrieval of archived Rrs observations ................................................................................. - 6 -

5.2 Retrieval of latest Rrs observations ..................................................................................... - 6 -

5.3 Incremental queries for retrieving large datasets with RflexAPI ........................................ - 6 -

5.4 WFS Command line client for systematic Rrs retrieval ....................................................... - 7 -

6 Data structure of Rrs observations ............................................................................................... - 8 -

6.1 FeatureType rflex:RrsMeasurement ................................................................................... - 8 -

6.2 rflex:RrsFerryboxMeasurement .......................................................................................... - 9 -

6.3 Decoding the Rrs data ........................................................................................................ - 11 -

7 Known issues ............................................................................................................................. - 12 -

Appendix A Rflex Client for Python (retrieve-rrs.py) ......................................................................... - 13 -


v1.0, 31.03.2015 - 1 -

1 Introduction

1.1 Purpose and scope

This document describes the functionality of the FerryScope Web Feature Service which serves a

combination of remote-sensing reflectance (Rrs) and time-matched flow-through ferrybox data. It is

intended as a quick reference to start harvesting data from this service. The service is hosted by

SYKE. BC operates a client that uses the data in a matchup flow against satellite imagery.

1.2 References

The following documents are referenced in this document.

ID Title Issue Date

[Simis

and

Olsson

2013]

Simis, S. G. H., and J. Olsson. 2013. Unattended processing of shipborne hyperspectral reflectance measurements. Remote Sens. Environ. 135: 202–212. [doi: 10.1016/j.rse.2013.04.001]

2013

1.3 Terms and acronyms

The following acronyms are used in this document.

Acronym Expansion

OGC Open Geospatial Consortium

WFS Web Feature Service

MSFD Marine Strategy Framework Directive

1.4 Document overview

After this formal introduction

Section 2 Overview of FerryScope, Rflex, and the Rrs data service.

Section 3 Defines the Rflex data service, access points, and data served

Section 4 Describes the graphical user interface to access Rrs data

Section 5 Describes the WFS machine-to-machine interface

Section 6 Defines the data structures of the features provided by the service

Section 7 Lists known issues of the service


- 2 - v1.0, 31.03.2015

2 Overview

2.1 The FerryScope project

FerryScope addresses two major concerns in the remote sensing of optically complex waters: (1) high

ambiguity in inverting remotely sensed signals to water quality parameters, and (2) significant

uncertainty in the atmospheric correction of the top-of-atmosphere recorded radiance. FerryScope

links highly automated water quality parameter measurements collected from merchant vessels at 3-

5 m depth with high-volume processing of satellite data, to produce regional water quality estimates

with unprecedented product confidence. SYKE develops the in situ observation network. Brockmann

Consult provides automated matchup-analysis and develops tools for user uptake of combined

satellite and in situ data. EMI develops demonstration products and leads research tasks that

increase our understanding of the bio-optical variability in the sea.

2.2 Rflex hyperspectral reflectance measurements

The Baltic Sea already has an extensive network of ship-based monitoring from 7 ferry transects,

covering all Baltic Sea basins. These systems sample at approximately 200 m horizontal resolution

from 3-5 m depth, which is under all but severely stratified conditions representative of surface

waters. Temperature, salinity, turbidity, and fluorescence by phytoplankton pigments chlorophyll-a

and phycocyanin (in summer) are obtained. Quality assurance of the existing ferrybox measurements

as well as laboratory-based analyses (nutrients, phytotaxonomic analysis) is supported with

automated water sample collection. The ferryboxes are thus an ideal platform to delineate horizontal

patterns in several MSFD indicators such as eutrophication, biodiversity, and water transparency.

Recently, a hyperspectral remote-sensing reflectance (Rrs) platforms was developed for automated

deployment in the FP7 PROTOOL project (Figure 2-1-Figure 2-2). A first system was tested in ESA-

Balmon while FerryScope implements the system on two ferries covering long routes across the

Baltic Sea, both spending 4-5 days per week at sea. FerryScope further improves the automation

aspect of data quality control and develops a set of data interfaces, which are described in this

document.

Figure 2-1 Rflex platform installed on M/S Finnmaid (Finnlines)

Figure 2-2 Rrs spectra collected with Rflex on the Baltic Sea


v1.0, 31.03.2015 - 3 -

3 The Rflex data service

This section defines the Rflex WFS and the data served by this service. A schematic overview of Rflex

and subsequent processing is shown in Figure 3-1.

Figure 3-1 Schematic overview of the Rflex platform (bottom section) and subsequent data flows.

3.1 WFS Characteristics

The Rflex WFs service can be accessed at

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS

Data is served in two different feature types:

rflex:RrsMeasurement contains all Rflex observations which have passed the quality control.

rflex:RrsFerryboxMeasurement contains the combined simultaneously Rflex and ferrybox

observations.

A description of the feature types can be downloaded using the following request:

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=DescribeFeatureType&v

ersion=2.0.0

The service supports only single coordinate reference system (EPSG:4326) and single output format (GML). The number of returned features per request is limited to 5000.

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=DescribeFeatureType&version=2.0.0

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=DescribeFeatureType&version=2.0.0


- 4 - v1.0, 31.03.2015

3.2 Data served by the WFS

3.2.1 Rrs data

All Rrs data passing automated quality control are accessible by the service. The quality control implies that a

suitable solution was found for the value of the skylight reflectance at the water surface. The ‘fingerprint’

algorithm used for this procedure is described in Simis and Olsson (2013) and not discussed here. The features

offered by the service have a few additional filters in place, these are:

Ship_id > 1

Ship code 1 refers to data collected on RV Aranda (see also Ship Codes below). These include

observations that are part of the experimental phase of Rflex, and not shared publicly.

FP_EdPAR > 150

PAR downwelling irradiance (in µmol photons m-2 s-1) should be higher than 150. Poor quality spectra are

very common under ‘dark’ conditions. They are not offered through the service.

FP_nind > 5

The number of spectral features used in calculating Rrs through the fingerprint algorithm (Simis and

Olsson 2013) should be higher than 6. This reduces the number of Rrs spectra with large blue or red

tilts being offered through the service.

3.2.2 Alg@line ferrybox data

Ferrybox flow-through data is served only when a simultaneous Rflex measurement exists which has

passed quality control steps described in Section 3.2.1. The Rflex and flow-through instruments are

not synchronized, therefore the ferrybox data in a 40-sec window around each Rrs observation are

retrieved. Mean and standard deviation of the ferrybox measurements are then combined with Rrs

data and served as a single object. The combined product is therefore only available when both sets

of intruments have succefully made an observation.

The ferrybox data are not quality controlled. The observed values are the result of annual

intercalibration of the sensors in the Alg@line network.

3.3 Ship codes

M/S Finnmaid: 23 Data collected since July 2013. Normally operating April – September. Deployment in 2015 on 10/4/2015 following docking of the ship.

M/S Transpaper: 33 Data collected since July 2014. Normally operating April – September. Deployment in 2015 from late April, due to change of route.


v1.0, 31.03.2015 - 5 -

4 Graphical user interface to Rflex data

The SYKE Rflex service homepage http://ferryscope.ymparisto.fi/Rflex provides two tools for non-admin

users: a WFS query tool and a simple web map application.

4.1 Interactive WFS query tool

The WFS query tool is accessed at http://ferryscope.ymparisto.fi/Rflex/console/client/index.xhtml or through

the “send requests” link on the home page.

The WFS query interface contains example queries for the WFS service in the drop-down menu.

Queries can be edited and executed in the web browser and the results are displayed inline in the

browser window. Use this interface to optimize ad-hoc queries before automating them.

4.2 Interactive WMS service

The web map can be used to visualize the WMS service which provides the latest observation

locations (Figure 4-1). The result set is limited to the 3000 latest measurements for the selected area.

The WMS is accessed at http://ferryscope.ymparisto.fi/Rflex/console/webservices/wms/wms.xhtml or through the

“show layers” link on the Rflex home page. To show the ship locations, activate the “Rflex Ship

Locations” – overlay on the top-right corner of the map viewer.

Figure 4-1 Screen capture of the WMS service

http://ferryscope.ymparisto.fi/Rflex

http://ferryscope.ymparisto.fi/Rflex/console/client/index.xhtml

http://ferryscope.ymparisto.fi/Rflex/console/webservices/wms/wms.xhtml


- 6 - v1.0, 31.03.2015

5 Machine-to-machine interface

The Rflex WFS is the machine-to-machine interface to the service. This section explains how to use

this interface to retrieve Rrs records.

These examples use stored queries (GetRrs, RflexAPI) that return rflex:RrsMeasurement feature

types. To query for the combined Rflex and ferrybox data in rflex:RrsFerryboxMeasurement feature

type, use stored queries GetFerrybox and RflexFerryboxAPI, respectively. The parameters and usage

of the corresponfing stored queries are identical, only the returned feature type is different.

5.1 Retrieval of archived Rrs observations

The main API is implemented as a stored query named RflexAPI. It is recommended to use this stored

query for filtering the requested datasets as this guarantees proper behaviour of the service.

However, ad-hoc queries can also be used but they are not covered by this document. The required

parameters of RflexAPI are described here:

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=DescribeStoredQueries&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:RflexAPI

To perform a query with RflexAPI, all parameters are required, for example

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:RflexAPI&ShipCode=23&fromDate=2013-01-10&toDate=2013-12-10&upperCorner=59%2018&lowerCorner=50%2026&count=10

5.2 Retrieval of latest Rrs observations

In addition to RflexAPI, the service contains another stored query called GetRrs which can be used to

retrieve the latest observations filtered only by ship code:

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=DescribeStoredQueries&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:GetRrs

To get the 10 latest processed measurements from ship #23 (Finnmaid), use

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:GetRrs&ShipCode=23&count=10

5.3 Incremental queries for retrieving large datasets with RflexAPI

The maximum number of returned observations is limited to 5000 to enforce users to implement the

service efficiently in downstream processing chains. WFS 2.0.0 parameters resultType, count and

startIndex can be used to retrieve larger dataset in smaller parts.

First obtain the number of hits on a query by using resultType=hits:

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery

_id=urn:x-inspire:storedQuery:RflexAPI&ShipCode=23&fromDate=2013-01-10&toDate=2013-12-

10&upperCorner=59%2018&lowerCorner=50%2026&resultType=hits

Slice the data request into suitable blocks, note that startIndex starts at 0

First block, 100 first entries










http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:RflexAPI&ShipCode=23&fromDate=2013-01-10&toDate=2013-12-10&upperCorner=59%2018&lowerCorner=50%2026&resultType=hits




v1.0, 31.03.2015 - 7 -

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery


10&upperCorner=59%2018&lowerCorner=50%2026&count=100&startIndex=0

Second block, only 24 entries left but count=100 can still be used http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery


10&upperCorner=59%2018&lowerCorner=50%2026&count=100&startIndex=100

Retrieve full set at once for comparison http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery


10&upperCorner=59%2018&lowerCorner=50%2026

5.4 WFS Command line client for systematic Rrs retrieval

A command line tool can be used to retrieve Rrs data from the FerryScope WFS and store them into

CSV tables on the local machine. The command line tool is implemented in Python (2.7). It is included

in the Annex of this document, and it can be downloaded from http://ferryscope.org/web_content/retrieve-

rrs.py . It can be called like this (example):

retrieve-rrs.py 23 2014-07-01 2014-07-31 rrs-23-2014-07.csv

In this call retrieve-rrs.py <ship> <start> <stop> <destfile>

23 is the ship identifier

2014-07-01 is the start date as YYYY-MM-DD

2014-07-31 is the stop date as YYYY-MM-DD

rrs-23-2014-07.csv is the file name of the newly created file with the rrs data as comma-

separated values table

The CSV file contains one comment line with the date format, one headline with column titles, and as

many lines as there are Rrs measurements in the time period from start to end. Example:

#dateFormat=yyyy-MM-dd'T'HH:mm:ss

ID TIME LATITUDE LONGITUDE SHIP

Sun Elevation edpar skyrat400 Rrs Pitch Rrs 1st wl Rrs lst wl

Rrs 001 (320.0) Rrs 002 (323.3) Rrs 003 (326.6) ... Rrs 192 (950.3) Rrs 193 (953.6)

RFLEX_RRSMEASUREMENT_2320140701062720 2014-07-01T06:27:20 54.879614 13.417840 23

30 270.0805 0.09542297 3.3 320.0 953.6

nan -0.0011299575537 -0.00103880580933 ... 0.00126361090397 nan

RFLEX_RRSMEASUREMENT_2320140701062820 2014-07-01T06:28:20 54.881690 13.428385 23

30 280.6569 0.0933562 3.3 320.0 953.6

nan -0.0028666256803 -0.00271040635345 ...

…

Figure 5-1: Example CSV file retrieved with retrieve-rrs.py

The columns are explained in section 6 Data structure of Rrs observations below.

The tool itself performs the following:

one “hits” query to retrieve the number of entries in the interval

repeated incremental queries with startIndex and count, using count=100 to retrieve 100

entries at a time

parsing of the WFS XML response, conversion of the packed binary Rrs values into formatted

float values

creation of the CSV file, generation of the headline, incremental formatted printing of

measurement lines

http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:RflexAPI&ShipCode=23&fromDate=2013-01-10&toDate=2013-12-10&upperCorner=59%2018&lowerCorner=50%2026&count=100&startIndex=0






http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs&request=GetFeature&version=2.0.0&storedquery_id=urn:x-inspire:storedQuery:RflexAPI&ShipCode=23&fromDate=2013-01-10&toDate=2013-12-10&upperCorner=59%2018&lowerCorner=50%2026



http://ferryscope.org/web_content/retrieve-rrs.py



- 8 - v1.0, 31.03.2015

6 Data structure of Rrs observations

This section defines the features returned by the Rflex data service and explains how to decode the

values.

6.1 FeatureType rflex:RrsMeasurement

The following table shows an example of a feature returned in the xml of a simple request. A

description of each field is given in the second column. Indentation is removed from this example.

XML field Description

<wfs:member> Start of feature

<rflex:RrsMeasurement gml:id="RFLEX_RRSMEASUREMENT_2320130712120022"

xmlns:rflex="www.syke.fi/Rflex">

Start of RrsMeasurement,

identifier and namespace

<rflex:utc>2013-07-12T12:00:22</rflex:utc> UTC time of measurement

<rflex:ship>23</rflex:ship> Ship code. The coding follows that of the Alg@line

network. At present, Rflex has been installed on:

01 – R/V Aranda (not available through the service)

23 – MS Finnmaid (Helsinki-Travemünde)

33 – MS Transpaper (Kemi-Oulu-Lübeck-Gothenburg)

<rflex:geom> Start of Geometry field

</!--Inlined geometry 'RFLEX_RRSMEASUREMENT_2320130712120022_RFLEX_GEOM'-->

<gml:Point gml:id="RFLEX_RRSMEASUREMENT_2320130712120022_RFLEX_GEOM"

srsName="urn:ogc:def:crs:EPSG::4326">

Start of Point Geometry field,

specifying Coordinate system (4326)

<gml:pos>56.036888 16.706977</gml:pos> Latitude and Longitude

</gml:Point> End of Point Geometry field

</rflex:geom> End of Geometry field

<rflex:sunelev>54</rflex:sunelev> Solar elevation in degrees

<rflex:edpar>492.94257</rflex:edpar> Downwelling Photosynthetically Active Irradiance

(PAR), measured between 400-700 nm. Units μmol

photons m-2 s-1

<rflex:skyrat400>0.066022195</rflex:skyrat400> Ratio of sky radiance over downwelling irradiance at

400 nm: πLsky(400)/Ed(400). A value close to 1 implies

cloud cover, a value < 0.2 indicates fair skies. See Simis

and Olsson 2012, RSE.

<rflex:rrs_1stwl>320.0</rflex:rrs_1stwl> Centre wavelength of the first waveband

<rflex:rrspitch>3.3</rflex:rrspitch> Waveband pitch

<rflex:rrs_lstwl>953.6</rflex:rrs_lstwl> Centre wavelength of the last waveband

<rflex:rrslen>193.0</rflex:rrslen> Number of values in rrs field

<rflex:rrs>000000000000F8FF346C18B6F28E593FA9E798511F8F563F50722DD1CBB4553F334BBCAE34E8553FB343DA6396115

63F0C3D22083020563F63A5CE805CAF563F35C656018CD9563FD86BF49229F8573F654A3C1F299F583FE29A44FF59F1583FF899CF

3478115A3F4B25D0D43ACC5A3F193092BE994B5B3F4DEF6FD137B15B3FB402F9173E395C3FB79A379444CF5C3FED95BD54D8125

D3FE78C915542A55D3F9C6F61BEECD25E3F54384B5DCA7F5F3FD37E8B842453603F430B5813060A613FAFA0F2BBD3C7613FBEB4

6B09C55F623F2DCCA0288AA1623FC51E9D45BEEE623F6AF242A57053633F2DA4714E09B4633FBE37377629F6633F9E689A482925

643F879AC46BBB4E643F14ECE38D9E76643F7B3F54CEBD60643F45C257D9AF82643F235580DC19C8643F674A0431A1E7643FEC6B

49167B33653FD51A146CE4BD653F77513037A659663F97E1092AB6F6663F1BAAC5330970673F0FD6C80C47F8673FDA724218E86C

683F0F9DE3836EDC683FD30A3C11F94A693F8EC636AB79C0693FD392203BB82C6A3F863741CC67966A3F4EEB331720ED6A3F9B8

The Rrs spectrum.

Decoding is described in the next section.


v1.0, 31.03.2015 - 9 -

2C8F67F416B3F72EDAC98A3AE6B3F8F2E0F42D64B6C3FDC333B1DA6D46C3FC98F312AC36A6D3FD22AF1ABD4156E3FEA725CF11F

B86E3F085962545E3F6F3FEAF07BAB1CB36F3F9421DEEF4F14703F4533B5E6DD67703F5B6BF3CD2EC2703F9935AE1E151A713F67

26199DD676713FBA0EEDE229D5713F719BBABC242B723FBD056FE33A7E723FF0D510740BBE723F30A65C6B46E2723F13E9BBE84

DEE723F926073095DEA723F58B2CABF54E6723F980FA93ACBE1723F3B048637C2CC723FC51FDA0A52A6723F7D30DB494782723F

C602851BAA5A723FD19BCAF7E430723F680043BD0ADD713FBFA584938D45713FFDF80CBB337C703F17F06E92C8066F3F85182A3

0FABD6C3F06A09128AC4F6A3F4EEA58798203683FF5AF422820FB653F90B7B18C086D643F9A34A97FBE67633FD81A671E85BF623

F2F0F01B2B743623F96609967ECD5613F8F027A630C7E613F797D27C32C28613FC42164AB8BDB603FFE8645EADA9D603FE091A96

88665603F17A9198E8040603F222F11CC7E27603F69A1C6C44303603FB777CC0FB1B65F3F03C372C1F2375F3F06BA6FD4CF7E5E3F

9737B1C018735D3FED0F931953505C3F9888041C156F5B3F2B78C3E7FDA45A3F15A14BBBB1105A3F21D35C4F13BA593F4378F204

B98D593F9426C2F24D94593F4465BD78BA9C593F0181C6DF8D66593FDFD6C11C8026593F846BA0FC33EF583FE9E584E29677583F

B4D76B3750A3573F05C4CF7E52A5563FA6CF2C4831B5553F75365B031D9E543F7429A2091565533F4BB1938687F4513F8CB0760B

2873503FE8A74CBF7ED44E3FE555937CC1AB4C3F9C6C654989E94A3FDDAAA5120794493FB7B48DA6D088483FE415BAE18D13483

FA0137408DEA6473FBDFEBED01870473F25DBBE95FC31473FDC9A5AC7200C473F2F5379881E0B473F586EB093D313463FAC8570

607407453F46C5EC2311AC453FEF1704AFD638463FA8DA28EBEE5C463FED72C6A53140463F25234AE46C11463F7C6BE521DA0746

3FB68D380F4DED453F3137B8F531EB453F1F416A3A59FE453F59800C20DDD3453F99C1FB4F541F463F1437FB24E810463FFE38DD

935B45463FE84AC591D129463F94889442D29D453FE0C00D785940453FBCEF9D56FCBA443F7278F1B7E24E443F6E5C8149671D44

3FE00B7930E95C433F79CA3A075BD6423F1AC8C79F1D4D423F4A7C8F0F6339423FCA7BECDC193B423F652F9A14CCBF413F26BC6

14263D6413FC6EE15248275413F867091356B9B413FF10ECD997697413F9C20EB5E7211413F5B27CF4DA561413F62A487E1908E41

3FAF227DE57038413F1976DCC0E1D4413F2609A358DD18413FA340F8F0A018413F4A4173D56D33403FA06E902EBD85403F07FFB

2200391403F71D082E99BD83E3F544FBAB75E1F3F3F30FF99762A103E3F2879144A03903C3F631899FF6AB83D3F442C1D43E93C3

D3F39CEE32F3C903B3F0C78A2B3B5153F3F5CB6B40E388F3B3FA1F2CF7F90763E3F9643CF1D80513D3F51BF5F5142373A3F4B40E4

5ABD463B3F259ADCE0BDA5353F56D1F1A46D253B3F9BC1D903E87A3D3FF377E37178FE383F000000000000F8FF</rflex:rrs>

</rflex:RrsMeasurement> end of this Rrs measurement

</wfs:member> end of this feature

6.2 rflex:RrsFerryboxMeasurement

The following table shows an example of a feature returned in the xml of a simple request. A

description of each field is given in the second column. Indentation is removed from this example.

This feature contains both Rflex and flow-through measuerements that are observed simultaneously.

XML field Description

<wfs:member> Start of feature

<rflex:RrsFErryboxMeasurement gml:id="

RFLEX_RRSFERRYBOXMEASUREMENT_2320140615093600" xmlns:rflex="www.syke.fi/Rflex">

Start of RrsFerryboxMeasurement,

identifier and namespace

<rflex:utc> 2014-06-15T09:36:00</rflex:utc> UTC time of measurement

<rflex:ship>23</rflex:ship> Ship code. The coding follows that of the Alg@line

network. At present, Rflex has been installed on:

01 – R/V Aranda (not available through the service)

23 – MS Finnmaid (Helsinki-Travemünde)

33 – MS Transpaper (Kemi-Oulu-Lübeck-Gothenburg)

<rflex:geom> Start of Geometry field

</!--Inlined geometry 'RFLEX_RRSFERRYBOXMEASUREMENT_2320140615093600_RFLEX_GEOM'--

>

<gml:Point gml:id="RFLEX_RRSFERRYBOXMEASUREMENT_2320140615093600_RFLEX_GEOM"

srsName="urn:ogc:def:crs:EPSG::4326">

Start of Point Geometry field,

specifying Coordinate system (4326)

<gml:pos>59.586807 23.204144</gml:pos> Latitude and Longitude

</gml:Point> End of Point Geometry field

</rflex:geom> End of Geometry field

<rflex:rflex_sunelev>52.0</rflex:rflex_sunelev> Solar elevation in degrees

<rflex:rflex_edpar>469.49884</rflex:rflex_edpar> Downwelling Photosynthetically Active Irradiance

(PAR), measured between 400-700 nm. Units μmol

photons m-2 s-1

<rflex:rflex_skyrat400>0.05899737</rflex:rflex_skyrat400> Ratio of sky radiance over downwelling irradiance at

400 nm: πLsky(400)/Ed(400). A value close to 1 implies


- 10 - v1.0, 31.03.2015

cloud cover, a value < 0.2 indicates fair skies. See Simis

and Olsson 2012, RSE.

<rflex:rflex_rrs_1stwl>320.0</rflex:rflex_rrs_1stwl> Centre wavelength of the first waveband

<rflex:rflex_rrspitch>3.3</rflex:rflex_rrspitch> Waveband pitch

<rflex:rflex_rrs_lstwl>953.6</rflex:rflex_rrs_lstwl> Centre wavelength of the last waveband

<rflex:rflex_rrslen>193.0</rflex:rflex_rrslen> Number of values in rrs field

<rflex:rflex_rrs>000000000000F8FF64C352D51C7452BF54F49E35B3A850BFD8CD3D0E743050BFC

6727A8FCBEF4FBF147F1C941DD74DBFF5741DD59F144CBF1A889629DF164BBF6869018EF4C148

BFB69DA7B463EC47BF75367DFC724C46BFC3ACA31DE30944BFE27D0AD32BA542BF3C1FD58370

3941BFB89B24A08F623FBF4D6E5D14ACB33BBF1B06216E269E37BFDAF6069ECF8234BF47757834

2D8A30BF24C86C2D0E8929BF10AC12DD3B1B23BF39D9B83EA80116BF6F1ED376574CF7BEE16D6

6C2B8D6053FB397AD7B642C1A3F5B7796E4BE71233F61F98C07E0E62A3FFF4229AF3814313F1C7

900CB4559343F7421B4AB4BC0373F27985E3DDD873A3F1C832F22A2433D3F4CE1CA3924D43F3F

D16D1869A927413FE360FCED69A7423F3AB721BA856C433F3AD7DA62B1F9433FF9DFA04F22F24

43FBD79138B08FD453F53238955A301473F6E04EA518E16483F4CC2ED29294E493F6730A96A189

94A3FBF0A2F8ED3C44B3FBB18342C5FE54C3F346E0B8297204E3F63E31A59BD774F3FBA219AD41

050503F8A08201E22D9503FDF354630715E513FD1FD7530DCF8513F1C91E73149BD523F94A3189

D976D533F3F9B246F77E6533FF78CE4A40F61543FD627E386A1FD543FD9189E5D2F9D553F2D5FE

742041A563F65D8AFF76A7C563F2FFECDB2B8F1563F028D01766D9A573FACBEF97AE33C583FDD

743D14A0B3583F82A942380E2E593F322F28FEA6B5593F4B281ACAE73C5A3FF9779E2705C05A3F

8B0A3BA84C4B5B3F475E197581BC5B3F626BF0C4CE065C3FE897A14523385C3F1753655BBF5E5C

3F3532E327D8905C3F2577E4BFB0C95C3FED172B279AED5C3F4BA498D916EF5C3FCCD8A034ACE

85C3FA9422E5B52BD5C3F7FE7B89DAA6D5C3F4E45BFB2BDCA5B3F4441B326B9C55A3FD25A796

C4D91593F3168CDE17C40583F16C9B9BF94A2563FA8EBC5CE1BB1543FDEB1340EBECD523F848E

8A3EB714513FC456D2FD99524F3F0A4F9EE3BE634D3F5F773050A52B4C3F3E785A0E883C4B3FD5

632A46116C4A3F9388B41624A5493F8EF662D85802493FA3FDD5F05268483F26CB16E7C8C0473F

F89768290B58473F04F01E7BECD9463F43864E501873463F1F8A7B6047FF453F18925EC24989453

F258A5FC9390E453F73DD7E681569443F30CD8F7DFAC6433F1763BFB88EFE423FF7419D8DC3474

23F08D3DAB472DC413F163F1D5AF2B7413F614E074EB8C5413F037CAED370EE413FDF8899693E

D6413FD4D139655FC7413F591BDD8B4E81413F687DC6D9E6DF403F370CD8C003BA3F3F27F1EA8

1A4583D3FF608C2FBB8543B3F1C6B375E0836393FD758156ED657373FE933A6AB1971353F188DC

5583B75333F14E34C758477313FC3657F56F2042F3F28F55998F8092B3FA1B3E6A6C333283F98F4

E11A6F3B253FD1B92E72D551233F1B3D4D0206E9213FF35808FDE458213F9D3B103F4EB3213FC2

2F0CC7C104213F6C5AF8183118213F3580F4B3146C203F74FD8CEF4FAD203F55A50E77912F213F4

80558B4D91D223F515950CCEA09213F0F9CF9983E0C203F0FDD6334BC311F3F05A8AD9848DD20

3F9442548D8814213F7A043C681CC6213F6B662E803FDC213FB0D0EA87499A213FA0D9FF555D6

8223FC7AD9F458876223F21DD43B89288223F9CD5CB4F5CBE223FA3E33BA021AD223F23A99BA5

94B2223FD3F557E04999223F9ABDAB213A8B213F3D4F74888934223F3C9A47CB0B50213F089731

90D12B213F342F909FCDD1203F359E4430283F203FCDAE6E9430A3213FBF53FFF314EE1E3F7DF8

C07330FE1F3F544909949923203FECEE56682F351F3FA0660F3FD3761E3FB50B37ABA0421E3FE44

94004701F193FD3C5D432BEEC1C3F068847D60DB71C3F9C1AEC0DE4D01D3F3612F0BF27741B3F

F7A62B3FD4891C3FD3AA135F315F1F3FBD599FA29D831B3F163E2CCD5E101B3FC3CA08CB7BBB1

93F9B20A4E033191A3F738798ACDFED193F01CDB540650C193FCB9472033255173FDAFFA84D23

A8183FB1E2E47A42EB0C3F2B03BFDD6C4F1A3F0524765FE1DB163FB53AEE53876D183FD12D947

F9F40133FC22B2C5D38B4153F321E622C11B8153F9E4FAF386A6E0E3F318F40BF7B9F173F0FF967

617FA8173F36480FBA5168193F1CD9BC74A334143F356A41E4BBA0133F000000000000F8FF</rfle

x:rflex_rrs>

The Rrs spectrum.

Decoding is described in the next section.

<rflex:al_timemin>2014-06-15T09:35:36</rflex:al_timemin> Start time of the Alg@line measurements

<rflex:al_timemax>2014-06-15T09:36:20</rflex:al_timemax> End time of the alg@line measurements

<rflex:al_num>3.0</rflex:al_num> Number of averaged Alg@line measurements

<rflex:al_sochlfl_avg>2.48</rflex:al_sochlfl_avg> Mean chlorophyll-a fluorescence (mg m-3)

<rflex:al_sochlfl_sd>0.05196152422705705</rflex:al_sochlfl_sd> Standard deviation of previous

<rflex:al_soflow_avg>3.15</rflex:al_soflow_avg> Mean flow rate (L min-1)

<rflex:al_soflow_sd>0.050000000000030555</rflex:al_soflow_sd> Standard deviation of previous


v1.0, 31.03.2015 - 11 -

<rflex:al_sopcfl_avg>0.11766666666666666</rflex:al_sopcfl_avg> Mean Phycocyanin fluorescence (arbitrary units)

<rflex:al_sopcfl_sd>0.0015275252316519533</rflex:al_sopcfl_sd> Standard deviation of previous

<rflex:al_sosal_avg>5.5760000000000005</rflex:al_sosal_avg> Mean of Salinity (PSU)

<rflex:al_sosal_sd>0.0017320508053823734</rflex:al_sosal_sd> Standard deviation of previous

<rflex:al_sotemp_avg>11.604999999999999</rflex:al_sotemp_avg> Mean of Temperature in flow system (°C)

<rflex:al_sotemp_sd>0.015099668870302736</rflex:al_sotemp_sd> Standard deviation of previous

<rflex:al_soturb_avg>1.05</rflex:al_soturb_avg> Mean of Turbidity (STU)

<rflex:al_soturb_sd>0.027712812921095863</rflex:al_soturb_sd> Standard deviation of previous

<rflex:al_soxtemp_avg>11.231</rflex:al_soxtemp_avg> Mean of Hull Temperature system (°C)

<rflex:al_soxtemp_sd>0.13322537295889827</rflex:al_soxtemp_sd> Standard deviation of previous

</rflex:RrsFerryboxMeasurement> End of this Rrs FerryboxMeasurement

</wfs:member> End of this feature

6.3 Decoding the Rrs data

The Rrs spectrum is represented in the XML response as a

hexadecimal string of double-precision floating point numbers (big-

endian byte order). The number of measurements is given in the

rrslen-property of the feature, normally 193.

6.3.1 Python (assuming big-endian environment) r = '000000000000F8FF0AC3F…'

rhex = r.decode("hex")

import struct

rrs = struct.unpack('d'*193,rhex)

A dedicated python client for Rflex which retrieves data from the WFS and takes care of dividing

requests according to the size of the result set, is included in Appendix A.

6.3.2 Matlab on windows (little-endian environment)

For little-endian environments the response string needs to be manually reordered to swap byte-

order without truncation problems. Matlab’s built-in swapbytes(hex2num(X))may truncate some values

and result in invalid spectra, so we need to flip the bytes manually, which is done in the for loop

below:

s = '000000000000F8FF0AC3F…’; %truncated for readability

n = 193;

X = reshape(s,16,n)';

Rrs = zeros(n,1);

for ii = 1:n,

x = X(ii,:);

y = fliplr(reshape(x,2,length(x)/2)')';

ss = fliplr(y(:)');

Rrs(ii)=hex2num(ss);

end

- 12 - v1.0, 31.03.2015

7 Known issues

There are a few known issues of the Rflex WFS. They refer to the system state at 2014-08-06 based

on degree 3.4-pre14.

WFS 2.0.0 sortBy parameter is not respected

Workaround: Use stored query RflexAPI which provides sorted data or send xml-formatted request with the following rule. Full examples are available in the query portal.

<fes:SortBy>

<fes:SortProperty>

<fes:ValueReference>rflex:utc</fes:ValueReference>



<fes:SortOrder>ASC</fes:SortOrder>

</fes:SortProperty>

</fes:SortBy>

Request portal resets the selected query to the default upon execution

Admin interface can not be used to edit the layer definition


v1.0, 31.03.2015 - 13 -

Appendix A Rflex Client for Python (retrieve-rrs.py)

To avoid formatting issues, get this script from http://ferryscope.org/web_content/retrieve-rrs.py

#!/usr/bin/python

__author__ = 'Copyright Brockmann Consult GmbH, Germany, 2014'

import urllib2

from xml.etree import ElementTree

import struct

import argparse

def retrieve_count(ship, start, stop):

url = 'http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs\

&request=GetFeature&version=2.0.0\

&storedquery_id=urn:x-inspire:storedQuery:RflexAPI\

&ShipCode={0}&fromDate={1}&toDate={2}\

&upperCorner=90%20-180\

&lowerCorner=-90%20180\

&resultType=hits'.format(ship, start, stop)

#print url

response = urllib2.urlopen(url)

content = response.read()

tree = ElementTree.fromstring(content)

count = int(tree.attrib['numberMatched'])

#print str(count)

return count

def retrieve_observations(ship, start, stop, count, file):

file.write("#dateFormat=yyyy-MM-dd'T'HH:mm:ss\n")

file.write('ID\tTIME\tLATITUDE\tLONGITUDE\tSHIP\tSun Elevation\

\tedpar\tskyrat400\tRrs Pitch\tRrs 1st wl\tRrs lst wl\

\t{0}\n'.format('\t'.join(map(lambda x: 'Rrs {0:03d} ({1:5.1f})'\

.format(x+1, 320 + x * (953.6-320) / 192), range(193)))))

cursor = 0

step = 100

while cursor < count:

url = 'http://ferryscope.ymparisto.fi/Rflex/services/RflexWFS?service=wfs\

&request=GetFeature&version=2.0.0\

&storedquery_id=urn:x-inspire:storedQuery:RflexAPI\

&ShipCode={0}&fromDate={1}toDate={2}\

&upperCorner=90%20-180\

&lowerCorner=-90%20180\

&startIndex={3}\

&count={4}'.format(ship, start, stop, cursor, step)

response = urllib2.urlopen(url)

content = response.read()

tree = ElementTree.fromstring(content)

for rrsMeasurement in tree.findall('{http://www.opengis.net/wfs/2.0}\

member/{www.syke.fi/Rflex}RrsMeasurement'):

id = rrsMeasurement.attrib['{http://www.opengis.net/gml/3.2}id']

time = rrsMeasurement.find('{www.syke.fi/Rflex}utc').text

sensor = rrsMeasurement.find('{www.syke.fi/Rflex}ship').text

(lat,lon) = rrsMeasurement.find('{www.syke.fi/Rflex}\

geom/{http://www.opengis.net/gml/3.2}\

Point/{http://www.opengis.net/gml/3.2}pos').text.split()

sunelev = rrsMeasurement.find('{www.syke.fi/Rflex}sunelev').text

edpar = rrsMeasurement.find('{www.syke.fi/Rflex}edpar').text

skyrat400 = rrsMeasurement.find('{www.syke.fi/Rflex}skyrat400').text

rrspitch = rrsMeasurement.find('{www.syke.fi/Rflex}rrspitch').text

rrs_1stwl = rrsMeasurement.find('{www.syke.fi/Rflex}rrs_1stwl').text

rrs_lstwl = rrsMeasurement.find('{www.syke.fi/Rflex}rrs_lstwl').text

rrs=rrsMeasurement.find('{www.syke.fi/Rflex}rrs').text

hex=rrs.decode('hex')

rrsValues=struct.unpack('<'+'d'*193, hex)

file.write('{0}\t{1}\t{2}\t{3}\t{4}\t{5}\t{6}\t{7}\t{8}\t{9}\t{10}\t{11}\n'\

.format(id,time,lat,lon,sensor,sunelev,edpar,skyrat400,\

rrspitch,rrs_1stwl,rrs_lstwl,'\t'\

.join(map(lambda x: str(x), rrsValues))))

cursor += step



- 14 - v1.0, 31.03.2015

if __name__ == '__main__':

parser = argparse.ArgumentParser()

parser.add_argument('ship', nargs=1, help="22 or 23")

parser.add_argument('start', nargs=1, help="something like 2013-06-01")

parser.add_argument('stop', nargs=1, help="something like 2013-06-30")

parser.add_argument('dest', nargs=1, help="something like data/rrs-23-2013-06.txt")

args = parser.parse_args()

ship = args.ship[0]

start = args.start[0]

stop = args.stop[0]

dest = args.dest[0]

count = retrieve_count(ship, start, stop)

print ship, start, stop, count, dest

with open(dest, "w") as f:

retrieve_observations(ship, start, stop, count, f)

rflex wfs api reference 13.04 -...

Documents