d3.3.2 communication solution design for field trial...

Communication solution design for field trial setup

SUNSEED, Grant agreement No. 619437 of 56

D3.3.2 Communication solution design for

field trial setup

Deliverable report



NOTICE The research leading to the results presented in the document has received funding from the European Community's 7th Framework Programme under the Grant Agreement number 619437. The content of this document reflects only the authors’ views. The European Commission is not liable for any use that may be made of the information contained herein. The contents of this document are the copyright of the SUNSEED consortium.



Document Information

1 PU Public

RP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential, only for members of the consortium (including the Commission Services)

Call identifier FP7-ICT-2013-11

Project acronym SUNSEED

Project full title Sustainable and robust networking for smart electricity distribution

Grant agreement number 619437

Deliverable number D3.3.2

WP / Task WP3 / T3.3

Type (distribution level)1 PU

Due date of deliverable 31.1.2016 (Month 24)

Date of delivery 30.1.2016 V1.0, updated 31.3.2016

Status, Version V2.0

Number of pages 56 pages

Responsible person, Affiliation Jurij Jurse, EP

Authors Ljupco Jorguseski, Manolis Chrysallos, Haibin Zhang, Onno Mantel, Casper

van den Broek TNO

Jimmy J. Nielsen (AAU)

Herve Ganem GTOSA

Tomaz Lovrencic, Blaž Petrnel, Robert Tošnjak, Bojan Dovč TS

Žiga Hribar, Bojan Miličić ES

Jurij Jurše EP

Reviewers Mihael Mohorčič, JSI

Zhong Fan (TREL)



Revision history Version Date Author(s) Notes Status

0.1 20.11.2015 L. Jorguseski ToC Draft

0.2 14.12.2015 J. J. Nielsen, L. Jorguseski, T. Lovrencic

Chapter 4, Chapter 5 Draft

1.0 30.2.2016 J. J. Nielsen, L. Jorguseski, T. Lovrencic, B., Miličič, J. Jurse, H. Ganem

Chapter 1, Chapter 2, Chapter 3 Submission Version

1.1 15.3.2016 B. Petrnel, R. Tošnjak, B. Dovč

Update of Chapter 2 and 3 Completion

1.2 29.3.2016 J. J. Nielsen, L. Jorguseski, J. Jurse

Merging and final editing Completion

1.3 31.3.2016 J.Jurše Final



Table of Contents Document Information ........................................................................................................................... 3

List of Figures ........................................................................................................................................... 7

List of Tables ............................................................................................................................................ 8

Abbreviations and acronyms ................................................................................................................... 9

SUNSEED project ................................................................................................................................... 10

Executive Summary ............................................................................................................................... 11

1 Introduction ................................................................................................................................... 13

2 Communication solutions for the SUNSEED field trial .................................................................. 15

2.1 Considered communication solutions for WAMS ................................................................. 15

2.1.1 Scenario A: WAMS SPM/PMC – mobile or fixed modem/router .................................. 15

2.1.2 Scenario B: WAMS SPM – satellite modem/router ....................................................... 20

2.2 Considered communication solutions for smart meters....................................................... 21

2.2.1 Scenario C: mobile or fixed modem/router – PLC data concentrator – smart meter ... 21

2.2.2 Scenario D: mobile or fixed modem/router – Ethernet or RS485 to Ethernet converter - smart meter ................................................................................................................................. 22

2.3 Communication solution in individual functional location (transformer station) ................ 23

3 Security solution for the SUNSEED field trial ................................................................................ 25

3.1 Communication security and architecture for field trial ....................................................... 25

3.2 Security architecture for credential management ................................................................ 29

3.3 Credential provisioning to the WAMS node.......................................................................... 31

4 Communication measurement procedures for the SUNSEED field trial ....................................... 33

4.1 Smart grid traffic characterization measurements ............................................................... 33

4.1.1 PCAP tracing locations ................................................................................................... 33

4.1.2 Available information from PCAP traces ....................................................................... 34

4.1.3 KPIs and statistics of interest from PCAP traces ........................................................... 35

4.1.4 Traffic Characterization Measurement procedure........................................................ 36

4.2 WAMS-SPM end-to-end delay measurement ....................................................................... 37

4.2.1 Measurement approach ................................................................................................ 38

4.3 End-to-end throughput measurement .................................................................................. 39

4.3.1 Throughput measurement setup in controlled laboratory environment............................ 40

4.4 Radio access related measurements ..................................................................................... 44

4.4.1 Relevant measurements derived from statistical counters/KPIs at the eNB ................ 45

4.4.2 Gemalto LTE modem ..................................................................................................... 48

4.4.3 External communication equipment ............................................................................. 49

5 Field trial communication measurements and relation to SUNSEED use cases ........................... 51

5.1 Measurements related to state estimation use case ............................................................ 51



5.2 Measurements related to demand-response use case ......................................................... 52

5.3 Measurements related to SG outage localization use case .................................................. 53

6 Conclusions .................................................................................................................................... 54

7 References ..................................................................................................................................... 56



List of Figures

Figure 1 High-level overview of the SUNSEED trial network (incl. different locations, communication systems) ................................................................................................................................................. 14 Figure 2: Communication scenario A.1: WAMS – LTE mobile modem/router. ..................................... 15 Figure 3: Communication scenario A.2: WAMS – UMTS mobile modem/router. ................................ 16 Figure 4: Communication scenario A.3: WAMS – xDSL or FTTH modem/router. ................................. 16 Figure 5: WAMS-SPM typical wiring diagram scheme in communication scenarios A.1 and A.2......... 17 Figure 6: WAMS-PMC typical wiring diagram scheme in communication scenarios A.1 and A.2. ....... 18 Figure 7: An example of inventory material with technical specification for scenario A ..................... 19 Figure 8: Communication scenario B.1: WAMS – satellite modem/router in combination with Wi-Fi (options B2). .......................................................................................................................................... 20 Figure 9: Communication scenario B: WAMS – satellite modem/router (options B1) in combination with Wi-Fi (options B2). ......................................................................................................................... 20 Figure 10: Communication scenario C.1: mobile modem/router – PLC data concentrator (DC) – smart meter. .................................................................................................................................................... 21 Figure 11: Communication scenario C.2: fixed modem/router – PLC data concentrator (DC) – smart meter. .................................................................................................................................................... 22 Figure 12: Communication scenario D.1: mobile modem/router – Ethernet or RS485 to Ethernet converter - smart meter. ....................................................................................................................... 22 Figure 13: Communication scenario D.2: fixed modem/router – Ethernet or RS485 to Ethernet converter - smart meter. ....................................................................................................................... 23 Figure 14: Selected communication scenarios for WAMS SPM devices and smart meters on individual transformer station. .............................................................................................................................. 24 Figure 15: Field trial security architecture ............................................................................................ 25 Figure 16: SUNSEED general back-end infrastructure. .......................................................................... 26 Figure 17: SUNSEED APNs related back-end infrastructure. ................................................................. 27 Figure 18: SUNSEED xDSL related infrastructure. ................................................................................. 28 Figure 19: SUNSEED VPN related back-end infrastructure. .................................................................. 29 Figure 20 : Communication and security architecture .......................................................................... 30 Figure 21: Practical implementation of security architecture............................................................... 31 Figure 22: security and credential provisioning workflow .................................................................... 32 Figure 23: PCAP location for packet sniffing within Telekom Slovenija network used in the trial ....... 34 Figure 24: Example of PCAP trace, showing implementation of a SIP protocol from [2] ..................... 35 Figure 25 Schematic view on the end-to-end delay measurement procedure for WAMS-SPM nodes 38 Figure 26: Measurement collection from WAMS-SPM. Upon completion of a measurement burst, a packet is assembled and sent to the TS core, where it is detected and a timestamp is made at the APN, and finally at the DB server, another timestamp is made when the data is stored in the database. ............................................................................................................................................... 39 Figure 27: End-to-end scheme .............................................................................................................. 41 Figure 28: Lab environment - test bed for end-to-end telecommunication compatibilities ................ 41 Figure 29: time of reading 10 registers from 1 smart meter is less than 2 seconds) ............................ 42 Figure 30: Laboratory test LV power grid configuration ....................................................................... 43 Figure 31: SNMP community ................................................................................................................. 50



List of Tables

Table 1 ................................................................................................................................................... 45



Abbreviations and acronyms

APN Access Point Name

EP Elektro Primorska

DB Database

DC Data Concentrator

DPSK Differential Phase Shift Keying

FTTH Fiber-to-the-Home

GGSN Gateway GPRS Support Node

HSPDA High Speed Downlink Packet Access

IP Internet Protocol

IT Information Technology

KPI Key Performance Indicator

LTE Long Term Evolution

LV Low Voltage

M2M Machine to Machine

MQTT MQ Telemetry Transport

MPLS Multiprotocol Label Switching

OFDM Orthogonal Frequency Division Multiplexing

PCAP Packet Capture

PCB Printed Circuit Board

P-GW Packet Gateway

PLC Power Line Carrier

SG Smart Grid

SGSN Serving GPRS Support Node

S-GW Serving Gateway

SM Smart Meter

S-FSK Spread Frequency-shift Keying

SNMP Simple Network Management Protocol

SPM Synchro Phasor Measurement

TAC Tracking Area Code

TCP Transmission Control Protocol

TS Telekom Slovenija

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications Service

WAMS Wide Area Measurement and Supervision

VPN Virtual Private Network

WAMS – PMC WAMS Power Measurement and Control

WAMS – SPM WAMS Synchro-Phasor Measurement

WAN Wide Area Networks

XMPP Extensible Messaging and Presence Protocol



SUNSEED project

SUNSEED proposes an evolutionary approach to utilization of already present communication networks from

both energy and telecom operators. These can be suitably connected to form a converged communication

infrastructure for future smart energy grids offering open services. Life cycle of such communication network

solutions consists of six steps: overlap, interconnect, interoperate, manage, plan and open. Joint

communication networking operations steps start with analysis of regional overlap of energy and

telecommunications operator infrastructures. Geographical overlap of energy and communications

infrastructures identifies vital DSO energy and support grid locations (e.g. distributed energy generators,

transformer substations, cabling, ducts) that are covered by both energy and telecom communication

networks. Coverage can be realised with known wireline (e.g. copper, fiber) or wireless and mobile (e.g. WiFi,

4G) technologies. Interconnection assures end-2-end secure communication on the physical layer between

energy and telecom, whereas interoperation provides network visibility and reach of smart grid nodes from

both operator (utility) sides. Monitoring, control and management gathers measurement data from wide area

of sensors and smart meters and assures stable distributed energy grid operation by using novel intelligent real

time analytical knowledge discovery methods. For full utilization of future network planning, we will integrate

various public databases (e.g. municipality GIS, weather). Applications build on open standards (W3C) with

exposed application programming interfaces (API) to 3rd parties enable creation of new businesses related to

energy and communication sectors (e.g. virtual power plant operators, energy services providers for optimizing

home energy use) or enable public wireless access points (e.g. WiFi nodes at distributed energy generator

locations). SUNSEED life cycle steps promise much lower investments and total cost of ownership for future

smart energy grids with dense distributed energy generation and prosumer involvement.

Project Partners

1. TELEKOM SLOVENIJE D.D.; TS; Slovenia

2. AALBORG UNIVERSITET; AAU; Denmark

3. ELEKTRO PRIMORSKA, PODJETJE ZA DISTRIBUCIJO ELEKTRICNE ENERGIJE D.D.; EP; Slovenia

4. ELEKTROSERVISI, ENERGETIKA, MERILNI LABORATORIJ IN NEPREMICNINE D.D.; ES; Slovenia

5. INSTITUT JOZEF STEFAN; JSI; Slovenia

6. GEMALTO SA; GTOSA; France

7. GEMALTO M2M GMBH; GTOM2M; Germany

8. NEDERLANDSE ORGANISATIE VOOR TOEGEPAST NATUURWETENSCHAPPELIJK ONDERZOEK - TNO; TNO;

The Netherlands

9. TOSHIBA RESEARCH EUROPE LIMITED; TREL; United Kingdom

Project webpage

http://www.sunseed-fp7.eu/

http://www.sunseed-fp7.eu/



Executive Summary

The deliverable outlines the specific scenarios for communication solutions on separate field trial locations which are geographically quite distinguished. Regarding the proposed scenarios the general communication SUNSEED back-end infrastructure with security architecture and provisioning the security credentials to the end nodes is defined. In particular, we focus on considered solutions for WAMS (SPM and PMC) and for smart meters, respectively. Of course, due to costs reduction and where possible, WAMS and smart meters can use the same gateway for a considered communication solution. For WAMS scenarios the typical wiring diagram scheme and an example of inventory material with technical specification was made. On the basis of coverage analyses from deliverable D5.1.2 the most appropriate communication scenario per each transformer station as the main functional location was assigned. In-depth analyses of the field trial locations and of the Telekom Slovenije infrastructure reviled that mobile and satellite communications are the most appropriate to connect majority of WAMS devices into the SUNSEED cloud. Therefore, we distinguish two communication scenarios, where WAMS (SPM/PMC) are using mobile or fixed modem/router (Scenario A) and WAMS SPM are using satellite modem/router (Scenario B). With respect to the analysis of fixed network access points presence and mobile network coverage from deliverables published in WP5 scenario A is further divided into three sub-scenarios, where gateway (modem/router) could connects the LTE network (scenario A.1), UMTS network (scenario A.2) or xDSL/FTTH network (scenario A.3). Regarding the position of a satellite antenna scenario B is also divided into two sub-scenarios where WAMS could be located at utility functional location and it is connects directly to satellite modem/router, while the location can have also Wi-Fi bridge to subordinate functional location (Scenario B.1). Other possibility is WAMS positions at subordinate functional location and connects to the satellite modem/router via Wi-Fi bridge (Scenario B.2). Similar as WAMS in scenario A, smart meters use mobile or fixed modems/routers as gateways for connecting into the SUNSEED cloud via mobile or fixed network. However, with regard to the exact smart meter model, they are connected to such gateways in different manner. We distinct two scenarios, where smart meters are connected to PLC data concentrator (DC), which is connected to mobile or fixed modem/router (Scenario C) or they are connected to mobile or fixed modem/router via RS485 to Ethernet converter (Scenario D). Sunseed IT selected security will insure the proper filtering of data transmission at the IP level . It consist of four main parts in general to support WAMS connectivity with the data centre via by using mobile connections with private APNs, WAMS connectivity with the data centre via xDSL or FTTH connections and WAMS connectivity with the data centre via VPN by using satellite links. The datacentre is behind the firewall with load balancing functionality for physical separation from the access network part. For end WAMS nodes access the authorization delegation model is used. . In such model, “Resource servers” (entities controlling access to some “resource” delegate the administration of the access control to some or all of their resources to an external authorization server. During the trial measurements it is important to characterize the ongoing traffic in terms of e.g. packet sizes, inter-packet arrival times, aggregated amount of traffic, etc. This realistic traffic



characterization based on the trial communication sessions is useful for modifying the existing traffic models used for performance analysis of communication networks for smart grids. In this deliverable a detailed procedure is explained regarding capturing packets from the smart grid applications during the trial via PCAP traces as well as definitions of the most important traffic KPIs (e.g. traffic volume, packet size, time series of packet generation, etc.). Next to the traffic characterization it is important to measure the communication network performance parameters. This deliverable describes end-to-end delay and throughput measurement procedures that will be used in the trial. For the end-to-end delay it is important to realize sufficiently accurate time synchronization between the two communication end-points. Finally, during the communication sessions it is important to measure the most important radio access communication network conditions (e.g. signal level, interference level, radio access load, etc.) in order to understand the end-to-end delay and throughput performance. This deliverable defines two approaches for measuring the radio access network conditions: the approach of using radio access related performance counters as provided by the vendor of radio access network, and the approach of using AT commands and/or SNMP commands to extract the necessary radio access performance information from the communication modems. During the SUNSEED trial the measurements of the electricity grid and the communication network will be used to assess the feasibility and the performance requirements for the three defined SUNSEED use cases, namely the real-time state estimation, demand response, and outage localization use case. For the electricity grid state estimation use case a procedure is defined regarding the testing of the accuracy and up to date information of the state-estimation function. As the reporting interval of the WAMS-PMC and WAMS-SPM nodes is varied from e.g. 15 min down to 1 s the impact on the communication network and end-to-end delays will be investigated as well as the ability of the state-estimation algorithm to predict a correct value and on time. For the demand response use case next to the measurements needed for the state estimation additional measurements and control information messages will be collected. As the delay in demand response cycles is typically around 15 min, for this use case it is important to check the reliability of these additional measurements and control messages (i.e. to check the stability and reliability of the control process) and not on the end-to-end delay performance for these messages. Finally, by instructing the WAMS-SPM and WAMS-PMC nodes to initiate ‘fake outage’ messages enriched with location information (e.g. GPS coordinates for WAMS-SPM) the feasibility of automatic grid outage localization will be assessed as the final SUNSEED use case.



1 Introduction

At the moment of writing of this deliverable most of the locations and solutions regarding the communication part of the SUNSEED field trial are determined. The goal of this deliverable is to present in detail the actually used communication network for the SUNSEED trial. Throughout the deliverable a reference is given to D5.1.2 where the preliminary/possible communication options are also presented. The end nodes in the SUNSEED trial (e.g. SM, WAMS-SPM, and WAMS-PMC) are communicating via the LTE and UMTS wireless telecommunication network of Telekom Slovenija, with exception at location Kneža, where also satellite links are utilized. The communication network set-up is illustrated in Figure 1 for the four SUNSEED field trial areas:

1) Area Kromberk, where: a. LTE base stations (or eNBs) are used for covering 10 WAMS-SPM nodes, 1 WAMS-

PMC nodes, 6 PLC concentrator nodes and 116 SM nodes. 2) Area Bonifika, where:

a. LTE base stations (or eNBs) are used for covering 7 WAMS-SPM nodes, 2 WAMS-PMC nodes, 5 PLC concentrator nodes and 535 SM nodes.

3) Area Razdrto, where: a. UMTS base stations (or eNBs) are used for covering 17 WAMS-SPM nodes, 2 WAMS-

PMC nodes, 7 PLC concentrator nodes and 10 SM nodes. 4) Area Kneža, where:

a. LTE base stations (or eNBs) are used for covering 3 WAMS-SPM nodes, 0 WAMS-PMC nodes, 2 PLC concentrator nodes and 0 SM nodes.

b. Satellite links covering 5 WAMS-SPM nodes, 0 WAMS-PMC nodes, 2 PLC concentrator nodes and 2 SM nodes. This is not illustrated in Figure 1 due to clarity reasons.

The data packets in the domain of the wireless communication network of Telekom Slovenija are flowing through the dedicated Access Point Name (APN) sunseed.ts.si. Outside the mobile communication network domain the measurements are forwarded to the dedicated SUNSEED field trial database, also illustrated in Figure 1.



Figure 1 High-level overview of the SUNSEED trial network (incl. different locations, communication systems)

The structure of the deliverable is as follows. Chapter 2 presents the different access network possibilities for the wide (or neighbourhood) area networks (WAN) for the WAMS-SPM, WAMS-CPM, PLC concentrators and SM nodes installed in the SUNSEED field trial. Chapter 3 explains the security solution for protecting the packet data streams and for provisioning the security credentials to the end nodes. In Chapter 4 we present the different communication measurement procedures that will be used during the SUNSEED trial execution. The relation between the measurements in the trial and the defined SUNSEED use cases is presented in Chapter 5. The deliverable is finalized with the conclusions in Chapter 6.



2 Communication solutions for the SUNSEED field trial

This chapter focuses on communication solutions for the SUNSEED field trial. In particular, chapter 2.1 focuses on considered solutions for WAMS (SPM and PMC) and chapter 2.2 for smart meters, respectively. Of course, due to costs reduction and where possible, WAMS and smart meters can use the same gateway for a considered communication solution.

2.1 Considered communication solutions for WAMS

The preliminary plan from D5.1.2 proposes and describes possible communication solutions for the WAMS with respect to gateway type (internal or external). However, in-depth analyses of the field trial locations and of the Telekom Slovenije infrastructure reviled that mobile and satellite communications are the most appropriate to connect majority of WAMS devices into the SUNSEED cloud. Therefore, we distinguish two communication scenarios, where:

WAMS (SPM/PMC) are using mobile or fixed modem/router (Scenario A),

WAMS SPM are using satellite modem/router (Scenario B).

2.1.1 Scenario A: WAMS SPM/PMC – mobile or fixed modem/router

In scenario A WAMS devices use mobile or fixed modems/routers as gateways for connecting into the SUNSEED cloud via mobile or fixed network, as depicted in Figure 2. With respect to the analysis of fixed network access points presence and mobile network coverage (see deliverables D5.1.1 and D5.1.2.) scenario A is further divided into three sub-scenarios, where:

- gateway (modem/router) connects to the LTE network (scenario A.1), - gateway (modem/router) connects to the UMTS network (scenario A.2), - gateway (modem/router) connects to the xDSL or FTTH network (scenario A.3).

With regard to each deployment location specifics, scenarios A.1 and A.2 are implemented so that communication antenna of the gateway is mounted either inside or outside the electrical box or the object where WAMS is mounted to overcome the Faraday cage/shield effect.

Communication networkWAN

Utility functional location(Transformer station)

LTE

ETHERNETSURGE ARRESTER

MODEM/ROUTERWAMS

(SPM/PMC)

Cellular network

Dataconcentrator

Figure 2: Communication scenario A.1: WAMS – LTE mobile modem/router.



Communication network

WAN


UMTS


MODEM/ROUTERWAMS

(SPM/PMC)

Cellular network

Data concentrator

Figure 3: Communication scenario A.2: WAMS – UMTS mobile modem/router.



xDSLFTTH

ETHERNET

MODEM/ROUTERWAMS

(SPM/PMC)

Network

Data concentrator

Figure 4: Communication scenario A.3: WAMS – xDSL or FTTH modem/router.

Next, Figure 5 and Figure 6 depict typical wiring of WAMS-SPM and WAMS-PMC in most often considered communication scenarios A.1 and A.2. The one should note that deployment locations’ wiring diagrams in practice can differ from the depicted examples due to locations’ specifics. Considering such diagrams, we prepared also detailed inventory of power and communication material and technical specification for each deployment location. An example of such documentation is depicted in Figure 7.



Figure 5: WAMS-SPM typical wiring diagram scheme in communication scenarios A.1 and A.2.



Figure 6: WAMS-PMC typical wiring diagram scheme in communication scenarios A.1 and A.2.



Figure 7: An example of inventory material with technical specification for scenario A

COMMUNICATION SCENARIO A WAMS – cellular modem/router

Description Unit Quantity

POWER EQUIPEMENT

Power protection elements

Miniature circuit breaker 230 V AC, piece 1

SPECIFICATIONS:

- Rated current I N=6A

- Tripping characteristic: B

- Other: DIN rail, 3-pole, 3 module

Surge arrester (for ovehead transformer station, 3 pieces per one WAMS) piece 3

SPECIFICATIONS:

- Max. continuous operating voltage U c = 275 - 320 V AC

- Protection lavel U p at I n (8/20) <= 3,2 kV

- Max. discharge current (8/20) I max >= 80 kA

- Impulse current I imp >= 25 kA

- Other: DIN rail, 1-pole, 1,5 module, Terminal cross section: 25mm2 (stranded), IEC: I,II

Surge arrester (for cable transformer station, 1 pieces per one WAMS) piece 3

SPECIFICATIONS:

- Max. continuous operating voltage U c = 275 - 320 V AC

- Protection lavel U p at I n (8/20) <= 1,5 kV

- Max. discharge current (8/20) I max >= 40 kA

- Impulse current I imp >= 12,5 kA

- Other: DIN rail, 3-pole, 3 module, Terminal cross section: 25mm2 (stranded), IEC: I,II

Fusse (low voltage NH knife-blade type, 1 pieces per one location) piece 3

SPECIFICATIONS:

- Rated current I N=100A

- inserted into 160 A fusse base

Power supply wires (quantities per one location)

PVC insulated wire H07V-K 1,5 mm2, black (fine stranded), for electonics devices supply m 3

PVC insulated wire H07V-K 1,5 mm2, blue (fine stranded), for electonics devices supply m 1

PVC insulated wire H07V-K 1,5 mm2, yellow/green (fine stranded), for electonics devices supply m 1

PVC insulated wire H07V-K 10 mm2, black (fine stranded), for phase to surge arrester conncetion m 3

PVC insulated wire H07V-K 16 mm2, yellow/green (fine stranded), for surge arrester to ground conncetion m 1

Other power material (din rail, connectors, busbars…)

COMMUNICATION EQUIPEMENT

Modem/router piece 1

SPECIFICATIONS:- 4G LTE Router, WAN, 3xLAN, WiFi 802.11.b/g/n

- dual SIM, VPN

Power supply 24 V, 40 W, 2 module piece 1

SPECIFICATIONS:

- AC240V/DC 24V

- In 1,66A , 40W

Protection for modem/router ( 2A) piece 1

SPECIFICATIONS:

-20kA lighting surge protection, one ground position

- Two passsive 10/100/1000 Ethernets ports

Protection for power supply Line-up terminal with fuse holder VSV4 piece 1

SPECIFICATIONS:

'- DC 2A

Antenna for LTE piece 1

SPECIFICATIONS:

- 2G/3G/4G Terminal Antenna for Cellular Gateways and Routers, LTE/GSM

Antenna for GPS piece 1

SPECIFICATIONS:

- gain 30Db, output impedance 50 ohm

Communication wires (quantities per one location)

PVC insulated wire H07V-K 1,5 mm2, - za ožičenje WAMSA m 3

Antene cable LTE 50 ohm, female, male SMA connectors m 1

Antene cable GPS 50 ohm, female, male SMA connectors m 1

STP cable cat 5e -data cable + 2x RJ45STP m 1



2.1.2 Scenario B: WAMS SPM – satellite modem/router

Scenario B enables connection of WAMS devices in regions with bad mobile network coverage, no mobile network coverage and no fixed broadband access points in the vicinity. In the SUNSEED field trial, an example of such region is the area of Kneža where we overcome natural challenging properties with the satellite communication links. With respect to the position of a satellite antenna we distinct two sub-scenarios, where:

- WAMS is located at utility functional location and connected directly to satellite modem/router, while the location can have also Wi-Fi bridge to subordinate functional location (Scenario B.1),

- WAMS is located at subordinate functional location connected to the satellite modem/router via Wi-Fi bridge (Scenario B.2).

Scenario B.1 is depicted in Figure 8 and Scenario B.2 is depicted in Figure 9. However, the wiring diagram and inventory of material for Scenario B is unfortunately unavailable at the time of writing of this deliverable.




MODEM/ROUTERWAMS(SPM)

Satellitecommunication

Figure 8: Communication scenario B.1: WAMS – satellite modem/router in combination with Wi-Fi (options B2).



ETHERNET

SURGE ARRESTER

MODEM/ROUTER

Satellitecommunication

WirelessComunication

(Wi-Fi)

Wi-Fi bridge(to subordinate functional location)

Communication networkNAN

ETHERNET

WAMS(SPM)

SURGE ARRESTER

Utility subordinate functional location(Transformer station)

Wi-Fi bridge

Figure 9: Communication scenario B: WAMS – satellite modem/router (options B1) in combination with Wi-Fi (options B2).



2.2 Considered communication solutions for smart meters

Though the solutions for connecting smart meters were discussed into details in D5.1.2, we had to identify most feasible options for their connection in the field trial. Similar as WAMS in scenario A, smart meters use mobile or fixed modems/routers as gateways for connecting into the SUNSEED cloud via mobile or fixed network. However, with regard to the exact smart meter model, they are connected to such gateways in different manner. We distinct two scenarios, where:

smart meters are connected to PLC data concentrator (DC), which is connected to mobile or fixed modem/router (Scenario C),

smart meters are connected to mobile or fixed modem/router via RS485 to Ethernet converter (Scenario D).

2.2.1 Scenario C: mobile or fixed modem/router – PLC data concentrator – smart

meter

One or three phase residential smart meters in the field trial use Power Line Carrier (PLC) communications to bind at least four customers (consumers, producers or prosumers) to data concentrator, which is typically connected to WAN gateway in a transformer station. As such, scenario C is further divided into two sub-scenarios, where such: - gateway (modem/router) connects to the LTE network (scenario C.1), - gateway (modem/router) connects to the UMTS network (scenario C.2). For visual presentation Figure 10 depicts Scenario C.1 and Figure 11 scenario C.2. The advantage of PLC is in overall coverage alongside the low voltage power grid, which eliminates the problems with extra communication infrastructure in the last mile. The most common PLC technologies use a single-carrier system with S-FSK (Spread Frequency-shift Keying) modulation, which lacks flexibility in selecting the carrier frequency, and hence, results in a low throughput, but reliability ([E. Hossain, Z. Han, H.V. Poor, 2013]). It is defined in standard IEC 61334 and often called PLAN defined. However, in Razdrto area common PLC technology will be upgraded to PLC G3, which is a narrow band technology offering a great balance between robustness, quality of service, data rate and cost. It uses Orthogonal Frequency Division Multiplexing (OFDM) and Differential Phase Shift Keying (DPSK).



LTEUMTS

ETHERNET

SURGE ARRESTER

MODEM/ROUTER

WAMS(SPM/PMC)

Cellular network

Dataconcentrator

0,4 kV

LV Power grid

PLC communication NAN

Figure 10: Communication scenario C.1: mobile modem/router – PLC data concentrator (DC) – smart meter.




WAN


xDSL, FTTH

ETHERNET

MODEM/ROUTER

WAMS(SPM/PMC)

Dataconcentrator

0,4 kV

LV Power grid

PLC communication NAN

Figure 11: Communication scenario C.2: fixed modem/router – PLC data concentrator (DC) – smart meter.

2.2.2 Scenario D: mobile or fixed modem/router – Ethernet or RS485 to Ethernet

converter - smart meter

This scenario enables connection of individual smart meters or a group of smart meters on the same functional location. In the field trial it is considered for industrial smart meters which do not support PLC connection and in case of LV power grid with very low number of smart meters (three or less) or very long distance among them. Scenario D is further divided into sub-scenarios, where: - gateway (modem/router) connects to the UMTS or LTE network (scenario D.1), - gateway (modem/router) connects to the xDSL or FTTH network (scenario D.2). The scenario D.1 is conceptually presented in Figure 12 and scenario D.2 in Figure 13. In both scenarios there are two possibilities of connecting smart meters to modem/router. First is a parallel connection of individual devices via Ethernet. Second is a serial connection through RS485 protocol.


Utility functional location(customer connection box)

LTEUMTS

ETHERNET

SURGE ARRESTER

MODEM/ROUTER

Cellular network

RS485-Ethernet converter

RS485RS485

Figure 12: Communication scenario D.1: mobile modem/router – Ethernet or RS485 to Ethernet converter - smart meter.




WANUtility functional location

(customer connection box)

xDSL, FTTH

ETHERNET

MODEM/ROUTER

RS485-Ethernet converter

RS485RS485

Figure 13: Communication scenario D.2: fixed modem/router – Ethernet or RS485 to Ethernet converter - smart meter.

2.3 Communication solution in individual functional location

(transformer station)

The aim of this chapter is assigning the most appropriate communication solution per each transformer station being considered as the main functional location. The assignment of communication solution is based on comprehensive communication network coverage analyses from D5.1.1 and their recent updates. The table in Figure 14 depicts planned communication solutions to connect WAMS devices and smart meters in all transformer stations in the field trial.



Figure 14: Selected communication scenarios for WAMS SPM devices and smart meters on individual transformer station.

Field trial

location

Utility functional location -

Transformer substation

(TS)

Communication

scenario for WAMS

SPM connection

Nm. of WAMS

SPM

Nm. Of WAMS

PMCNotes

Communication scenario

for residental smart

meters

Number of Data

concetrators

Number of PLC

residental smart

meters

Number of non

PLC residental

smart meters

Communication

scenario for

industrial smart

meters

Number of

industrial smart

meters

Meblo A1 1 1

Modem/router

conform with IEC

61850-3

C 1 11 D 2

Primarna A1 1 not presented D 2

Kotlarna A1 2

Modem/router

conform with IEC

61850-3

C 2 24 D 12

Površinska A1 1

Modem/router

conform with IEC

61850-3

C 1 23 D 6

A+A A1 1

Modem/router

conform with IEC

61850-3

not presented D 1

Pikolud A1 1

Modem/router

conform with IEC

61850-3

C 1 19 D 3

Jogi A1 1

Modem/router

conform with IEC

61850-3

C 1 8 D 3

MFE Ilmest A1 1 not presented D 1

SE Meblo Jogi A1 1 not presented D 1

Bonifika 2 A1 1

Modem/router

conform with IEC

61850-3

C 1 37 D 11

Agraria A1 1 not presented D 1

Bonifika 1 A1 1 2

Modem/router

conform with IEC

61850-3

C 1 81 D 8

Bonifika 3 A1 1

Modem/router

conform with IEC

61850-3

C 1 13 D 6

Špar A1 1 not presented D 2

Črpališče Kanal grande A1 1

Modem/router

conform with IEC

61850-3

C 1 4 D 3

Koper 3 (DVTP)without WAMS

installationC 1 360 D 8

Obrtni center not in operation D

Markovec vzhod A1 1 not presented D 1

RP Razdrto A2 1

Modem/router

conform with IEC

61850-3 C 1 9 not presented

Profiles peletirnica A2 1 not presented D 1

Razdrto vas A2 1 1

Modem/router

conform with IEC


KZ Razdrto A2 1 C 1 10 not presented

Kovinoplastika A2 1 D 1 D 1

Razdrto A2 1

Modem/router

conform with IEC

61850-3 C 1 6D

1

Razdrto Kamnolom A2 1 not presented D 1

Asfaltna baza Laže undefined 1 not presented D 1

Laže undefined 1

Modem/router

conform with IEC


Asfaltna baza Senožeče A2 1 D 1 D 1

Ravni A2 1 D 3 not presented

Senožeče 1 A2 1 C 1 20 not presented

Nanos A2 1 not presented D 1

Tri hiše A2 1 C 1 11 not presented

Hruševje mlin A2 1 1 D 3 not presented

C.P. Nanos A2 1 D 2 not presented 1

MVE Razdrto 1 not presented D 1

Laže Cestarska hiša not in operation

Zuza A1 1

Modem/router

conform with IEC

61850-3

not presented D1

Klavže A1 1 C 1 37 D 1

Borovnica undefined 1 C 1 5 not presented

Sela nad Podmelcem undefined 1 C 1 22 not presented

Logar undefined 1 D 1 not presented

Loje undefined 1 C 1 5 not presented

Mohor B2 1 D 2 not presented

Knežke Ravne B1 1 C 1 4 not presented

Knežke ravne vas B1,B2 1 C 1 5 D 1

HE Podmelec A1 1 not presented D 1

MHE Kneža undefined 1 not presented D 1

mHE Knežke Ravne 2 B1,B2 1 not presented D 1

mHE Strmec Hvala B1,B2 1 not presented D 1

KNEŽA (TOLMIN)

KROMBERK (NOVA GORICA)

BONIFIKA (KOPER)

RAZDRTO (SEŽANA)



3 Security solution for the SUNSEED field trial

3.1 Communication security and architecture for field trial

Figure 15 provide an overview of the IT level security in place for the trial. Those security measure will insure the proper filtering of data transmission at the IP level.

Figure 15: Field trial security architecture

The SUNSEED back-end infrastructure is depicted in Figure 16. It consist of four main parts in general supporting:

- WAMS connectivity with the data centre via by using mobile connections with private APNs

- WAMS connectivity with the data centre via xDSL or FTTH connections

- WAMS connectivity with the data centre via VPN by using satellite links.



Figure 16: SUNSEED general back-end infrastructure.

The datacentre is behind the firewall with load balancing functionality for physical separation from the access network part. It consist of several virtual servers hosting XMPP and MQTT brokers for receiving WAMS measurements as well as management servers for controlling WAMS nodes. Measurements are then sent from XMPP or MQTT server to the RABBITMQ broker, which distributes the data into several queues, each for a different measurement type (according to the description in deliverable D4.1.1). The data from each queue is then stored into raw database (MongoDB replica Set). There is also virtual server hosting GUI and API, for accessing and writing the data into the database. Access to the API has several layers of security:

API access secured with personal tokens,

HTTPS instead of unencrypted HTTP for transferring the data,

Implemented rate limiters for preventing too frequent requests or even DDOS attacks.

The ASR900 is a demarcation aggregation services router between Telekom Slovenije’s core network, other services delivery networks and the SUNSEED data centre. It is connected through Telekom IP/MPLS core to the JUNIPER MX480 MPLS core router. Juniper MX480 also has a connection to GGSN/EPG where WAMS traffic from different APNs is aggregated. On the other hand JUNIPER M320 router is used as a laboratory demarcation router to provide connection between IP/MPLS network and laboratory test environment. This separates the test environment from the IP/MPLS network and is used to for injecting test systems or services into the Telekom Slovenije’s core network. This router also serves as an interconnecting point to the Sunseed network for WAMS traffic coming via VPNs. Next, there is a test laboratory firewall (PF SENSE) used for firewalling purposes and the termination of OpenVPN sessions used for transporting WAMS traffic over the public internet via



satellite links. It enables also admin access for external users and connecting to test laboratory servers where we host open source monitoring platform Nagios. This is used for monitoring all the SUNSEED field trial devices via SNMP. On the mobile network part we use private APNs with no access to public internet and RADIUS server for authentication of the on-field devices. For the purpose of routing on the LTE router we use routing behind mobile station. In this case we send the LAN IP address information, (e.g., WAMS IP address) from the LDAP server to the GGSN as a radius profile server attribute called “framed-route”. This way we notify GGSN of a routing prefix for the specific location or mobile device, so that the network has the information about the LAN IP addresses of devices using such type of connection. Similar solution is used in case of devices connected through xDSL. Satellite access uses open VPN connections established between the on-field routers/modems and the test laboratory PF SENSE firewall. In all three cases the on-field devices receive IPs from the private address space. The traffic through the Telekom Slovenije IP MPLS core is transported using layer 3 MPLS VPN service, via several firewalls with or without load balancing functionality. Firewalls with such functionality enable additional layer of security by limiting the traffic from/to allowed hosts within the Sunseed cloud, filtering the source/destination ports and providing the security also on the application layer. The part related to mobile network’s APNs is more into details depicted in Figure 17.

DATACENTER

SERVERS

DATACENTER

FW

BALANCER

MODEM

RUT950

or

IR915P

GGSN

APN

sunseed.test.ts.si

TS MPLSMODEM

RUT950

or

IR915P

APN

sunseed.ng.ts.si

MODEM

RUT950

or

IR915P

APN

sunseed.to.ts.si

MODEM

RUT950

or

IR915P

APN

sunseed.ra.ts.si

MODEM

RUT950

or

IR915P

APN

sunseed.kp.ts.si

WAMS

and/or

data

concentrator

Multiservice Edge

Router

JUNIPER

MX480

Aggregation

Services Router

JUNIPER

M320 LAB

Aggregation

Services Router

ASR 9000

MODEM

RUT950

or

IR915P

WAMS

and/or

data

concentrator

WAMS

and/or

data

concentrator

WAMS

and/or

data

concentrator

WAMS

and/or

data

concentrator

Figure 17: SUNSEED APNs related back-end infrastructure.

WAMS and/or PLC data concentrators are connected to the mobile modems/routers. These connect to private APNs, where:

- “sunseed.ng.ts.si” is used for the area of Kromberk,

- “sunseed.to.ts.si” is used for the area of Kneža,

- “sunseed.ra.ts.si” is used for the area of Razdrto,

- “sunseed.kp.si” is used for the area of Bonfika, and

- “sunseed.test.test.si” is used for testing the services.



In case of APN connections, the GGSN is used as the PDP contexts’ termination point. It is also connected into the SUNSEED VRF (virtual route forwarding) instance of the MPLS core via JUNIPER MX480 to access test laboratory servers and the SUNSEED datacentre. The part related to xDSL is depicted in more detail in Figure 18.

DATACENTERSERVERS

DATACENTERFW

BALANCER

BRASSE600

TS MPLS

xDSLCPE

WAMSand/ordata

concentrator

Aggregation Services Router

JUNIPER M320 LAB


ASR 9000

Figure 18: SUNSEED xDSL related infrastructure.

When WAMS devices and/or data concentrators are using xDSL access points they connect to customer premises equipment (CPE), which are typically xDSL modems with DHCP functionality. CPE establishes the PPPoE connection to the BRAS SE600 which enables connection with the SUNSEED VRF instance within the IP MPLS network. The part related to VPN is depicted mode detail in Figure 19.



Public internet

SATELITE

INHAND 915

WAMS

SATMODEM

WI-FIBRIDGE

VPN

WI-FIBRIDGE

DATACENTERSERVERS

DATACENTERFW

BALANCER


ASR 9000

TS MPLS


JUNIPER M320 LAB

PF SENSEFW BALANCER

WAMSOption 1

Option 2

Figure 19: SUNSEED VPN related back-end infrastructure.

Here, WAMS devices connect to modem/router directly or over Wi-Fi bridges. Next, the satellite modem connects to the satellite network, connected to public Internet. Therefore, we use OpenVPN connection between the modems/routers and the PF Sense firewall balancer in the Telekom Slovenije’s test laboratory, to make the connection private. As already discussed above, the traffic is then securely routed to datacentre via JUNIPER M320 and ASR 9000 routers.

3.2 Security architecture for credential management

The security architecture implemented in SUNSEED is described in Figure 20. This architecture is based on an authorization delegation model. In such model, “Resource servers” (entities controlling access to some “resource” delegate the administration of the access control to some or all of their resources to an external authorization server. Figure 20 summarizes this security architecture enabling smart devices (WAMS or smart meters) to communicate securely with remote applications located in the cloud. On the left hand side of the figure the data collecting devices (smart meters or WAMS) need to obtain credentials in order to send their data to some »data dissemination' service«. Such service will be in the case of SUNSEED based upon the use of a publish/subscribe protocol such as MQTT or XMPP. The devices negotiate and obtain the credentials required during a security bootstrap phase involving the authenticated interaction with an authorization server centralizing the definition of all access rights policies for the SUNSEED communication ecosystem.



Figure 20 : Communication and security architecture

For WAMS devices, the credentials obtained after the security bootstrap phase are securely stored in an embedded secure element soldered on the WAMS Printed Circuit Board. On the right hand of Figure 20, the data transmitted by the WAMS devices is handled by a number of cloud applications which need to »subscribe« to the information streams originating for the SUNSEED devices. Credentials are be also required by those applications to enable them to subscribe to the desired information. Those credentials are also obtained in a security bootstrap phase via a negotiation with the authorization server. Figure 21 shows a specific implementation of the architecture described in Figure 20 corresponding to the actual workflow which will be validated in during the SUNSEED pilot.



Figure 21: Practical implementation of security architecture

This pilot involves the deployment of WAMS device in the field. Such devices may be either connected directly to the Wide area network using for example a global LTE connection. They may also be located in a Local Area Network and transmit their data via a router or possibly a data concentrator. MQTT is the protocol used by the WAMS to transmit their data. The transmitted MQTT messages transit via an MQTT server located in the cloud. In order to publish data via the MQTT server, the WAMS devices need to obtain credentials from the Authorization server using the protocols described [1]. In the pilot, the data originating from the WAMS will be received by a database application located in the cloud will index and archive the data. At a later stage, the data will be delivered to requesting data processing and analysis applications. The database application will subscribe to the information topics used used by the WAMS devices to publish. Credentials will be required to be able to subscribe; they will also be obtained via a negotiation with the authorization server.

3.3 Credential provisioning to the WAMS node

Figure 22: describes the manufacturing and provisioning workflow involved to produce an operational WAMS node. This workflow involves the following steps:

1. The embedded secure element is loaded at manufacturing time with a software image and with initial secrets enabling at a later stage to remotely provision diversified new secrets inside the secure element.

2. The initially configured secure element is soldered on the WAMS PCB during the WAMS hardware manufacturing phase.



3. A predefined software image including the operating system and all the application software is loaded in the WAMS flash memory. At this point the WAMS may be deployed in the field and connected to either to a Wide area or a private area network.

4. An initial security bootstrap phase is taking place, including the following steps: a. The initial secrets provisioned in all secure elements (embedded in all manufactured

WAMS) at manufacturing time are mass loaded during a one time operation in the secure element management platform to make possible remote management by this platform of the credentials contained in the WAMS secure elements.

b. The communication services to secure (in the SUNSEED case, MQTT publishing) are configured in the authorization server as well as access control policies, granting publishing rights to the suitable MQTT topics to all the WAMS deployed in the fields

c. The definition of access control policies in the authorization server triggers the authorization server to initiate the remote provisioning of the authentication credentials as well as the access control lists in the MQTT server via a resource server proxy as described in [1].

d. The authorization server initiates a request to the secure element management platform to provision the authentication credentials ( in the SUNSEED case PKI client certificates) in the secure element. The flow of operations involved is also described in [1]. The credentials are provisioned via the WAMS IP communication channel (PLC, wifi, 4G...)

5. The WAMS is operational and ready to securely publish MQTT messages using the credentials remotely provisioned in the secure element.

Figure 22: security and credential provisioning workflow



4 Communication measurement procedures for the SUNSEED field

trial

This chapter focuses on the measurement procedures envisaged to be used during the SUNSEED trial. First, the measurement procedure for derivation of the traffic characteristics generated by the different measurement end nodes is explained in Section 4.1. The procedure for deriving the most important communication network key performance indicators (KPIs) such as end-to-end communication delay and throughput are illustrated in Section 4.2 and Section 4.3, respectively. The chapter is finalized with the measurement procedure for deriving the overall radio access KPIs in Section 4.4.

4.1 Smart grid traffic characterization measurements

By measuring the network traffic of WAMS and SM nodes in the field trial, an empirical traffic model that describes their communication patterns can be established. While the counter-based measurements used for the traffic measurement campaign in D3.1 did reveal some details about the WAMS-like devices under study (that were used since the WAMS nodes were not available at that time), a lot of information could not be extracted from those measurements. For example, it was not possible to characterize the distributions of packet sizes and inter-arrival times of packet flows, which is usually considered key to a traffic model. Only the measurement of cumulated amount of bytes within certain time intervals was possible. Therefore, during the SUNSEED trial, we intend to obtain more fine grained measurements of the communication network traffic in a smart grid network. The standard way to obtain such fine grained measurements is to capture network traffic traces, for example by using Wireshark, TCPdump or similar tools that output packet capture (PCAP) trace files.

4.1.1 PCAP tracing locations

PCAP traces can be captured on different types of devices provided that the operating system allows the tracing application to monitor the relevant network interfaces. This is possible for most off-the-shelf laptop and desktop computers running Windows, Linux and Mac OS operating systems. The most important decision for the SUNSEED trial in regards to tracing is where in the trial network to capture the trace files. For this, we have considered four options for trace locations, namely 1) tracing on all source nodes, 2) placement of tracing bridge between WAMS/SM and communication network, 3) tracing in TS core network, and 4) tracing at destination node. These options are described in more detail in Appendix A. Out of the considered options, the most suitable and realistic choice would be to use tracing in the TS core network. Even though the traffic model that can be extracted from such measurements may be slightly inaccurate as discussed in Appendix A, we consider the benefit of having a single measurement location and thus the possibility of capturing all trial measurements easily to outweigh this downside.



Figure 23: PCAP location for packet sniffing within Telekom Slovenija network used in the trial

4.1.2 Available information from PCAP traces

The PCAP trace files will provide us with a timestamped list of events on the monitored network interfaces. For each layer in the protocol stack, we will have access to the header and payload information, allowing us to perform very detailed analysis. The information that is considered important for our future analyses is the following:

Timestamp when the IP packet was generated

Source IP address and port

Destination IP address and port

Transport protocol used, e.g., UDP or TCP

Application type

Application payload size

Application type payload content

TCP sequence number

An example of a PCAP trace captured with a Wireshark is shown in Figure 24.



Figure 24: Example of PCAP trace, showing implementation of a SIP protocol from [2]

4.1.3 KPIs and statistics of interest from PCAP traces

In the following we will consider the KPIs and statistics of interest in the traffic analysis. First and foremost, KPIs should be measured that reveal the load that the SUNSEED trial imposes on TS’ cellular and core networks. In addition, the measurements should be used to create a statistical traffic model, which can be used to create and evaluate the load and performance of artificial scenarios, e.g. using network simulations. These statistical models should thus describe the traffic generation behaviour of individual WAMS and SM nodes. The KPIs and statistics can be calculated for different levels, namely:

for the entire SUNSEED trial (i.e., all data captured in PCAP trace)

for a specific trial location

for a particular device

per flow (TCP)

per message type (e.g., control data, measurement, …)

For the first three levels, it makes sense to consider KPIs for both uplink and downlink directions. The specific features of interest that we intend to characterize are addressed in the following and are inspired by similar work [3],[4]. Each of the features may be considered for the different levels specified above.

Traffic volume/bandwidth: Both the uplink and downlink traffic volumes should be considered, as well as the uplink/downlink ratio. From this, by taking the trial duration into



account, the occupied bandwidth/throughput can also be calculated. Both mean values and empirical probability distributions are of interest.

Packet size: The packet size is a key parameter to be determined for a traffic generation model. An empirical probability distribution of the used packet sizes is of highest interest here, since typically in M2M packets are either small control packet or large bulk payload packets, therefore the mean is not very representative.

Time series of packet generation: The timely behaviour of packet generation can be characterized using either the packet inter-arrival time (subtracting timestamps of consecutive packet events) or by analysing the periodicity from the power spectral density (periodogram) or autocorrelation function.

Session analysis: For TCP sessions, there are several aspects that would be interesting to consider, namely the active time, average session length, session inter-arrival time, and session total traffic volume. Both empirical probability distributions and mean values are of interest.

Joint parameter determination: Fitting inter-arrival and packet size independently may not result in the correct aggregated traffic volume. Therefore, a joint model parameter fitting procedure should be used to find the combination of parameters that best describe the observed data. For this, machine learning techniques can be used to model parameter estimation.

4.1.4 Traffic Characterization Measurement procedure

The SUNSEED communication architecture uses a publish-subscribe principle, where the number of data flows depends on the number of active traffic sources and the number of subscribers. We need to know the following configuration information while measuring the traffic:

a) How many nodes are installed and what is the configuration for reporting data (e.g. the time interval)?

b) How many nodes are activated (or allowed to publish part of the information)? c) What information can be published (as specified in D2.2.2 [5]) and what has been subscribed

to by different applications?

It should be stressed here that the non-SUNSEED nodes will have an influence on the delay/throughput of the generated packets and not on the generated traffic volume and inter-arrival times from the SUNSEED nodes. To be able to characterize how well the proposed communication architecture scales with an increasing number of nodes, it may be necessary to gradually increase the number of active devices in the field trial traffic characterization experiment. That is, we should have different phases, where for each phase we note down the configuration and then we capture PCAP traces for a period of time, e.g., 1 week. Hereafter, we repeat for the next phase with a higher number of devices. Before the measurement campaign is started for each of the deployed SM, WAMS-SPM, and WAMS-PMC node the following information should be determined and saved:

a) The IP address of the node, which will be fixed and permanently bound to the IMSI of the LTE modems for the whole duration of the SUNSEED trial.



b) The serving cell of the node (and its access type UMTS or LTE, correspondingly TAC, etc.) i.e. where in the TS cellular network this node is located and served. It is expected that the serving cell is constant most of the time, since devices are static. However, handovers may happen in case that the serving cell becomes unavailable or if the radio propagation conditions change, e.g., due to adverse weather or new/moving obstacles affecting the propagation channel.

During the measurement campaign, the measurement reporting period of the different nodes (where applicable) should be known and saved. For example, as the WAMS-SPM measurement reporting period is configurable during the measurement campaign it should be known (logged) what reporting period was configured per WAMS-SPM node. Similarly, this measurement reporting configuration should be known and saved also for the SM and WAMS-PMC nodes. Additionally, during the measurement campaign the packets transmitted from the installed SM, WAMS-SPM and WAMS-PMC nodes should be marked with regard to their payload in order to distinguish between:

a) Power line measurement reports in uplink, commands towards the installed nodes, etc. which are all relevant for the derivation of the SG traffic models.

b) Communication network related reports in uplink from e.g. WAMS-SPM or WAMS-PMC node or from the Teltonika LTE modems (connected to PLC concentrators) as these packets should be excluded from the derivation of the SG traffic models.

4.2 WAMS-SPM end-to-end delay measurement

This section details the measurement procedure for the end-to-end delay from WAMS-SPM nodes. The following assumptions hold (see also Figure 25):

- WAMS-SPM nodes send power line measurements with time-stamp based on GPS time reference.

- WAMS-SPM nodes, APN server in LTE/UMTS core network and database server where all SUNSEED field trial data is stored are time-synchronized via GPS time reference.

The current proposal for deriving the end-to-end delay is to use IP packet trace as all WAMS-SPM nodes in the trial will have fixed IP addresses. Consequently, the time-stamps of the data packets are read at:

- APN server in LTE/UMTS core network. Subtract the time-stamp in the IP packet from the current GPS based time at APN server to calculate the WAMS-SPM to APN server delay TAPN,LTE and TAPN,UMTS for the LTE and UMTS network, respectively.

- SUNSEED database server. Subtract the time-stamp in the IP packet from the current GPS based time at SUNSEED’s database server to calculate the end-to-end delay between WAMS-SPM to SUNSEED APN delay.

Note here that in the SUNSEED trial the SM and WAMS-PMC nodes are not planned for GPS time synchronization with the e.g. SUNSEED’s APN server or the database server. Therefore, the end-to-end delay measurements for these nodes will not be possible. The measurement set-up also assumes that the delay between the APN server at Telekom Slovenija and the SUNSEED measurement database TAPN, Database is the same regardless if the packet has travelled via the LTE or the UMTS access network.



For the statistical processing of the measurement data the end-to-end delay measurements can be considered as one set or, alternatively, the end-to-end delay measurements can be divided into groups depending if the access network is LTE, UMTS, or satellite.

Figure 25 Schematic view on the end-to-end delay measurement procedure for WAMS-SPM nodes

4.2.1 Measurement approach

The phasor measurements obtained in the WAMS-SPM nodes are collected in a proactive manner to the SUNSEED database (DB) server, meaning that upon finishing a measurement burst, the WAMS-SPM assembles an IP packet, which is immediately queued and sent towards the DB server. For measuring the end-to-end delay of the time-critical WAMS-SPM measurements, we will exploit the fact that the WAMS-SPM nodes are very accurately time synchronized. Figure 26 shows how the latency measurements for the two measurement points (APN and DB) are obtained. In this figure, τ1 represents the WAMS-SPM to LTE/UMTS APN delay, while τ2 represents the end-to-end WAMS-SPM to SUNSEED APN delay.



The delay introduced by the communication network depends largely on the traffic load in the network, as well as the priority level of smart grid traffic in scheduling. During the SUNSEED trial there will be no prioritization of the SUNSEED traffic, in other words the traffic will be handled in best effort manner. Optionally, some prioritization policy of the SUNSEED traffic might be used in the core network (BRAS level). Thus, it would be interesting and relevant to measure the end-to-end delay performance at different traffic load of the day, and with (if feasible) different scheduling priority levels.

Figure 26: Measurement collection from WAMS-SPM. Upon completion of a measurement burst, a packet is assembled and sent to the TS core, where it is detected and a timestamp is made at the APN, and finally at the DB server, another timestamp is made when the data is

stored in the database.

4.3 End-to-end throughput measurement

In order not to distort the end-to-end delay measurements in Section 4.2 and consequently also the throughput measurements, as explained in this section, the reporting of the communication network measurements should be non-overlapping with the actual power line measurements reporting. For example, the reporting of the communication network measurements is preferably done at the end of each power line measurement reporting session, which session can be for example at the end of the measurement hour, day or week. It should be stressed here that from the point of view of SG real-time control of the end-to-end throughput is not as relevant as the end-to-end delay. Furthermore, due to the limitations in the level of details in the communication network measurements it will not be feasible to determine the throughput for all individual nodes (e.g. the installed SM and WAMPS-PMC nodes due to lack of GPS time reference) and on all network segments (i.e. throughput in the radio access segment and/or PLC segment, mobile core, TS transport network after the SUNSEED APN etc.). Therefore, it is expected that the only end-to-end communication throughput measurement will be feasible for the WAMS-SPM nodes defined as the packet size divided by the packet end-to-end delay as determined in Section 4.2..



Additionally, as already mentioned above, another option for the throughput measurements would be to derive it from the (uplink and downlink) traffic volumes generated by the SM and WAMS-PMC nodes. Then appropriate measures have to be taken in order to estimate the “useful content”. By useful content we define the total data stripped out of packet headers and transmission artefacts (retransmissions, redundant coding etc.). This can be done in two ways:

From the PCAP traces (used for traffic characterisation) we can filter out measurement packets from a specific source-destination pair and calculate avg. throughput.

Count the amount of received data in database.

From this, by taking into account the aggregation of the downlink and uplink traffic volume, and the trial measurement duration, the occupied bandwidth/throughput can also be calculated. Both mean values and empirical probability distributions are of interest. Note that this approach for the end-to-end measurements is less accurate than the approach based on the end-to-end delay of the WAMS-SPM packets.

4.3.1 Throughput measurement setup in controlled laboratory

environment

We investigate the limits of PLC communication from smart meters to PLC concentrators under varying number of smart meters, reporting intervals and line conditions. Real time smart meter communication is bound at 1 min to 15 min reporting intervals. PLC concentrators are linked with LTE router-modems via telecom operator network to central database within utility network management environment. Gathered data are used for real time state estimation. Background It is preferred to use as small number of PMU units to achieve observation of distribution grids. Real time smart meter readings are used to augment PMU measurements to achieve full observability. Gathering measurements in real time from smart meters (e.g. 1 min reporting interval) may present challenge to PLC communication channel as it differs greatly from classical AMI for billing purposes smart meter data gathering, i.e. once per day. Test setup We set up a test measurement facility in laboratory-controlled environment that consists of:

Different types of Smart meters (37) (Landis+Gyr, Iskraemeco)

PLC concentrator (Landis+Gyr)

LTE router-modem (Teltonika RUT550)

PLC communication channel (power line), between smart meters and PLC concentrator

Ethernet communication channel, between PLC concentrator and LTE mode

Wireless communication channel, between LTE modem and database

Power load

Grid line measurement unit (Elspec G4500)

Ethernet packet measurement unit (Wireshark)

Remote management software for PLC concentrator and LTE modem

Database PLC channel is S-FSK 1, 3 phase smart meters of different types and producers are connected with single PLC line to PLC concentrator, and their reporting interval is varying (1 – 15 min).



Figure 27: End-to-end scheme

Figure 28: Lab environment - test bed for end-to-end telecommunication compatibilities

All measurements are gathered at PLC concentrator and transported to database via high bandwidth LTE (4G) cellular network. Transport is via RabbitMQ message queue protocol over TCP directly to SQL database. With using existing S-FSK technology, we transmit all needed billing data for DSO with common reliability and we try to definite additional data volume transfer for other needs like our applications in the project. We describe additional tasks in concentrator for periodic register reading for each meter. To get real financial valuation we decide to calibrate WAMS devices with official registered meters.



To receive full information we decided that is sufficient to read 10 primary values from meter registers (U1, U2, U3, I1, I2, I3, I0, PF1, PF2, PF3). In practice, we managed to read all data from 34 meter in time less than 2 minutes. In long-term test, we get all billing data (daily, 15min periods, and monthly data) and periodical reading of our 10 registers per meter each 5 minutes with 100% success.

Figure 29: time of reading 10 registers from 1 smart meter is less than 2 seconds)

Results and conclusion Having dense network of PLC-connected smart meters communicating with short reporting intervals over present S-FSK-specified PLC channels is a hard task. Error rates influence the state estimation significantly. LTE cellular communication is a viable communication link for gathered smart meter data. Disturbances from different home devices could override signal and there could be blind windows for receiving measuring data. With incoming new technology PLC OFDM based on ERDF G3 protocol we are expecting more reliable signal transfers and possibly more capacity for transmitted signals. Further work is required to valuate new incoming communication technologies and to establish field trial test results under operating distribution grid conditions that is planned within project itself. Objective: Imitation of real conditions and environment for operating measurement system. The goal is to imitate some real topology segments of future field test. On this model we can measure and observe behaviour of PLC communications depending on other influences (overvoltage’s, transients, disturbances, noise of nonlinear loads at end users....). The physical model can help us to test PLC communications at extreme and rear circumstances in the net and to fine tuning of computer model of the net to get more accurate results closer to reality.



Equipment: • HES/MDC • concentrator DC450 • Meters E450 • Basic Test bed with modular structure allowing different configurations • Changeable modules to imitate length and characteristics of power lines • Coupling and Filter circuits for injection of disturbance in to net • RF Signal Generator to imitate disturbances • Standard loads and loads from field coursing malfunction of PLC communication Test configuration:

Figure 30: Laboratory test LV power grid configuration

Purpose of measurement: The main purpose of this testing is to construct permanent modular testing bed to imitate real circumstances on the real power net and have possibility to repeat some transients or other



phenomena and repeated it many times and observe how they influence on correct work or what caused malfunction or aging of elements of measurement system. We like to build and define circuit with a common topology and constructed from most common widget of today measurement system with common loads to get some kind of “Nominal model”, where all parameter and values can be measured and results can be repeated. This measurement should be calibrate and comparable to field test as much as possible. On this basis we can in future compare new equipment (new products, different produces...), technologies (PLC G3) and evaluate with measured quantities and numbers to define advantages, cost efficient, interoperability and complementary with existing system. Some typical measured parameters are: Maximum length of line to the next meter, life time, capacity and reliability of data transmission, noise rate, resistance on different disturbances, and time between maintenance... To have possibility to examine some single events from field trials and to fine turning of computer simulations to get as much closer result to reality. The model will help us to do case studies to find reason for multiple similar malfunction of equipment by imitating the suspected influences in similar environments. Some typical problems on the fields are: over current disconnection without known reasons, low success rate of communication and too many relay connection on short distances, malfunction of power supply of meters because of high frequency disturbances, crosstalk over MV transformer stations... The method of measurement:

1) Definition and description of the network;

2) Calculating and preparing appropriate 4 pole wiring for imitation of topology of different realistic field scenarios; Commissioning the system and get familiar with its calibrations, possibilities and options; By changing the configuration of the circuit - determine the limits of the system

3) 4. Verifying operation of the test bed with field tries and makes calibrations

4) 5. To reconcile all scenarios and test with boundary conditions to ensure the repeatability of “normal model”. Task is to determinate the common values for today existing measurement systems in DSO.

5) With the addition of equipment from different manufacturers - compliance of equipment operation (interoperability)

6) Establishment of operation and testing of various types of communications between the communicator and measurement centres (PLC, GPRS ...)

7) Comparison of existing technology FSK with G3, and on the comparative methods to identify the differences and benefits of the new technologies

4.4 Radio access related measurements

The radio access related measurements are useful for obtaining information about the performance of the LTE (and UMTS where applicable) radio interface in uplink and downlink at the LTE (or UMTS) cells of interest during the SUNSEED trial. The approach for collecting these radio access related measurements is at the eNode B side via statistical counters as explained in Section 4.4.1. Additionally, relevant radio access information can be gathered via Gemalto's or Teltonika's LTE (or UMTS) modem as presented in Section 4.4.2 and 4.4.3, respectively.



4.4.1 Relevant measurements derived from statistical counters/KPIs at the eNB

Table 1

Parameter Equation

RACH (%) (EUtranCellFDD.pmRaSuccCbra + EUtranCellFDD.pmRaSuccCfra) / EUtranCellFDD.pmRaAttCbra + EUtranCellFDD.pmRaAttCfra) * 100

User download throughput (kbit/s)

(EUtranCellFDD.pmPdcpVolDlDrb + EUtranCellFDD.pmPdcpVolDlDrbTransUm – EUtranCellFDD.pmPdcpVolDlDrbLastTTI) / EUtranCellFDD.pmUeThpTimeDl * 1000

User upload throughput (kbit/s)

EUtranCellFDD.pmUeThpVolUl / EUtranCellFDD.pmUeThpTimeUl * 1000

Cell_DL_thr EUtranCellFDD.pmPdcpVolDlDrb + EUtranCellFDD.pmPdcpVolDlDrbTransUm – EUtranCellFDD.pmPdcpVolDlDrbLastTTI) / ROP2

Cell_UL_thr EUtranCellFDD.pmUeThpVolUl / ROP

IntP AVERAGE(EUtranCellFDD.pmRadioRecInterferencePwr)

I_Pucch AVERAGE(EUtranCellFDD.pmRadioRecInterferencePwrPucch)

PrbDl% Avg AVERAGE(EUtranCellFDD.pmPrbUtilDl)

PrbUl%Avg AVERAGE(EUtranCellFDD.pmPrbUtilUl)

Table 2: LTE radio counters.

Counter Description Unit

EUtranCellFDD.pmRaSuccCbra The number of successfully detected RA Message 3 for CBRA3.

PEG

EUtranCellFDD.pmRaSuccCfra The number of successfully detected RA Message 3 for CFRA4.

PEG

EUtranCellFDD.pmRaAttCbra The number of detected CBRA preambles in the cell.

PEG

EUtranCellFDD.pmRaAttCfra The number of detected CFRA preambles in the cell.

PEG

EUtranCellFDD.pmPdcpVolDlDrb The total volume (PDCP SDU) on Data Radio Bearers that has been transferred (UM and AM) in the downlink direction. When carrier aggregation is used,

1 kbit

2 ROP: result output period (in seconds), e.g. ROP = 900 in case of 15 min reports. 3 CBRA: contention-based random access preamble. Randomly selected preamble used for initial network access, access following a radio link failure, handover between cells, downlink data transfer requiring UE synchronization, uplink data transfer requiring UE synchronization. 4 CFRA: contention-free (explicitly signalled) random access preamble. Reserved preamble used for handover between cells and downlink data transfer requiring UE synchronization.



a PDCP SDU can be sent over multiple cells (PCell/SCell(s)). The total volume (PDCP SDU) on Data Radio Bearers that has been transferred (UM and AM) in the downlink direction is measured on PCell.

EUtranCellFDD.pmPdcpVolDlDrbTransUm The total volume (PDCP SDU) on Data Radio Bearers for RLC UM that has been transmitted in the downlink direction in the PDCP layer. When carrier aggregation is used, a PDCP SDU can be sent over multiple cells (PCell/SCell(s)). The total volume (PDCP SDU) on Data Radio Bearers for RLC UM that has been transmitted in the downlink direction in the PDCP layer is measured on PCell.

1 kbit

EUtranCellFDD.pmPdcpVolDlDrbLastTTI The total volume (PDCP SDU) on Data Radio Bearers that has been transferred (acknowledged by the UE) in the downlink direction in the last TTI when a buffer is emptied.

1 kbit

EUtranCellFDD.pmUeThpTimeDl The effective DL transport time comprises those periods from when the first part of the PDCP SDU of the DL buffer was transmitted on Uu until the buffer is emptied, excluding the TTI emptying the buffer. When carrier aggregation is used, a PDCP SDU can be sent over multiple cells (PCell/SCell(s)). The effective DL transport time, comprising those periods from when the first part of the PDCP SDU of the DL buffer was transmitted on Uu until the buffer is emptied, excluding the TTI emptying the buffer, is registered on PCell.

ms

EUtranCellFDD.pmUeThpVolUl The UL DRB volume used for UL UE Throughput. It comprises of the MAC SDU volume received on Uu, excluding the volume

kbit



received in the first 4 data receptions of an UL buffer transfer and the TTI emptying the UL buffer.

EUtranCellFDD.pmUeThpTimeUl The UL volume transfer time used for UL UE Throughput. It comprises of time periods from when the 5th MAC SDU data reception of an UL buffer transfer on Uu until the buffer is emptied, excluding the TTI emptying the buffer.

ms

EUtranCellFDD.pmRadioRecInterferencePwr The accumulated interference power for one PRB.

mW * 2^(-44)

EUtranCellFDD.pmRadioRecInterferencePwrPucch The measured Noise and Interference Power on PUCCH, according to 36.214. PDF ranges: [0]: N+I <= -121 [1]: -121 < N+I <= -120 [2]: -120 < N+I <= -119 [3]: -119 < N+I <= -118 [4]: -118 < N+I <= -117 [5]: -117 < N+I <= -116 [6]: -116 < N+I <= -115 [7]: -115 < N+I <= -114 [8]: -114< N+I <= -113 [9]: -113 < N+I <= -112 [10]: -112 < N+I <= -108 [11]: -108 < N+I <= -104 [12]: -104 < N+I <= -100 [13]: -100 < N+I <= -96 [14]: -96 < N+I <= -92 [15]: -92 < N+I

dBm/PRB5

EUtranCellFDD.pmPrbUtilDl A distribution that shows the downlink Physical Resource Block (PRB) pair utilization (total number of used PRB pairs by available PRB pairs) on the Physical Downlink Shared Channel (PDSCH). PDF ranges: [0]: 0 % <= Utilization < 10 % [1]: 10 % <= Utilization < 20 %

5 PRB: physical Resource Block. PRB defines number of subcarriers for a predetermined amount of time (time-frequency dimension) and it is handled by a scheduling function at the eNB.



[2]: 20 % <= Utilization < 30 % [3]: 30 % <= Utilization < 40 % [4]: 40 % <= Utilization < 50 % [5]: 50 % <= Utilization < 60 % [6]: 60 % <= Utilization < 70 % [7]: 70 % <= Utilization < 80 % [8]: 80 % <= Utilization < 90 % [9]: 90 % <= Utilization

EUtranCellFDD.pmPrbUtilUl A distribution that shows the uplink Physical Resource Block (PRB) pair utilization (total number of used PRB pairs by available PRB pairs) on the Physical Uplink Shared Channel (PUSCH). PDF ranges: [0]: 0 % <= Utilization < 10 % [1]: 10 % <= Utilization < 20 % [2]: 20 % <= Utilization < 30 % [3]: 30 % <= Utilization < 40 % [4]: 40 % <= Utilization < 50 % [5]: 50 % <= Utilization < 60 % [6]: 60 % <= Utilization < 70 % [7]: 70 % <= Utilization < 80 % [8]: 80 % <= Utilization < 90 % [9]: 90 % <= Utilization

4.4.2 Gemalto LTE modem

The relevant AT command for collecting the radio access parameters when the LTE modem is camping on the cell is: Syntax:^SMONI: ACT,EARFCN,Band,DL bandwidth,UL bandwidth,Mode,MCC,MNC,TAC,Global Cell ID,Phys-ical Cell ID,Srxlev,RSRP,RSRQ,Conn_state

Example Answer: ^SMONI: 4G,6300,20,10,10,FDD,262,02,BF75,0345103,350,33,-94,-7,NOCONN

The relevant parameter to be measured when the LTE modem is camping on the cell are RSRP and RSRQ. These two parameters can be used for deriving the coverage conditions (e.g. pathloss via RSRP value and knowledge of the transmission power of the cell’s reference signal – RS) and downlink interference/SINR conditions (e.g. by deriving the downlink SINR from the RSRQ value). Additionally, the Global Cell ID should be also recorded in order to be known which cell is serving the WAMS node. The relevant AT command when the LTE modem is in active state is: ^SMONI: ACT,EARFCN,Band,DL bandwidth,UL bandwidth,Mode,MCC,MNC,TAC,Global Cell ID,Phys-ical Cell ID,TX_power,RSRP,RSRQ,Conn_state Example Answer: ^SMONI: 4G,6300,20,10,10,FDD,262,02,BF75,0345103,350,90,-94,-7,CONN



The difference between idle and active ^SMONI response is that instead of the parameter Srxlev (i.e. signal strength threshold for base station selection as defined in TS 25.304) the Tx power (i.e. used uplink power) is measured and that the connection state changes from NOCONN to CONN. The used uplink power is useful to determine the following:

a) Maximum number of PRBs that can be used for the uplink transmission based on the estimated path-loss (from RSRP measurements) and average uplink interference level from the 15 min counter IntP or I_Pucch as illustrated in Section 4.4.1.

b) Possible uplink SINR for the uplink transmission, based on the average uplink interference level from the 15 min counter IntP or I_Pucch as illustrated in Section 4.4.1.

4.4.3 External communication equipment

Monitoring of communication equipment should be done in organized and manageable way. SNMP, or Simple Network Management Protocol, is a standard network management protocol widely used in TCP networks and provides a method of managing the device – SNMP agent, e.g. LTE modem (or other equipment) through the running the central computer of network management software, i.e. SNMP server. In production there is 3 different versions (v1, v2 and v3), which mainly differs in performance, security, confidentiality and manager-to-manager communications. For mobile clients in Sunseed pilot, comm. security is handled when setting up the connection, i.e. on mobile gateway (with private APNs), therefore there is no need to use controversial SNMP security model, but Community-Based SNMP v2 (SNMP v2c) satisfy the network management requirements. At protocol level SNMP defines following protocol data units (PDUs) namely:

- GetRequest to retrieve the value of variable or list of variables from client, - SetRequest to request for changing the value of variable or list of variables, - GetNextRequest to discover available variables ant their values, - GetBulkRequest where server request client to respond with multiple bindings in the request, - Response return variable bindings and acknowledge server’s request, - Trap which is used to send asynchronous notification from the agent to the server and - InformRequest which acknowledges asynchronous notification (due to unreliable UDP nature

of SNMP protocol).

In Sunseed pilot monitoring the communication infrastructure is managed both by the asynchronous (traps) and synchronous, i.e. period reporting (pooling) approach, such that first provides any sudden warning or errors, like “mobile signal strength has fallen below defined threshold” and the latter for monitoring statistics, like “number of transmitted upload or download bytes”. Request is defined by OID (Object Identifier), which uniquely identify managed objects in a MIB (Management Information Base) hierarchy. Example of pooling technique on Inhand LTE modem is reading the status of RSSI signal: Code when numeric-based OID is dislayed Numeric OIDs and raw values can also be displayed in more meaningful texual names and sensible formatted values using MIB files and cmd tool like snmptranslate. The example above identifies a particular object, which can be displayed using list of MIB identifiers (provided by vendor of comm. equipment, i.e. Inhand) to: Code when text-based OID is dislayed



SNMP community management also provide different access levels (“read-only”, “read-write”), hence enabling and using specific community strings also provides operating the equipment on remote (Figure 31).

Figure 31: SNMP community

Different equipment provides various OID, some of them are “general purpose”, while others remain in vendor’s domain. Regarding the Sunseed pilot, OIDs related to RF signal (RSSI signal strength, attached mobile base station), identification (imsi, equipment name, serial number) and network layer (number of UL and DL bytes, mobile IP, connection uptime) is of great importance. SNMP data from server is then being processed with monitoring tool (Nagios), where additional parameters can be calculated and displayed in graphical form, e.g. graphs of outages. Furthermore, data is also populated in dedicated GIS system, where one (SG operator) have overview and control over SNMP events in geo-spatial domain.



5 Field trial communication measurements and relation to

SUNSEED use cases

5.1 Measurements related to state estimation use case

The WAMS-SPM power line measurements, as well as additional SM and WAMS-PMC power line measurements will be reported via TCP/IP protocol meaning that the power line measurements will be eventually available at the SUNSEED database for post-processing. As a consequence, the issue is if the reported power line measurements is available on time (to be post-processed by the state estimation algorithm) and less of an issue if the power line measurements will be available at all. However, it can happen that particular measurement is not available or not valid due to some other reasons (e.g. transient phenomena in SPM as discussed in D4.1.1 [D4.1.1], communication outage from some nodes, …). The state estimation uses the following power line measurement:

SPM: o measurements/data: node_id , v1, v2, v3, th1, th2 , th3, psp_v, psp_th, status, f1, f2,

f3, f4, df1, df2, df3, df4, , stamp1, stamp2 o period: 1 second or more; format JSON see D2.2.2

PMC: o measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3,

stamp, o period: 1 second or more; format JSON see D2.2.2

SM: o measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3, vv1,

vv2, vv3, stamp, o period: 15 minutes; format JSON see D2.2.2

The overall delay Dall at which the data is available for further processing in data storage can be calculated as follow: Dall = Dprocesing_node + Dcommunications+ Dprocesing_storage While the processing delays at the node and on the server side are quite deterministic, the communication’s delay depends on the availability/congestion and the type of the communication path. Although the state estimation works also if some measurements are not available (resulting in higher error), it is of paramount importance that it waits at least Dall before processing the data and SE calculation. Thus, the knowledge about the communication delays is essential in particular as in some cases where the satellite link is used (Kneža testbed) it can be more than 250ms, which is a few times more than in LTE. During the SUNSEED trial, the reporting power line measurement period for the WAMS-SPM nodes will be configured from few minutes (e.g. 15, 10, 5, or 1) down to 1 second. Next to this, the reporting period of power line measurements of WAMS-PMC might be also configured towards lower values (e.g. lower than 15 min) in particular as it can provide more “up to date” injection information, which improves the state estimation. Thus, it is advantageous that WAMS-PMC



measurements are in line with WAMS-SPM. The SM measurements will be collected on 15 minutes interval, which is the standard metering interval. The trial measurements will aim at answering the following important questions:

a) Is it feasible to have real-time SG state estimation with the installed types of measurement nodes (WAMS-SPM, WAMS-PMC and SM) and using the installed communication technologies (e.g. LTE, UMTS, and PLC)? How real-time can we follow the SG state, is this on seconds or few minutes level? What are changes in DS between two consecutive intervals?

b) What is the impact on the communication network at the telecom operator (e.g. LTE or UMTS cellular network) in terms on resource/capacity use for different real-time SG state estimation periods (e.g. from 1 second to 15 min)?

c) Do the higher SE update rates justify the higher use of communication resources (e.g. use of communication resources vs. SE performance)?

d) Anything else…

5.2 Measurements related to demand-response use case

Demand-response use case can be considered as the subset of SE use case from the power line measurement perspective. It means that if the SE power measurements are in place no additional power line measurements need to be collected, as all necessary data are already available. However, in the case that demand response use case is used without SE, then the following power line measurements need to be collected:

PMC: o measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3,

stamp, o period: 15 minutes or less; format JSON see D2.2.2

SM: o measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3, vv1,

vv2, vv3, stamp, o period: 15 minutes; format JSON see D2.2.2

In addition to power line data there is additional data send from/to nodes that are connected to controllable appliances. Using FPAI these nodes send data regarding the control space of their energy consumption and receive data regarding allocation of their energy consumption:

Control space information o Registration of control space o Period On demand, typically less than once every 15 minutes; format XML see D2.2.2

and D4.4.1 o Update of control space o Period: On demand, typically less than once every 15 minutes; format XML see

D2.2.2 and D4.4.1Allocation information

Allocation information o Allocation o Period: On demand, typically less than once every 15 minutes; format XML see

D2.2.2 and D4.4.1 o Allocation update



o Period: On demand, typically less than once every 15 minutes; format XML see D2.2.2 and D4.4.1

As the general period for the demand-response use case is 15 minutes, the communication is not that sensible to delay, but rather to the reliability in particular for the control data.

5.3 Measurements related to SG outage localization use case

During the SUNSEED trial the installed WAMS-SPM and WAMS-PMC nodes will be configured to transmit ‘artificial power outage alarms/events’ while it is expected that the deployed SM nodes from Landys Gyr cannot be configured to send these ‘artificial outage messages’. As part of this outage alarms the GPS location of the WAMS-SPM or WAMS-PMC nodes will be reported. Using these enriched power outage messages it will be shown during the trial how quickly a power outage can be localized in the power grid.



6 Conclusions

The deliverable outlines the specific scenarios for communication solutions on separate field trial locations which are geographically quite distinguished. Regarding the proposed scenarios, the general communication SUNSEED back-end infrastructure with security architecture and provisioning the security credentials to the end nodes is defined. In particular, we focus on considered solutions for WAMS (SPM and PMC) and for smart meters, respectively. Of course, due to costs reduction and where possible, WAMS and smart meters can use the same gateway for a considered communication solution. For WAMS scenarios the typical wiring diagram scheme and an example of inventory material with technical specification was made. On the basis of coverage analyses from deliverable D5.1.2 the most appropriate communication scenario per each transformer station as the main functional location was assigned. In-depth analyses of the field trial locations and of the Telekom Slovenije infrastructure reviled that mobile and satellite communications are the most appropriate to connect majority of WAMS devices into the SUNSEED cloud. Therefore, we distinguish two communication scenarios, where WAMS (SPM/PMC) are using mobile or fixed modem/router (Scenario A) and WAMS SPM are using satellite modem/router (Scenario B). With respect to the analysis of fixed network access points presence and mobile network coverage from deliverables published in WP5 scenario A is further divided into three sub-scenarios, where gateway (modem/router) could connects the LTE network (scenario A.1), UMTS network (scenario A.2) or xDSL/FTTH network (scenario A.3). Regarding the position of a satellite antenna scenario B is also divided into two sub-scenarios where WAMS could be located at utility functional location and it is connected directly to satellite modem/router, while the location can have also Wi-Fi bridge to subordinate functional location (Scenario B.1). Other possibility is WAMS positions at subordinate functional location and connects to the satellite modem/router via Wi-Fi bridge (Scenario B.2). Similar as WAMS in scenario A, smart meters use mobile or fixed modems/routers as gateways for connecting into the SUNSEED cloud via mobile or fixed network. However, with regard to the exact smart meter model, they are connected to such gateways in different manner. We distinct two scenarios, where smart meters are connected to PLC data concentrator (DC), which is connected to mobile or fixed modem/router (Scenario C) or they are connected to mobile or fixed modem/router via RS485 to Ethernet converter (Scenario D). Sunseed IT selected security will insure the proper filtering of data transmission at the IP level. It consists of four main parts in general to support WAMS connectivity with the data centre via by using mobile connections with private APNs, WAMS connectivity with the data centre via xDSL or FTTH connections and WAMS connectivity with the data centre via VPN by using satellite links. The datacentre is behind the firewall with load balancing functionality for physical separation from the access network part. For end WAMS nodes access the authorization delegation model is used. . In such model, “Resource servers” (entities controlling access to some “resource” delegate the administration of the access control to some or all of their resources to an external authorization server. During the trial measurements it is important to characterize the ongoing traffic in terms of e.g. packet sizes, inter-packet arrival times, aggregated amount of traffic, etc. This realistic traffic



characterization based on the trial communication sessions is useful for modifying the existing traffic models used for performance analysis of communication networks for smart grids. In this deliverable a detailed procedure is explained regarding capturing packets from the smart grid applications during the trial via PCAP traces as well as definitions of the most important traffic KPIs (e.g. traffic volume, packet size, time series of packet generation, etc.). Next to the traffic characterization it is important to measure the communication network performance parameters. This deliverable describes end-to-end delay and throughput measurement procedures that will be used in the trial. For the end-to-end delay it is important to realize sufficiently accurate time synchronization between the two communication end-points. Finally, during the communication sessions it is important to measure the most important radio access communication network conditions (e.g. signal level, interference level, radio access load, etc.) in order to understand the end-to-end delay and throughput performance. This deliverable defines two approaches for measuring the radio access network conditions: the approach of using radio access related performance counters as provided by the vendor of radio access network, and the approach of using AT commands and/or SNMP commands to extract the necessary radio access performance information from the communication modems. During the SUNSEED trial the measurements of the electricity grid and the communication network will be used to assess the feasibility and the performance requirements for the three defined SUNSEED use cases, namely the real-time state estimation, demand response, and outage localization use case. For the electricity grid state estimation use case a procedure is defined regarding the testing of the accuracy and up to date information of the state-estimation function. As the reporting interval of the WAMS-PMC and WAMS-SPM nodes is varied from e.g. 15 min down to 1 s the impact on the communication network and end-to-end delays will be investigated as well as the ability of the state-estimation algorithm to predict a correct value and on time. For the demand response use case next to the measurements needed for the state estimation additional measurements and control information messages will be collected. As the delay in demand response cycles is typically around 15 min, for this use case it is important to check the reliability of these additional measurements and control messages (i.e. to check the stability and reliability of the control process) and not on the end-to-end delay performance for these messages. Finally, by instructing the WAMS-SPM and WAMS-PMC nodes to initiate ‘fake outage’ messages enriched with location information (e.g. GPS coordinates for WAMS-SPM) the feasibility of automatic grid outage localization will be assessed as the final SUNSEED use case.



7 References

[1] FP7 SUNSEED Project, Deliverable D3.4.1, “Security architecture for smart grids“, SUNSEED Consortium, 2015.

[2] Available from http://www.backtrack-linux.org/wiki/index.php/Pentesting_VOIP, retrieved on March 2016.

[3] Shafiq, Muhammad Zubair, et al. "A first look at cellular machine-to-machine traffic: large scale measurement and characterization." ACM SIGMETRICS Performance Evaluation Review 40.1 (2012): 65-76.

[4] Shafiq, M. Zubair, et al. "Large-scale measurement and characterization of cellular machine-to-machine traffic." IEEE/ACM Transactions on Networking (TON) 21.6 (2013): 1960-1973.

[5] FP7 SUNSEED Project, Deliverable D2.2.2, “Preliminary specification of SUNSEED field trial requirements for communication, metering, and data collection“, SUNSEED Consortium, 2016.

d3.3.2 communication solution design for field trial...

Documents