a scalable and distributed electrical power monitoring system utilizing cloud computing
DESCRIPTION
CUTE 2013TRANSCRIPT
A Scalable and Distributed Electrical Power Monitoring System
U:lizing Cloud Compu:ng
Ryousei Takano, Hidemoto Nakada, Toshiyuki Shimizu, Tomohiro Kudoh
Informa(on Technology Research Ins(tute,
Na(onal Ins(tute of Advanced Industrial Science and Technology (AIST), Japan
CUTE2013@Da Nang, Vietnam, Dec. 19 2013
Outline
• Background • System overview • Design of hardware and soKware • Deployment at AIST campus • Summary
2
Background
3
• The power consumpPon of data centers and networks becomes an issue of vital importance to IT industries.
• In Japan, there is an urgent need for green IT technologies aKer March 11th, 2011.
Fukushima Daiichi Nuclear Power Plant
Google data center in the Dalles, Oregon
Mo:va:on • VisualizaPon is key to plan power savings. • The total system cost and scalability are problem.
– Our server room has over 100 racks. – Our campus are geographically distributed in Japan.
• The system has to be low-‐cost, scalable, and also ease to develop applicaPons. ➡ Cheap power monitoring hardware ➡ Power monitoring soKware uPlizing cloud compuPng ➡ A simple REST API
4
System overview
5
…
…
Data collecting unit
Viewer
Alert
Google App Engine
Data store
Update power usage using REST w/ JSON Retrieve data using
REST w/ JSON
Data collecting unit
1
2
3
4
Power measuring unit
Data collecting unit
Applications
Data store
Observe the state of power consumption Plan power saving
Small start Go big
6
4 sensors 32 ports = 128 sensors
Google App Engine
Data store
...
Google App Engine
Data store
Sensors can be incrementally installed.
Low-‐cost power measuring unit
7
RJ-‐45 port
Signal processing board (dsPIC30F3013)
Clamp-‐on current sensor (max: 4)
• Send data to data collecPng unit every second. • The producPon cost is approximately 120 USD, including the cost of 4 current sensors.
Data collec:ng unit (1/2)
8
• Gather data from up to 32 power measuring unit • Push data to GAE
– Can be placed behind NAT
To power measuring unit (Not Ethernet, data transfer and power supply)
To GAE via the Internet (Ethernet/100BaseT)
Data collec:ng unit (2/2)
9
RJ-45 ports x 32
Power Measuring Unit
CPU board (T-SH7706LSR) - SH3 Linux - Buildroot 2011.05 - pmon.py Serial/Parallel converter
(Xilinx Spartan-3E)
data store
Data store and REST protocol
10
pmon.py pmon.py
pmon.py Application
Application
config DB
power logging DB Configuration
Update
Get, query
Power measuring unit “sensor”
Current sensor “probe”
…
…
Data collecting unit “unit”
data store
Google App Engine
• PaaS cloud service for web applicaPons – Java, Python, and Go are supported – Your applicaPon will have URL like hgp://XXXX.appspot.com
• Scalable and stable data storage – Data are replicated to 5 different datacenters – Allows 2 of them to be lost during operaPon
• Maintenance free – No need to manage, almost
• Cost effecPve – Almost free of charge
11
REST API
12
path method descrip:on
/update POST Upload data
/latest GET Get all data for the last minute
/latest,N GET Get all data for the last N minutes
/summary.s/YYYYmmDDHHMMSS,N GET Get all data for each second start from YYYYmmDDHHMMSS, for N seconds
/summary.m/YYYYmmDDHHMM,N GET Get all data for each minute start from YYYYmmDDHHMM, for N minutes
/summary.h/YYYYmmDDHH,N GET Get all data for each hour start from YYYYmmDDHH, for N hours
/summary.d/YYYYmmDD,N GET Get all data for each day start from YYYYmmDD, for N days
/query.s/LOC/YYYYmmDDHHMMSS,N GET Get data for locaPons that name start with LOC
/query.m/LOC/YYYYmmDDHHMM,N GET
/unit-‐config/UNIT_ID GET Get configuraPon data
/unit-‐config/UNIT_ID PUT Set configuraPon data
Update from Data collec:ng unit
Each data collecPon unit sends data every 20 seconds
– POST the following JSON string
13
{ "id": "UNIT_ID" "Pme": "1319837460” /* elapsed seconds from the UNIX epoch Pme */ "power": { /* data for the last 20 seconds per measurement point */ "sensor0.0": [VAL0, VAL1, VAL2, VA3, ..., VAL19], "sensor0.1": [VAL0, VAL1, VAL2, VA3, ..., VAL19], "sensor1.0": [VAL0, VAL1, VAL2, VA3, ..., VAL19], .... } }
GAE xxx.appspot.com/update
Data retrieval
An applicaPon periodically (e.g., 1 min) gets data from GAE
– GET the following JSON string
14
GAE xxx.appspot.com/latest,N
{ "Pme": “1319837460” /* epoch Pme */ "PmeStr": “201110290631” /* human readable Pme in JST */ "power": { "LOCATION0": [1234] "LOCATION1": [1234] "LOCATION2": [1234] "LOCATION3": [1234] "LOCATION4": [1234] ... } }
Viewer applicaPon
Deployment at AIST campus
15
…
…
Server room
Data collecPng unit
10 data collecPng units 177 power measuring units 620 measurement points
Sensor module Clamp-‐on current transformer
Data collecPon unit
Clean room
Viewer
billing service
Google App Engine
Datastore
Update power usage using REST w/ JSON Retrieve data using
REST w/ JSON
Data collecPng unit
1
1
2
2 3
1
2
3
gather
store
view
Installa:on in AIST server room
16
Power measuring unit
Clamp-on current sensor
Power distribution board
data store
Data collecting unit in free access floor
Summary • Our proposed system helps reduce total system cost and
improve scalability by employing low-‐cost power measuring units (30 USD per measurement point), and uPlizing cloud compuPng. – The development of the system was completed within 3 months. – We have successfully operated it over 2 years for to provide an
electricity billing service and evaluate the power efficiency of data processing middleware.
• Future work – Tolerance for network failures. – More applicaPons: e.g., server consolidaPon on a private cloud to
reduce power consumpPon.
17
Q&A
18
This research was parPally supported by the NEDO research project enPtled “Research and Development Project for Green Network/System Technology (Green IT Project).”
Thanks for your agenPon!
Visualiza:on Applica:ons
19
less than 90% of the upper limit less than 95% of the upper limit more than 95% of the upper limit Offline
(a) Web applicaPon (b) Desktop applicaPon