accumulo summit 2015: preventing bugs: how phemi put accumulo to work in the field [geo]
TRANSCRIPT
Preventing BugsPutting Accumulo To Work In The Field
Russ Weeks, PHEMI
Orchard
Precision Agriculture
Sensors
Gateways
Why We ❤ Accumulo
• Continuous Aggregations
• Time-series Data
• Spatiotemporal Data
Farming is Risky!
Adrian Baggaley, FruitForum
Horticultural Sciences Department, University of Florida
USDA
Aggregation Schema
Time-Series Data
• Sensor messages arriving in constant, high-velocity stream
• Common use case: “Show me the last 2 weeks of data for sensor X”
No problem!
Row CF CQ Val
M0 msg timestamp 2014/07/01T19:30
M0 msg sensor_id 1138
M0 msg msg_id 1138:2014/07/01T1930
M0 dev battery 3.3v
M0 env temp 20.3c
M0 env windspd 2.96m/s
Lexicode + Transpose…
Row CF CQ Val
msg:msg_id:1138:2014/07/01T2020 M842 1
msg:msg_id:1138:2014/07/01T2010 M767 1
msg:msg_id:1138:2014/07/01T2000 M684 1
msg:msg_id:1138:2014/07/01T1950 M592 1
msg:msg_id:1138:2014/07/01T1940 M356 1
msg:msg_id:1138:2014/07/01T1930 M0 1
• But… my old database could dereference those message IDs automatically!
• All I had to do was this join, see…
Time-Series Index
Row CF CQ Val(sensor_id:1138,2014/07/01T2020,battery,3.3v) […] M842 1(sensor_id:1138,2014/07/01T2020,windspd,1.83m/s) […] M842 1(sensor_id:1138,2014/07/01T2020,temp,18.6c) […] M842 1(sensor_id:1138,2014/07/01T2010,battery,3.2v) […] M767 1(sensor_id:1138,2014/07/01T2010,windspd,1.51m/s) […] M767 1(sensor_id:1138,2014/07/01T2010,temp,18.8c) […] M767 1
Spatiotemporal
Map Tiles: OpenStreetMap
Pos. 0 Pos. 212-1
Spatiotemporal
OK, But Why Not HBase?
• Iterators are awesome
• Customer is interested in cell-level security
• Future applications may involve controlled data-sharing
Thanks!