time series analysis by javascript ll matsuri 2013
TRANSCRIPT
![Page 1: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/1.jpg)
Time Series analysis by JavaScript
LL matsuri 2013/08/24
1
![Page 2: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/2.jpg)
Time Series analysis by JavaScript
前処理
LL matsuri 2013/08/24
2
![Page 3: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/3.jpg)
{twitter: “muddydixon”}✓Data Mining✓Data Visualization✓Love D3.js
✓working @ NIFTY
3
![Page 4: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/4.jpg)
NIFTY Cloud
4
![Page 5: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/5.jpg)
Visualization✓What is Visualization?
communicating informationclearly and effectively
5
![Page 6: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/6.jpg)
Visualization on Web✓Profit✓Watch KGI / KPI continuously✓Share visualization via URL
✓empowered representation by HTML5 / css3
-> D3.js6
![Page 7: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/7.jpg)
Visualization on Web : D3.js
7
![Page 8: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/8.jpg)
Time Series Visualization✓Visualize “time series” of ✓KGI / KP of services✓Metrics of system resources
8
![Page 9: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/9.jpg)
Sample{"time":"2013-01-02T08:54:31.000Z","cpu":0.6283,"memory":0.4427}{"time":"2013-01-02T08:54:22.000Z","cpu":0.6103,"memory":0.6494}{"time":"2013-01-02T08:54:24.000Z","cpu":0.6096,"memory":0.4888}{"time":"2013-01-02T08:54:25.000Z","cpu":0.6811,"memory":0.641}{"time":"2013-01-02T08:54:14.000Z","cpu":0.6028,"memory":0.6452}{"time":"2013-01-02T08:54:06.000Z","cpu":0.5565,"memory":0.4346}{"time":"2013-01-02T08:54:10.000Z","cpu":0.4161,"memory":0.358}{"time":"2013-01-02T08:55:01.000Z","cpu":0.5256,"memory":0.5969}{"time":"2013-01-02T08:55:47.000Z","cpu":0.6145,"memory":0.3213}{"time":"2013-01-02T08:56:24.000Z","cpu":0.6896,"memory":0.5109}{"time":"2013-01-02T08:56:35.000Z","cpu":0.443,"memory":0.633}{"time":"2013-01-02T08:56:11.000Z","cpu":0.4746,"memory":0.4777}{"time":"2013-01-02T08:56:42.000Z","cpu":0.503,"memory":0.5732}{"time":"2013-01-02T08:56:38.000Z","cpu":0.4731,"memory":0.4024}{"time":"2013-01-02T08:57:01.000Z","cpu":0.405,"memory":0.5203}{"time":"2013-01-02T08:57:54.000Z","cpu":0.4496,"memory":0.5324}{"time":"2013-01-02T08:57:04.000Z","cpu":0.3528,"memory":0.3145}{"time":"2013-01-02T08:57:49.000Z","cpu":0.6659,"memory":0.4285}{"time":"2013-01-02T08:57:19.000Z","cpu":0.3237,"memory":0.6979}{"time":"2013-01-02T08:58:31.000Z","cpu":0.4045,"memory":0.6581}{"time":"2013-01-02T08:58:45.000Z","cpu":0.3134,"memory":0.3261}{"time":"2013-01-02T08:58:44.000Z","cpu":0.4382,"memory":0.419}{"time":"2013-01-02T08:58:38.000Z","cpu":0.6983,"memory":0.3493}{"time":"2013-01-02T08:58:34.000Z","cpu":0.6645,"memory":0.6272}{"time":"2013-01-02T08:59:54.000Z","cpu":0.604,"memory":0.3287}{"time":"2013-01-02T08:59:25.000Z","cpu":0.574,"memory":0.4856}{"time":"2013-01-02T08:59:19.000Z","cpu":0.4775,"memory":0.3998}{"time":"2013-01-02T08:59:34.000Z","cpu":0.5047,"memory":0.6702}{"time":"2013-01-02T08:59:46.000Z","cpu":0.672,"memory":0.5021}{"time":"2013-01-02T08:59:53.000Z","cpu":0.3278,"memory":0.6679}{"time":"2013-01-02T08:59:29.000Z","cpu":0.4494,"memory":0.3319}{"time":"2013-01-02T08:59:43.000Z","cpu":0.6149,"memory":0.6988}{"time":"2013-01-02T09:00:40.000Z","cpu":0.4249,"memory":0.6441}{"time":"2013-01-02T09:00:52.000Z","cpu":0.5086,"memory":0.5189}{"time":"2013-01-02T09:00:02.000Z","cpu":0.3182,"memory":0.4765}{"time":"2013-01-02T09:00:02.000Z","cpu":0.4201,"memory":0.4376}{"time":"2013-01-02T09:01:20.000Z","cpu":0.4181,"memory":0.5856}{"time":"2013-01-02T09:01:46.000Z","cpu":0.6682,"memory":0.5992}{"time":"2013-01-02T09:01:22.000Z","cpu":0.3787,"memory":0.4777}{"time":"2013-01-02T09:01:54.000Z","cpu":0.6057,"memory":0.3374}{"time":"2013-01-02T09:01:42.000Z","cpu":0.3485,"memory":0.6665}{"time":"2013-01-02T09:01:45.000Z","cpu":0.3628,"memory":0.6092}{"time":"2013-01-02T09:01:47.000Z","cpu":0.6096,"memory":0.5961}{"time":"2013-01-02T09:01:02.000Z","cpu":0.5355,"memory":0.6442}
9
![Page 10: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/10.jpg)
可視化エンジニアマネージャー
デイリーで売上データ見せて
了解しました
10
![Page 11: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/11.jpg)
可視化エンジニアマネージャー
やっぱり、月次と週次で
…了解しました
11
![Page 12: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/12.jpg)
可視化エンジニアマネージャー
あと売上の平均と分散も
うっ、つらい
12
![Page 13: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/13.jpg)
Summarizekeyvalues = {}for d in data time = new Date(d.time.getFullYear(), d.time.getMonth(), d.time.getDate(), d.time.getHours(), d.time.getMinutes()) keyvalues[time] = [] unless keyvalues[time] keyvalues[time].push d
aggregated = []for time, values of keyvalues obj = {time: time, cpu: 0, memory: 0} for d in values obj.cpu += d.cpu obj.memory += d.memory aggregated.push obj
console.log aggregated
13
![Page 14: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/14.jpg)
ProblemsLarge
Time perspectives / Statistics Avoid to get data each time
Hourly / Daily / Weekly / Monthly / YearlySummary / Average / Stdev
14
![Page 15: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/15.jpg)
Problems
メンドクサイ
15
![Page 16: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/16.jpg)
Series.js
16
![Page 17: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/17.jpg)
Series.js✓Providing utility methods for time series✓sum, sum square, mean, median, quantile✓variance, standard devience, covariance, correlation✓auto covariance, auto correlation✓Providing aggregation✓minutes, hour, day, week, month, year✓Coming soon✓auto regression, spectrum✓moving average, holt-winters, exponential moving average
17
![Page 18: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/18.jpg)
Series.js APIs example# class methodsSeries.sum([1,2,3,4,5]) #=> 15Series.mean([1,2,3,4,5]) #=> 3
# instance methodss = new Series()s.put d for d in [1,2,3,4,5]s.mean() #=> 3
18
![Page 19: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/19.jpg)
Series.js APIs example# accessorSeries.y((d)-> d.v).sum([{v:1},{v:2},{v:3},{v:4},{v:5}]) #=> 15s = new Series().y((d)-> d.v)s.put d for d in [{v:1}, {v:2}, {v:3}, {v:4}, {v:5}]s.mean() #=> 3
19
![Page 20: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/20.jpg)
Series.js APIs example# aggregationdata = [{t: 1, v:1}, {t: 1, v:2}, {t:2, v:3}, {t:2, v:4}, {t:1, v:5}]Series.y((d)-> d.v).aggregation(Series.sum).key((d)-> d.t)(data)#=> [{t:1, v: 8}, {t: 2, v: 7}]
20
![Page 21: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/21.jpg)
Sample{"time":"2013-01-02T08:54:31.000Z","cpu":0.6283,"memory":0.4427}{"time":"2013-01-02T08:54:22.000Z","cpu":0.6103,"memory":0.6494}{"time":"2013-01-02T08:54:24.000Z","cpu":0.6096,"memory":0.4888}{"time":"2013-01-02T08:54:25.000Z","cpu":0.6811,"memory":0.641}{"time":"2013-01-02T08:54:14.000Z","cpu":0.6028,"memory":0.6452}{"time":"2013-01-02T08:54:06.000Z","cpu":0.5565,"memory":0.4346}{"time":"2013-01-02T08:54:10.000Z","cpu":0.4161,"memory":0.358}{"time":"2013-01-02T08:55:01.000Z","cpu":0.5256,"memory":0.5969}{"time":"2013-01-02T08:55:47.000Z","cpu":0.6145,"memory":0.3213}{"time":"2013-01-02T08:56:24.000Z","cpu":0.6896,"memory":0.5109}{"time":"2013-01-02T08:56:35.000Z","cpu":0.443,"memory":0.633}{"time":"2013-01-02T08:56:11.000Z","cpu":0.4746,"memory":0.4777}{"time":"2013-01-02T08:56:42.000Z","cpu":0.503,"memory":0.5732}{"time":"2013-01-02T08:56:38.000Z","cpu":0.4731,"memory":0.4024}{"time":"2013-01-02T08:57:01.000Z","cpu":0.405,"memory":0.5203}{"time":"2013-01-02T08:57:54.000Z","cpu":0.4496,"memory":0.5324}{"time":"2013-01-02T08:57:04.000Z","cpu":0.3528,"memory":0.3145}{"time":"2013-01-02T08:57:49.000Z","cpu":0.6659,"memory":0.4285}{"time":"2013-01-02T08:57:19.000Z","cpu":0.3237,"memory":0.6979}{"time":"2013-01-02T08:58:31.000Z","cpu":0.4045,"memory":0.6581}{"time":"2013-01-02T08:58:45.000Z","cpu":0.3134,"memory":0.3261}{"time":"2013-01-02T08:58:44.000Z","cpu":0.4382,"memory":0.419}{"time":"2013-01-02T08:58:38.000Z","cpu":0.6983,"memory":0.3493}{"time":"2013-01-02T08:58:34.000Z","cpu":0.6645,"memory":0.6272}{"time":"2013-01-02T08:59:54.000Z","cpu":0.604,"memory":0.3287}{"time":"2013-01-02T08:59:25.000Z","cpu":0.574,"memory":0.4856}{"time":"2013-01-02T08:59:19.000Z","cpu":0.4775,"memory":0.3998}{"time":"2013-01-02T08:59:34.000Z","cpu":0.5047,"memory":0.6702}{"time":"2013-01-02T08:59:46.000Z","cpu":0.672,"memory":0.5021}{"time":"2013-01-02T08:59:53.000Z","cpu":0.3278,"memory":0.6679}{"time":"2013-01-02T08:59:29.000Z","cpu":0.4494,"memory":0.3319}{"time":"2013-01-02T08:59:43.000Z","cpu":0.6149,"memory":0.6988}{"time":"2013-01-02T09:00:40.000Z","cpu":0.4249,"memory":0.6441}{"time":"2013-01-02T09:00:52.000Z","cpu":0.5086,"memory":0.5189}{"time":"2013-01-02T09:00:02.000Z","cpu":0.3182,"memory":0.4765}{"time":"2013-01-02T09:00:02.000Z","cpu":0.4201,"memory":0.4376}{"time":"2013-01-02T09:01:20.000Z","cpu":0.4181,"memory":0.5856}{"time":"2013-01-02T09:01:46.000Z","cpu":0.6682,"memory":0.5992}{"time":"2013-01-02T09:01:22.000Z","cpu":0.3787,"memory":0.4777}{"time":"2013-01-02T09:01:54.000Z","cpu":0.6057,"memory":0.3374}{"time":"2013-01-02T09:01:42.000Z","cpu":0.3485,"memory":0.6665}{"time":"2013-01-02T09:01:45.000Z","cpu":0.3628,"memory":0.6092}{"time":"2013-01-02T09:01:47.000Z","cpu":0.6096,"memory":0.5961}{"time":"2013-01-02T09:01:02.000Z","cpu":0.5355,"memory":0.6442}
21
![Page 22: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/22.jpg)
Summarize with Series.jsSeries.y((d)-> d.cpu).aggregation(Series.sum) .minute((d)-> d.time)(data)#=> [{ t: 'Sun Jan 13 2013 23:29:00 GMT+0900 (JST)', y: 2.0095 }, { t: 'Sun Jan 13 2013 23:30:00 GMT+0900 (JST)', y: 3.4370 }, { t: 'Sun Jan 13 2013 23:31:00 GMT+0900 (JST)', y: 1.58180 }, { t: 'Sun Jan 13 2013 23:32:00 GMT+0900 (JST)', y: 2.2742 }, { t: 'Sun Jan 13 2013 23:33:00 GMT+0900 (JST)', y: 3.4541 }, { t: 'Sun Jan 13 2013 23:34:00 GMT+0900 (JST)', y: 4.6035 }, { t: 'Sun Jan 13 2013 23:35:00 GMT+0900 (JST)', y: 2.1165 }, { t: 'Sun Jan 13 2013 23:36:00 GMT+0900 (JST)', y: 1.6177 },
22
![Page 23: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/23.jpg)
TODO
PerformanceRefactoringDocument
23
![Page 24: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/24.jpg)
TODO
Client side
24
![Page 25: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/25.jpg)
TODO
Client sideさっきpushしました
25
![Page 26: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/26.jpg)
CROSS 2014
26
![Page 27: Time Series Analysis by JavaScript LL matsuri 2013](https://reader033.vdocuments.net/reader033/viewer/2022042815/557c3600d8b42acc738b4e92/html5/thumbnails/27.jpg)
以下、ビールを飲んで笑顔な技術者の顔が続いていると思ってください
27