cet(capture everything)プロジェクトにおけるﰀ機械学...
TRANSCRIPT
-
SOFTWARE JAPAN 2016
CET(Capture EveryThing)
ICT
2
-
2
-
,
2006: 2006-2008:
2008-2015:
2013: 2015: /
3
-
=
-
5
CSWEB
AP
RD
IT
201210110
-
6
-
6
-
8
CET
-
CET
9
-
CET
10
-
CET
11
CET Capture EveryThing
ELB
Elasticsearch,Kibana
BigQuery
S3
APIELB Cloud BigtableGCP
Cloud Pub/Sub
Cloud Dataproc,Apache Spark
-
Air
12
-
Spark Streaming
UU
1
13
-
Spark Streaming URL
2
CET Capture EveryThing
ELB Cloud Pub/Sub
Cloud DataprocApache Spark
14
Amazon SNS
AWS Lambda
Slack
-
3
15
-
16
CET/
-
17
SQL / DB
-
CETGCP
18
AWSGCP1/10CET http://engineer.recruit-lifestyle.co.jp/techblog/2015-11-27-yoshida-interview-2/
-
CET(AET)
19
AWS S3
Elastic Search
TreasureData
RedShift
Spark Notebook
Docker
AWS S3
Cloud Bigtable
RUNDECK
Adhoc(EC2)
Spark(EMR)
Compute Engine
-
/
COLD: RedShift, TreasureData/
HOT: CETElastic Search /
1,000 (AWS)
r3.2xlarge
BigTableAPI
Rundeck Ansible
20
-
Rundeck
21
-
DBIDAUC
22
-
23
-
24
-
25
-
26
-
27
/
-
28
-
29
Play frameworkDeep)
-
30
-
31
-
32
-
SparkREC2 | Tech Blog | RECRUIT LIFESTYLEhttp://engineer.recruit-lifestyle.co.jp/techblog/2015-08-19-sparkr/
4gihyo.jp http://gihyo.jp/dev/serial/01/recruit-lifestyle/0004
1/3
33
-
@JAWS-UG Meguro #2 http://www.slideshare.net/RecruitLifestyle/ss-53400381
CET | Tech Blog | RECRUIT LIFESTYLE http://engineer.recruit-lifestyle.co.jp/techblog/2015-11-02-yoshida-interview-1/
2/3
34
-
AWSGCP1/10CET http://engineer.recruit-lifestyle.co.jp/techblog/2015-11-27-yoshida-interview-2/
3/3
35