shib: webui tool provides crossover of hive and mpp
DESCRIPTION
TRANSCRIPT
![Page 1: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/1.jpg)
Shib:WebUI tool provides crossover
of Hive and MPP@tagomoris2014/07/08
Hadoop Conference Japan 2014 LT
14年7月8日火曜日
![Page 2: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/2.jpg)
@tagomorisLINE Corporation Analytics Platform Team
14年7月8日火曜日
![Page 3: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/3.jpg)
14年7月8日火曜日
![Page 4: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/4.jpg)
14年7月8日火曜日
![Page 5: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/5.jpg)
14年7月8日火曜日
![Page 6: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/6.jpg)
Shib:WebUI tool provides crossover
of Hive and MPP
14年7月8日火曜日
![Page 7: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/7.jpg)
14年7月8日火曜日
![Page 8: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/8.jpg)
14年7月8日火曜日
![Page 9: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/9.jpg)
14年7月8日火曜日
![Page 10: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/10.jpg)
Hive Clients
•Hive CLI / Beeline
•Hue (Beeswax)
•Other (Shib, WebHive, ...)
14年7月8日火曜日
![Page 11: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/11.jpg)
Troubles 1.hive> SELECT param, count(*) FROM tbl WHERE .... GROUP BY param;
14年7月8日火曜日
![Page 12: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/12.jpg)
Troubles 1.
(40 minutes later...)
14年7月8日火曜日
![Page 13: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/13.jpg)
Troubles 1.VALUE10032 1VALUE10033 1VALUE10034 2VALUE10035 1VALUE10036 1VALUE10037 1VALUE10038 508VALUE10039 1VALUE10041 2Time taken: 2263.201 secondshive>
14年7月8日火曜日
![Page 14: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/14.jpg)
Troubles 1.VALUE4932 1VALUE4933 1VALUE4934 2VALUE4937 1VALUE4938 1VALUE4942 1VALUE4943 1VALUE4944 1VALUE4945 2VALUE4946 1VALUE4947 1
14年7月8日火曜日
![Page 15: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/15.jpg)
Troubles 1.
Save all of your query’s results
AUTOMATICALLY
14年7月8日火曜日
![Page 16: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/16.jpg)
Troubles 2.hive> alter table tbl drop partition (....);
hive> drop table tbl;
14年7月8日火曜日
![Page 17: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/17.jpg)
Troubles 2.
Separate Query executor
FROM Administrator’s Client
14年7月8日火曜日
![Page 18: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/18.jpg)
Troubles 3.hive> set mapred.job.priority=‘VERY_HIGH’;
hive> SELECT ....
14年7月8日火曜日
![Page 19: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/19.jpg)
Troubles 3.hive> create temporary function foobar as ‘....’;
hive> SELECT foobar(param) ...;
FAILED: SemanticException [Error 10011]: Line 1:96 Invalid function ‘foobar’
hive>
14年7月8日火曜日
![Page 20: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/20.jpg)
Troubles 3.
Enforce well-configured environment
for almost all of daily queries
14年7月8日火曜日
![Page 21: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/21.jpg)
Troubles 4.Hey, please count yesterday’s logs.
Sure. Wait few minutes...
(10 minutes later...)Done.
Could you send me by .xlsx?
Ah....
14年7月8日火曜日
![Page 22: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/22.jpg)
Troubles 4.Could you send me by .xlsx?
(copy files from HDFS)(open TSV files)
(paste all lines on Excel)(open mailer and write e-mail)
(send a mail)
Ah....
OK. Of course.
14年7月8日火曜日
![Page 23: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/23.jpg)
Troubles 4.(paste all lines on Excel)
(open mailer and write e-mail)(send a mail)
(find to miss to attach xlsx)
(send a mail again)
14年7月8日火曜日
![Page 24: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/24.jpg)
Troubles 4.Could you send me by .xlsx?
(15 minutes later...)
Ah....
OK. Of course.
Done!
14年7月8日火曜日
![Page 25: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/25.jpg)
Troubles 4.(15 minutes later...)
Done!
Oh, I also need result about the day before yesterday!
14年7月8日火曜日
![Page 26: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/26.jpg)
Troubles 4.
Make all queries shared with permalink and user-friendly formats!
14年7月8日火曜日
![Page 27: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/27.jpg)
Troubles 4. (fixed)Hey, please count yesterday’s logs.
Sure. Wait few minutes...
OK. I started query on this URL.So download result after completion.
Estimated time is 10 minutes.
Thanks!
14年7月8日火曜日
![Page 28: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/28.jpg)
Shib
14年7月8日火曜日
![Page 29: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/29.jpg)
Shib
•Hive Web Client (w/ hiveserver1/2)
•Query permalinks•Result datastore (TSV/CSV)
•“SELECT” only! (DDL prohibited)•Database/table access control•Query list sharing by tag•HTTP Proxy API (POST to execute query!)
14年7月8日火曜日
![Page 30: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/30.jpg)
Shib:WebUI tool provides crossover
of Hive and MPP
14年7月8日火曜日
![Page 31: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/31.jpg)
Shib
•Multi query engine support
•Switch between hiveserver and hiveserver2
•Facebook Presto support
•Switch between Hive and Presto!
14年7月8日火曜日
![Page 32: Shib: WebUI tool provides crossover of Hive and MPP](https://reader034.vdocuments.net/reader034/viewer/2022051818/54b754ee4a7959737d8b4571/html5/thumbnails/32.jpg)
Shib will supports more features.
Enjoy stress-less queries!
14年7月8日火曜日