vernemq @ paris erlang user group june 29th 2015
TRANSCRIPT
![Page 1: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/1.jpg)
erlang.paris Meetup / June 29th 2015, Paris / André Graf (@der_graf)
VerneMQ - A distributed MQTT message broker developed in Erlang
![Page 2: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/2.jpg)
Intro
- André Graf, Co-Founder at Erl.io (2012)- Programming in Erlang since 2008- Basel Switzerland
@der_graf
![Page 3: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/3.jpg)
Simple Publish/Subscribe protocol over TCP/IP that utilizes
a centralized message broker
MQTT
![Page 4: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/4.jpg)
Different use cases…
MQTT
![Page 5: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/5.jpg)
https://www.flickr.com/mmmavocado/
![Page 6: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/6.jpg)
https://www.flickr.com/photos/christianhaugen
![Page 7: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/7.jpg)
https://www.flickr.com/photos/janitors
![Page 8: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/8.jpg)
Different use cases… ...but similar requirements.
MQTT
![Page 9: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/9.jpg)
constrained devices in constrained environments
MQTT
![Page 10: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/10.jpg)
| different QoS Levels | small header sizes (min 2B)| durable sessions| last will testament
MQTT
![Page 11: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/11.jpg)
Some MQTT Facts
- Current Spec Version 3.1.1- open OASIS Standard since 2014- Invented 1998 at IBM and Arcom
![Page 12: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/12.jpg)
VerneMQ Architecture from a bird’s eye perspective
![Page 13: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/13.jpg)
Erlang VM
ninenines/ranch
ninenines/cowboy
vmq_server
vmq_plugin
helium/eleveldb
helium/plumtree
uwiger/jobsvmq_commonsfeuerlabs/exometer_core
vmq_passwd
vmq_acl
vmq_bridge
vmq_systree
vmq_snmp
vmq_graphite
larshesel/vmq_elixir
![Page 14: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/14.jpg)
erlio/vernemq release project
basho/nodepackage
basho/cuttlefish
basho/clique
rebar/rebar3
![Page 15: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/15.jpg)
vmq_reg_trie<<gen_server>>
3 processes / MQTT connection
vmq_server
vmq_session<<gen_fsm>>
vmq_ranch<<ranch_protocol>>
vmq_queue<<gen_fsm>>start_link start_link
vmq_reg<<gen_server>>
vmq_reg_sup<<supervisor>>
vmq_reg_trie<<gen_server>>
monitor
ETSETS
![Page 16: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/16.jpg)
vmq_reg_trie<<gen_server>>
vmq_server
vmq_session<<gen_fsm>>
vmq_ranch<<ranch_protocol>> sync
vmq_reg<<gen_server>>
vmq_reg_sup<<supervisor>>
vmq_reg_trie<<gen_server>>
ETS
vmq_session<<gen_fsm>>
vmq_ranch<<ranch_protocol>>
vmq_queue<<gen_fsm>>async batch/async
matchfold
MQTT publish example
TCP Conn Client A
TCP Conn Client BETS
PUB/a/b/c
SUB/a/b/+
![Page 17: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/17.jpg)
Clustering VerneMQ
![Page 18: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/18.jpg)
Distribution
What is ACTIVELY distributed?
| Subscriber Data
| Online Messages
| Cluster Status Information
| ...
![Page 19: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/19.jpg)
Distribution
What is NOT distributed?
| offline messages (future)
![Page 20: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/20.jpg)
Distributing Subscriber Data
- Initially using Mnesia- Move to Plumtree/LevelDB
Paradigm Shift:
Consistent -> eventual Consistent
![Page 21: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/21.jpg)
Distributing online Messages
- Initially over Erlang Distribution layer- Move to own communication layer
- buffering in case of ‘nodedown’s- batching requests for higher throughput
![Page 22: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/22.jpg)
Extending VerneMQ
![Page 23: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/23.jpg)
erlio/vmq_plugin
%% iterates over all Plugins serving the Hook 'HookName'
vmq_plugin:all(HookName :: atom(), HookArgs :: [any()]).
%% iterates over all Plugins serving the Hook 'HookName'
%% until one returns 'ok' | {'ok', any()}
vmq_plugin:all_till_ok(HookName :: atom(), HookArgs :: [any()]).
%% calls the top priority Plugin that serves the Hook 'HookName'
vmq_plugin:only(HookName :: atom(), HookArgs :: [any()]).
![Page 24: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/24.jpg)
erlio/vmq_plugin
%% enabling an OTP Application Pluginvmq_plugin_mgr:enable_plugin(Application :: atom()).
%% in Application.app.src{env, [
{vmq_plugin_hooks, [
{HookName, Module, Function, Arity, []}
...
]}
]}
![Page 25: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/25.jpg)
erlio/vmq_plugin
Available Hooks:
- auth_on_register/5- auth_on_publish/6- auth_on_subscribe/3- on_register/3- on_publish/6- on_subscribe/3- on_deliver/6 (planned)
![Page 26: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/26.jpg)
erlio/vmq_plugin
Available Hooks:
- msg_store_write_sync/2- msg_store_write_async/2- msg_store_delete_sync/1- msg_store_delete_async/1- msg_store_fold/2- msg_store_read/1
![Page 27: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/27.jpg)
erlio/vmq_plugin
Checkout vmq_plugin and the Demo Plugin App
https://github.com/erlio/vmq_plugin
https://github.com/dergraf/vernemq_demo_plugin
![Page 28: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/28.jpg)
To Sum Up
- MQTT is a great Protocol for various use cases (monitoring/metering/mobile/..)
- VerneMQ brings scalability and powerful extensibility to the MQTT world.
- Erl.io provides support and development services for VerneMQ
![Page 29: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/29.jpg)
http://verne.mq | http://erl.io@der_graf | @vernemq | @erlio_basel
Merci Beaucoup!Je suis content de répondre a vos questions.
![Page 30: VerneMQ @ Paris Erlang User Group June 29th 2015](https://reader030.vdocuments.net/reader030/viewer/2022032620/55c5a26cbb61eb48468b481f/html5/thumbnails/30.jpg)
References
https://verne.mq
http://mqtt.org
https://github.com/erlio/vernemq
https://github.com/erlio/vmq_plugin
https://github.com/dergraf/vernemq_demo_plugin
https://github.com/helium/plumtree
http://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf
https://groups.google.com/forum/#!topic/mqtt/c1dlCdA5eOc