Download - The Future of Messaging: RabbitMQ and AMQP
![Page 1: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/1.jpg)
The Future of Messaging: RabbitMQ and AMQP
Eberhard Wolff Architecture & Technology Manager
adesso AG, Germany
![Page 2: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/2.jpg)
About me • Eberhard Wolff • Architecture & Technology Manager at adesso • adesso is a leading IT consultancy in Germany • Speaker • Author (i.e. first German Spring book) • Blog: http://ewolff.com • Twitter: @ewolff • [email protected]
![Page 3: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/3.jpg)
Overview • Why Messaging, AMQP and RabbitMQ
• Basic AMQP
• Advanced AMQP
• Advanced Spring-AMQP
![Page 4: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/4.jpg)
RPC • Predominant approach
– RMI, SOAP Web Services, CORBA, HttpInvoker, Burlap, Hessian
• Calls remote methods with parameter • …and waits for response • Problems:
– Explicitly tells the server what to do i.e. tight coupling
– What about network failures? – What about long latencies?
![Page 5: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/5.jpg)
Why Messaging? • Decoupling
– Data, no action i.e. receiver can react arbitrarily
– Asynchronous i.e. decoupled by time
• Reliable – Message can be stored-and-
forwarded – Redelivery until message
processed • Solves typical problems of
distributed systems
Component
Component Messages
![Page 6: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/6.jpg)
Why Messaging? • But: Requires different architecture • Very different from calling remote methods • Asynchronous
• AJAX has the same model
• See for example “Patterns of Enterprise Integration”
![Page 7: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/7.jpg)
Why AMQP? • Open standard protocol • Standard wire protocol • i.e. just one client library – no matter which
implementation you are using • Less vendor lock in • Efficient
– Binary wire protocol • Support in all major languages • Supported on most OS platforms
![Page 8: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/8.jpg)
What about JMS? • JMS has been the default for Java messaging
system for 10+ years • But:
• Only standardized on the API level • Less flexible than AMQP
• Mapping AMQP/JMS is being defined • There is
git://github.com/pieterh/openamq-jms.git
![Page 9: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/9.jpg)
Why Rabbit? • Because it has a kewl name • Numerous protocols supported • Most popular choice on EC2 • Foundation for demanding systems e.g.
NASA’s cloud initiative Nebula • Implemented in Erlang • Clustering built in • Currently in 2.4.1 • Supports AMQP 0.8, 0.9, 0.9.1
![Page 10: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/10.jpg)
Broad Support in RabbitMQ
![Page 11: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/11.jpg)
Broad Support in the JVM Space • Grails Plug In • Java Client • Scala / Lift support
• We will discuss Spring support in detail • Spring AMQP project 1.0.0.RC1 • http://www.springsource.org/spring-amqp
![Page 12: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/12.jpg)
Why Erlang? • Originally designed for telephone
switches by Ericsson • Much easier to develop scalable and fault
tolerant systems (by factors) • See Motorola presentation:
http://www.slideshare.net/Arbow/comparing-cpp-and-erlang-for-motorola-telecoms-software
• Good tool for reliable and scalable systems
![Page 13: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/13.jpg)
Erlang‘s Model
Light weight
process with state
Monitor
Link to monitor, restart
Light weight
process with state
Light weight
process with state
Messages Messages
![Page 14: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/14.jpg)
Why Erlang? • Let it crash
– If a process fails, it can be easily restarted – Different approach to fault tolerance – Otherwise lots of error handling
• Message Passing in the Core – RabbitMQ is a messaging system…
• Light-weight process model – Scalabiliy
![Page 15: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/15.jpg)
Very Basic AMQP • Queues: Store messages • Queues might be
– Durable: Survive server restarts – Exclusive: For one connection – autoDelete: Deleted if connection closes
• Queue usually created by consumer • All resources are dynamic • Producer sends a message to a Queue • Let’s see some code…
![Page 16: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/16.jpg)
Spring’s RabbitTemplate • Send & receive message • AmqpTemplate: Generic AMQP interface • RabbitOperations: Rabbit specific interface:
(adds just a callback) • RabbitTemplate: Implementation • Spring might provide support for other AMQP
implementations later • Common interface
![Page 17: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/17.jpg)
Spring’s MessageConverter • Messages are binary data • RabbitTemplate uses MessageConverter
to convert between objects and messages • Can also send binary data if preferred • Default: SimpleMessageConverter
– byte[] directly transferred – String converted with configurable encoding – Serializable are serialized – Content type set accordingly
• JsonMessageConverter converts from / to JSON using Jackson
• MarshallingMessageConverter converts from / to XML using Spring's OXM mapping
![Page 18: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/18.jpg)
Spring‘s AdminTemplate • Main purpose: Configure the AMQP
infrastructure • E.g. create queues
• AmpqAdmin: Generic AMQP interface • RabbitAdmin: Rabbit specific
![Page 19: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/19.jpg)
Code ConnectionFactory conFactory = new SingleConnectionFactory("localhost");RabbitTemplate template = new RabbitTemplate(conFactory);RabbitAdmin admin = new RabbitAdmin(conFactory);admin.declareQueue(new Queue("myQueue"));template.convertAndSend("myQueue", "Hi AMQP!");String receive = (String) template.receiveAndConvert("myQueue");Assert.assertEquals("Hi AMQP!", receive);
![Page 20: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/20.jpg)
Basics of AMQP • Sending messages directly to queues is not
enough • What about e.g. pub / sub?
• Exchange: Route messages (stateless) • Messages are byte-streams • Example used the default exchange
• More dynamic, flexible and cleaner than JMS
![Page 21: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/21.jpg)
AMQP in a nutshell Exchange routes message Stateless Usually created by producer No queue: Message discarded
X
Queues buffer messages Usually created by consumer
Binding binds an Exchange to a Queue
![Page 22: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/22.jpg)
AMQP in a nutshell
AMQP protocol
Producer and Consumer might be written in Java, C#, Python, Ruby …
X
C
C
P
RabbitMQ AMQP protocol
![Page 23: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/23.jpg)
Exchange: Route Messages • The type of Exchange defined the routing
algorithm used • Binding provides selector for routing • Exchange is addressed by name
• Some standard types • Can provide additional ones
X
![Page 24: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/24.jpg)
Fanout Exchange • Broadcast to all bound queues • Fast • Simple
• amq.fanout is mandatory
• To broadcast information
X
![Page 25: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/25.jpg)
X
C
C
C
P
Fanout
Fanout Exchange X
![Page 26: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/26.jpg)
Queue fanoutQueue = new Queue("fanoutQueue");admin.declareQueue(fanoutQueue);FanoutExchange fanoutExchange= new FanoutExchange("myFanout");admin.declareExchange(fanoutExchange);admin.declareBinding( BindingBuilder.from(fanoutQueue). to(fanoutExchange));template.setExchange("myFanout");template.convertAndSend("Hi Fanout!");String receive = (String) template.receiveAndConvert("fanoutQueue");Assert.assertEquals("Hi Fanout!", receive);
![Page 27: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/27.jpg)
Direct Exchange • Routing based on one routing key • amq.direct and the default Exchange (no
name) always exist
• To send work orders to a specific worker
X
![Page 28: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/28.jpg)
X
C
C
C
P
Direct Exchange
express
normal
express normal
Direct Exchange
![Page 29: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/29.jpg)
Queue directQueue = new Queue("direct");admin.declareQueue(directQueue);admin.declareBinding(BindingBuilder .from(directQueue) .to(new DirectExchange("amq.direct")) .with("helloKey"));template.setExchange("amq.direct");template.convertAndSend("amq.direct","dropMe", "I will be dropped!");template.convertAndSend("amq.direct","helloKey", "Hi Direct!");Assert.assertEquals("Hi Direct!", template.receiveAndConvert("direct"));Assert.assertNull( template.receiveAndConvert("direct"));
![Page 30: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/30.jpg)
Topic Exchange • Routing based on routing pattern • amq.topic is mandatory
• E.g. for public / subscribe scenarios
X
![Page 31: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/31.jpg)
Topic Exchange
X C
C
P
Topic Exchange
order.*
invoice.*
order.DE invoice.USD
![Page 32: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/32.jpg)
Headers Exchange • Routing based on one or more headers and
an expression • amqp.match is mandatory
• Complex routing roles
X
![Page 33: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/33.jpg)
Other Features • Message can be persistent • Request / response using correlations
possible
• Redelivery / acknowledgement possible
• Clustering with e.g. Linux HA possible • ...or send message through multiple channels
and drop duplicates
![Page 34: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/34.jpg)
More about RabbitMQ and Spring
![Page 35: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/35.jpg)
Configuring Rabbit Resources with Spring
• Spring enables decoupling of your application code from the underlying infrastructure
• The container provides the resources
• The application is simply coded against the API
![Page 36: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/36.jpg)
<bean id="connectionFactory" class="org.sfw.amqp.rabbit.connection.SingleConnectionFactory"> <property name="username" value="guest"/> <property name="password" value="guest"/> <constructor-arg value="localhost" /> </bean>
Configuring a ConnectionFactory
• Can easily modify configuration options
![Page 37: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/37.jpg)
<bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate"> <constructor-arg ref="connectionFactory" /> <property name="routingKey" value=”invoice.USD" /> </bean>
Defining a RabbitTemplate Bean • Provide a reference to the ConnectionFactory • Optionally provide other references
– MessageConverter – Routing key and exchange to be used if none is
specified
![Page 38: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/38.jpg)
The MessageListener • So far: Calling receive() on RabbitTemplate • Needed: Something that is called when a new
message appears • The API defines this interface for
asynchronous reception of messages
public void onMessage(Message) { // handle the message }
![Page 39: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/39.jpg)
Spring’s MessageListener Container
• Spring provides lightweight containers to call MessageListeners
• SimpleMessageListenerContainer • Advanced scheduling and endpoint
management options available • i.e. thread pools, concurrent consumers,
transaction handling
![Page 40: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/40.jpg)
Defining a Message Listener Container
• Every time a new message appears on my.amqp.queue the messageListener is called
<bean class="org.sfw.amqp.rabbit.listener.SimpleMessageListenerContainer"> <property name="connectionFactory" ref="connectionFactory" /> <property name="queueNames" value="my.amqp.queue" /> <property name="messageListener" ref="messageListener" /> </bean>
![Page 41: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/41.jpg)
Spring's message-driven objects • MessageListener means the receiver
depends on Spring API • Why not just a POJO? • MessageListenerAdapter takes a POJO and
makes it a MessageListener • i.e. calls consume on Bean consumer <bean id="messageListenerAdapter"
class="org.sfw.amqp.rabbit.listener.adapter.MessageListenerAdapter"> <property name="delegate" ref="consumer" /> <property name="defaultListenerMethod" value="consume" /> <property name="messageConverter" ref="jsonMessageConverter" /> </bean>
![Page 42: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/42.jpg)
Easier Using Namespaces
• Results in the same Spring Beans
<rabbit:listener-container connection-factory="connectionFactory“ message-converter="jsonMessageConverter"> <rabbit:listener ref="consumer" method="consume"
queue-names="my.amqp.queue2" /> </rabbit:listener-container>
![Page 43: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/43.jpg)
Consumer code
• No dependency on AMQP! • But: What about the result of the method? • Send to the Reply-To address given in
message properties with same correlationId as original method
@Componentpublic class Consumer { public String consume(String message) { return …; }}
![Page 44: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/44.jpg)
Client Code
• Message sent to destination with routing key • Reply-To set to exclusive, autodelete, non-
durable queue • Response received through Reply-To
converted and returned • Easy request-response! • Beware of potential latency
String response = (String) rabbitOperations.convertSendAndReceive( "my.fanout", "", "test");
![Page 45: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/45.jpg)
Create Environment Using Namespaces
• ...if you don‘t like API calls
<rabbit:fanout-exchange name="my.fanout2"> <rabbit:bindings> <rabbit:binding queue="my.amqp.queue2" /> </rabbit:bindings> </rabbit:fanout-exchange> <rabbit:queue name="my.amqp.queue2" />
![Page 46: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/46.jpg)
Conclusion: AMQP • Ubiquitous Messaging • AMQP: Protocol standard • Better scalability • Dynamic resources
![Page 47: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/47.jpg)
Conclusion: Spring AMQP • Easy to use • Flexible (e.g. message encoding) • Allows scalable message handling • Full support for AMQP and RabbitMQ
![Page 48: The Future of Messaging: RabbitMQ and AMQP](https://reader031.vdocuments.net/reader031/viewer/2022020122/55510970b4c9057b478b543e/html5/thumbnails/48.jpg)
More • http://springsource.org/spring-amqp • Also a .NET version available • …and support Spring Integration • http://blog.springsource.com/2011/04/01/
routing-topologies-for-performance-and-scalability-with-rabbitm
• Transaction support
• …and there is very similar JMS support