what is kafka & why is it important? (ukoug tech17, birmingham, uk - december 2017)
TRANSCRIPT
![Page 1: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/1.jpg)
What is Apache Kafka & Why is it Important?
The Event Fabric bringing IT together
What is Apache Kafka & Why is it Important? | UKOUG Tech17 1
µ
µ
What is
Apache Kafka & Why is it Important?
![Page 2: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/2.jpg)
It would be so nice if I could
publish my ideas and actions,
accessible near instantly for
everyone who is interested
Heck, I do not even know these people
and they may not know me [personally]
– just my pearls of wisdom. And if they
are late to the party, they can also
check out the historic archives of my
eloquence
Without fretting about the numbers of
readers involved and whether they are
in the same time zone as me and online
when I publish my messages – and
which device they use
![Page 3: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/3.jpg)
It would be so nice if I could
publish my ideas and actions,
accessible near instantly for
everyone who is interested
Heck, I do not even know these people
and they may not know me [personally]
– just my pearls of wisdom. And if they
are late to the party, they can also
check out the historic archives of my
eloquence
Without fretting about the numbers of
readers involved and whether they are
in the same timezone as me and online
when I publish my messages – and
which device they use
![Page 4: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/4.jpg)
• Decoupled communication
• 0, 1 or many followers
• Scalable number of messages (and parties)
• Reliable (mostly available, few messages lost)
• Full history
• Open: cross device, cross location
• Not Sub-second, near real-time fast
• Rate limited (#messages/minute)
• Size limited (140-280 characters)
• Format limited (text)
• Not for private interactions
• Not (really) for programmatic use
![Page 5: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/5.jpg)
5
Oracle
Database
ORDERS
Oracle Database
DVX_ORDERS
![Page 6: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/6.jpg)
6
Oracle
Database
ORDERS
Oracle Database
DVX_ORDERS
![Page 7: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/7.jpg)
What is Apache Kafka and why is it important? 7
Oracle
Database
ORDERS
Oracle Database
DVX_ORDERS
µOracle Application
Container Cloud
Oracle DBaaS Cloud
µLocally running
Node application
![Page 8: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/8.jpg)
What does the Twitter for System Driven Event Interaction look like?
What is Apache Kafka and why is it important? 8
• Decoupled communication – organized per topic
• 0, 1 or many Consumers per Topic
• Scalable number of messages (and parties)
• Reliable (distributed)
• Full history
• Open: libraries in many technologie & REST APIs
![Page 9: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/9.jpg)
9
Oracle
Database
ORDERS
Oracle Database
DVX_ORDERS
µOracle Application
Container Cloud
Oracle DBaaS Cloud
µLocally running
Node application
Oracle Event Hub
![Page 10: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/10.jpg)
What does the Twitter for System Driven Event Interaction look like?
What is Apache Kafka and why is it important? 10
• Decoupled communication – organized per topic
• 0, 1 or many Consumers per Topic
• Scalable number of messages (and parties)
• Reliable (distributed)
• Full history
• Open: libraries in many technologie & REST APIs
• Near real-time fast
• No Rate Limit
• No enforced size limit
• Anything goes (it’s all byte[])
• On premises or in cloud, private or trusted
• Very much for programmatic use
![Page 11: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/11.jpg)
Events
Producers
Consumers
Robust, Scalable, Fast,
History Retention
Containerized/Cloud-
enabled
Open
![Page 12: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/12.jpg)
Messaging as we know it
• JMS, Oracle Advanced Queuing, IBM MQ, MS MQ, RabbitMQ, MQTT,
XMPP, WebSockets, Oracle Coherence, …
• Challenges
• Costs
• Scalability (size and speed)
• (lack of) Distribution (and therefore availability)
• Complexity of infrastructure
• Message delivery guarantees
• Lack of technology openness
• Deal with temporarily offline consumers
• Retain history
![Page 13: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/13.jpg)
Introducing Apache Kafka
• ..- 2010 – creation at Linkedin
• Message Bus | Event Broker
• High volume, low latency, highly reliable, cross technology
• Scalable, distributed, strict message ordering, ….
• 2011/2012 – open source under the Apache Incubator/ Top Project
• Kafka is used by many large corporations:
• Walmart, Cisco, Netflix, PayPal, LinkedIn, eBay, Spotify, Uber, Sift
Science, Zalando, The New York Times, Airbnb, Coursera, ING Bank,…
• And embraced by many software vendors & cloud providers
• Client libraries available for Node, Java, C/C++, Python, Ruby, PHP, Go,
Rust, .NET, Perl, Scala DSL, Clojure, Swift and more
![Page 14: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/14.jpg)
Producers
Consumers
tcp
tcp
![Page 15: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/15.jpg)
Producers
Consumers
Topic
![Page 16: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/16.jpg)
KAFKA TERMINOLOGY
• Topic
• Message• == ByteArray
• Broker
• Producer
• Consumer
Producer Consumer
TopicBroker
Key
Value
Time
Message
![Page 17: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/17.jpg)
Producers
Consumers
TopicBroker
Key
Value
Time
![Page 18: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/18.jpg)
CONSUMING
• Messages are available to consumers only when they have been committed
• Kafka does not push• Unlike JMS
• Read does not destroy• Unlike JMS Topic
• (some) History available• Offline consumers can catch up
• Consumers can re-consume from the past
• Delivery Guarantees• Ordering maintained
• At-least-once (per consumer) by default; at-most-once and exactly-once can beimplemented
![Page 19: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/19.jpg)
Producers
Consumers
TopicBroker
Key
Value
Time
![Page 20: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/20.jpg)
Producers
Consumers
TopicBroker
tcp
tcp
![Page 21: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/21.jpg)
WHAT’S SO SPECIAL?
• Durable
• Scalable• High volume
• High speed
• Available
• Distributed
• Open
• Quick start
• Free (no license costs)
• “Self Fulfilling Prophecy” (positive feedback loop)
![Page 22: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/22.jpg)
CONFLUENT == ENTERPRISE KAFKA
• Freemium model• Support
• Training
• Confluent Cloud
• Platform and Tools
![Page 23: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/23.jpg)
![Page 24: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/24.jpg)
<CTRL F5>
![Page 25: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/25.jpg)
Application
Server
F5
F5
F5
CTRLF5
CTRLF5
CTRLF5
![Page 26: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/26.jpg)
Application
Server
F5
F5
F5
CTRLF5
CTRLF5
CTRLF5
![Page 27: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/27.jpg)
Application
Server
F5
F5
F5
CTRLF5
CTRLF5
CTRLF5
![Page 28: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/28.jpg)
FAST DATA AND ACTIVE UI
• Handle influx
• Publish findings instantaneously
• Update UI & notify end user immediately
• Analyze in real time
• Decoupled components
• No data loss when a component is temporarily down
• Scalable with volume of events and of number of clients
![Page 29: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/29.jpg)
THE CASE AT HAND
Client
Client
Client
Client
Show live
tweet feed
for
conferencesShow live
tweet
aggregates
per
conference
Allow users
to like tweets
–and show
live list of
liked tweets
Show a live
list of top 3
liked tweets
per
conference
Tweets on #ukoug17
#ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 30: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/30.jpg)
DEMO - REAL TIME, CROSS CLOUD, CROSS TECHNOLOGY PUSH
Tweets on #ukoug17
#ukoug_tech17
#ukoug_apps17
#ukoug_jde17
Client
Client
Client
Client
you
![Page 31: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/31.jpg)
THE CASE AT HAND – STEP ONE
Client
Client
Client
Client
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
Show live
tweet feed
for
conferences
Tweets
Topic
![Page 32: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/32.jpg)
THE CASE AT HAND – STEP ONE AND TWO
Client
Client
Client
Client
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
Show live
tweet feed
for
conferences
Tweets
Topic
![Page 33: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/33.jpg)
KAFKA CONSUMER IN NODEGET EVENTS PUSHED INTO APPLICATION
![Page 34: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/34.jpg)
THE CASE AT HANDSERVER SENT EVENTS FOR PUSH BACK
Client
Client
Client
Client
Show live tweet
feed for
conferences
Tweets
Topic
Server Sent
Event
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 35: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/35.jpg)
SERVER SENT EVENT – SERVER SIDE
Client
Client
Client
Client
Server Sent
Event
![Page 36: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/36.jpg)
Client
Client
Client
Client
Server Sent
Event
SERVER SENT EVENT – CLIENT SIDE
![Page 37: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/37.jpg)
LIVE TWEET STREAM
Server Sent
Event
![Page 38: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/38.jpg)
THE CASE AT HANDTWEET LIKES – CLIENT TO SERVER TO ALL CLIENTS
Client
Client
Client
Client
Show live
tweet feed
for
conferences
Tweets
Topic
SS
E
Allow users
to like tweets
–and show
live list of
liked tweets
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 39: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/39.jpg)
THE CASE AT HANDWEB SOCKETS – FOR BI DIRECTIONAL PUSH
Client
Client
Client
Client
Show live
tweet feed
for
conferences
Tweets
Topic
SSE
WebSockets
Allow users
to like tweets
–and show
live list of
liked tweets
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 40: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/40.jpg)
TWEET LIKES BROADCASTING
WebSockets
WebSockets
![Page 41: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/41.jpg)
THE CASE AT HANDSTREAMING ANALYSIS OF TWEET EVENTS
Client
Client
Client
Client
Show live
tweet feed
for
conferences
Tweets
Topic
SSE
WebSockets
Allow users
to like tweets
–and show
live list of
liked tweets
Show live
tweet
aggregates
per
conference
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 42: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/42.jpg)
THE CASE AT HAND - STREAMING ANALYSIS OF TWEETS
Client
Client
Client
Client
Show live
tweet feed
for
conferences
Tweets
Topic
WebSockets
Allow users
to like tweets
–and show
live list of
liked tweets
Show live
tweet
aggregates
per
conference
tweetAnalytics
Topic
Streaming
Tweets
Aggregation
µ
SSE
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 43: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/43.jpg)
KAFKA STREAMS
• Real Time Event [Stream] Processing integrated into Kafka• Aggregations & Top-N• Time Windows• Continuous Queries• Latest State (event sourcing)
• Turn Stream (of changes) into Table(of most recent or current state)
• Part of the state can be quite old
• A Kafka Streams client will have statein memory
• Always to be recreated from topic partitionlog files
• Note: Kafka Streams is relatively new• Only support for Java clients
![Page 44: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/44.jpg)
KAFKA STREAMS
Topic
Filter
Aggregate
Join
Topic
Map
(Xform)
Publish
Topic
![Page 45: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/45.jpg)
EXAMPLE OF KAFKA STREAMS
TopicgroupBy
Aggregate
Join
Topic
Map
(Xform)
Publish
TweetMessageConference
Text
Author
Hashtag
Set Conference as key
Sum/Avg/Top3 by key
(==conference)
As JSON
Round aggregate
to nearest 100
Latest Conference
Details
Topic: CountTweetsPerConference
and possibly per time
window
![Page 46: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/46.jpg)
KAFKA STREAMS –RUNNING COUNT TWEETS PER CONFERENCE
![Page 47: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/47.jpg)
STREAMING TWEET ANALYTICS PUSHED TO CLIENTS
Server Sent
Event
![Page 48: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/48.jpg)
THE CASE AT HAND - STREAMING ANALYSIS OF TWEET LIKES
Client
Client
Client
Client
Show live
tweet feed
for
conferences
Tweets
Topic
WebSockets
Allow users
to like tweets
–and show
live list of
liked tweets
Show live
tweet
aggregates
per
conference
tweetAnalytics
Topic
Streaming
Tweets
Aggregation
µ
SSE
Show a live
list of top 3
liked tweets
per
conference
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 49: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/49.jpg)
KSQL FOR DECLARATIVE STREAM ANALYTICS THROUGH CONTINUOUS QUERIES
create table tweetAnalytics as
select conference
, count(*)
from tweetsTopic
group by conference
create stream retweets
as
select *
from tweetsTopic
where text like 'RT%'
![Page 50: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/50.jpg)
VISUALIZING KSQL VS KAFKA STREAMS
![Page 51: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/51.jpg)
THE CASE AT HAND - STREAMING ANALYSIS OF TWEET LIKES
Client
Client
Client
Client
Show live
tweet feed
for
conferences
Tweets
Topic
WebSockets
Allow users
to like tweets
–and show
live list of
liked tweets
Show live
tweet
aggregates
per
conference
tweetAnalytics
Topic
Streaming
Tweets
Aggregation
µ
SSE
Show a live
list of top 3
liked tweets
per
conference
Likes
Aggregation
µ
tweetLike
Topic
Top3TweetLikes
PerConference
Tweets on #ukoug17 #ukoug_tech17
#ukoug_apps17
#ukoug_jde17
![Page 52: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/52.jpg)
WEBSOCKETS – SERVER SIDE
![Page 53: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/53.jpg)
RUNNING TOP 3 OF BEST LIKED TWEETS PER CONFERENCE
Server Sent
Event
![Page 54: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/54.jpg)
END TO END FLOW CLOUD ENABLED
API
Cache
EventHub CS
µ
Tweets
Aggregation
µ
LikesTweets
UI µClient
Chrome
Client
Firefox
Likes
Aggregation
µ
API
µ
Tweet
Count
Likes
Top3
![Page 55: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/55.jpg)
Key aspects of this demo – What Kafka can do for you
• Bridging Cloud(s) and on premises systems
• Providing decoupled interaction between microservices
• Performing Streaming Analysis
• Bridging technologies (Java, Node, …)
• Bridging the availability (no | one | multiple instances)
• Provide semi-push based synchronization
• Open
• Scalable
• Reliable & Available
• Fast
• Complete historical record
What is Apache Kafka and why is it important? 58
![Page 56: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/56.jpg)
Oracle embracing Apache Kafka
• Event Hub Cloud Service = Managed Apache Kafka platform
• Managed Topics have been announced too
• Kafka as source for Golden Gate and ODI
• Data Pipeline with Data Hub (Apache Cassandra) & Event Hub
• Oracle Service Bus Kafka Adapter
• Integration Cloud
• Stream Analytics (aka Stream Explorer fka Oracle Event Processor)
• Oracle Native Container and Microservices Platform
• Fn Serverless Platform
• JET and ADF real time push based on Apache Kafka
• In general – the bridge between on premises [public] Cloud
What is Apache Kafka and why is it important? 59
![Page 57: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/57.jpg)
Summary
• => == =>
• Apache Kafka is emerging as platform of choice for message exchange in a world of
• Microservices
• CQRS and Data Source Synchronization
• Clouds
• Fast Data (IoT) and Streaming Analysis
• Real time data integration & distribution
• Oracle is rapidly embracing Apache Kafka on various levels
• Getting started with Apache Kafka is not very hard at all
• The platform is open source – and has broad client support (Java, Node, …)
• Many resources are available – tutorials, blog article, demonstrations, presentation
slides and recordings of conference sessions, samples on GitHub
What is Apache Kafka and why is it important? 60
![Page 58: What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December 2017)](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a648a317f8b9a8e568b4a4f/html5/thumbnails/58.jpg)
Thank you!
What is Apache Kafka and why is it important? 61
• Blog: technology.amis.nl
• Email: [email protected]
• : @lucasjellema
• : lucas-jellema
• : www.amis.nl, [email protected]