what's new in v11 - fluentd casual talks #3 #fluentdcasual
TRANSCRIPT
![Page 1: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/1.jpg)
Sadayuki Furuhashi!uentd.org
What’s new in v11
![Page 2: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/2.jpg)
Non-stop restart
supervisor process
worker process
v10 (old):
listen TCP socket heartbeat
![Page 3: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/3.jpg)
Non-stop restart
supervisor process
worker process
v10 (old):
Downtime!
restart
initializing...
heartbeat
![Page 4: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/4.jpg)
Non-stop restart
supervisor process
worker process
v11:
listen TCP socket heartbeat
![Page 5: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/5.jpg)
Non-stop restart
supervisor process
worker process
v11:
listen TCP socket
restart
Zero downtime!
initializing...
heartbeat
![Page 6: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/6.jpg)
Multiprocess
supervisor process
worker processheartbeat
<worker> ...</worker>
<worker> ...</worker>
<worker> ...</worker>
![Page 7: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/7.jpg)
Multiprocess
worker process
・ in_tail・ out_groupcounter・ out_webhdfs
<worker> ... </worker>
<worker>
</worker>
・ in_forward・ out_VERY_UNSTABLE
<worker>
</worker>
Separate importantlogging fromexperimental plugins
![Page 8: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/8.jpg)
Error stream
{“event”: 1, ...}
{“event”: 2, ...}
{“event”: 3, ...}
{“event”: 4, ...}
{“event”: 5, ...}
{“event”: 6, ...}
v10 (old):
chunk
chunk
<match **>
OK
ERROR!
?
ERROR!
?
?
event
<source>
Error handlingdepends on plugin(throw away! etc.)
![Page 9: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/9.jpg)
Error stream
{“event”: 1, ...}
{“event”: 2, ...}
{“event”: 3, ...}
{“event”: 4, ...}
{“event”: 5, ...}
{“event”: 6, ...}
v10 (old):
chunk
chunk
<match **>
OK
ERROR!
OK
ERROR!
OK
OK
event
<source>
Fluentd corehandles errorevents
<label @ERROR> <match **> ...
特殊ラベル
OK
OK
![Page 10: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/10.jpg)
Plugin version managementv11:
$ cat /etc/td-agent/Gemfile
gem “fluent-plugin-s3”, “~> 0.11.3”
gem “fluent-plugin-webhdfs”, “~> 0.2.2”
gem “fluentd”, “~> 0.11.2”
$
![Page 11: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/11.jpg)
Log level per pluginv11:
<source>
type forward
log_level error
</source>
<match **>
type mongo
log_level debug
</match>
![Page 12: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/12.jpg)
Variables in a config filev11:
<source>
type forward
port “#{ENV[‘FLUENTD_LISTEN_PORT’]}”
</source>
![Page 13: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/13.jpg)
<server> @set port 9800 @include “/etc/fluentd/server.conf”</server>
<server> @set port 9801 @include “/etc/fluentd/server.conf”</server>
Variables in a config filev11?:
![Page 14: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/14.jpg)
<match raw.**> type flowcounter add_tag_prefix counted</match>
<match counted.**> type growthforecast</match>
Streaming processing without tag hackingv10 (old):
<match **> type flowcounter
<match **> type growthforecast </match></match>
v11:
![Page 15: What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual](https://reader034.vdocuments.net/reader034/viewer/2022052321/554be99cb4c9055a368b4ca0/html5/thumbnails/15.jpg)
v11
Reliability &Performance
✓ Non-stop restart✓ Multiprocess✓ Error stream
✓ Variables in a config file✓ Streaming processing without tag hacking
✓ Plugin version management✓ Log level per plugins
Management
Configuration