realtime video streaming the open source way
TRANSCRIPT
![Page 1: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/1.jpg)
MADRID · NOV 27-28 · 2015
Realtime video streaming
MADRID · NOV 27-28 · 2015the opensource way
Iván Belmonte
![Page 2: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/2.jpg)
MADRID · NOV 27-28 · 2015
Github: ibelmonte
SysdivisionTwitter: @sysdivisionWebsite: http://sysdivision.com
Twitter: @ibelmonte
Iván Belmonte
![Page 3: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/3.jpg)
MADRID · NOV 27-28 · 2015
Real time weather reporting system for outdoor sports.2 main elements:- HD cameras in surfing spots (beaches)- Weather stations in surfing spots
GOAL:
Weather forecast typically fails. Spotfav gives you a window to your spot, to check the weather conditions before moving.
![Page 4: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/4.jpg)
MADRID · NOV 27-28 · 2015
What forecast says the day before
![Page 5: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/5.jpg)
MADRID · NOV 27-28 · 2015
What you expect when you take your car to go to the spot
![Page 6: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/6.jpg)
MADRID · NOV 27-28 · 2015
What’s actually going on
![Page 7: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/7.jpg)
MADRID · NOV 27-28 · 2015
IMPORTANT NOTE:
I’m a backend developer/sysadmin
I know nothing about video and audio processing
In this project I am A VICTIM
![Page 8: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/8.jpg)
MADRID · NOV 27-28 · 2015
Video manipulation and transmission is complex
- Plenty of difficult concepts: encoding, muxing, transcoding, encapsulation, bitrates, fps…
- Different media servers, different players, some codecs may not be supported
- Several transmission protocols depending on your needs: RTP, RTSP, RTMP, HLS, HDS, Shoutcast, HTML5…
Added complexity: LIVE streaming
![Page 9: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/9.jpg)
MADRID · NOV 27-28 · 2015
Looking for a standard setup
Most common case: H264 over RTSP
Affiliates: get this if you want to get connected to our network
![Page 10: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/10.jpg)
MADRID · NOV 27-28 · 2015
There are several options
Commercial: Adobe media server, Wowza, Flussonic, Flumotion…
SaaS:Ustream, Justin.tv (defunct)…
Open source: ffmpeg, ffserver, libav, Live555, Red5, crtmpd, vlc, Erlyvideo…
![Page 11: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/11.jpg)
MADRID · NOV 27-28 · 2015
OUR SCENARIO
1 Camera, H264 over RTSP
1. The goal
Embed the camera in a website and also in a smartphone app
2. Nice to have
Not permanently connected (connect on demand)Tolerant to network cuts
3. Science fiction
Be able to filter by referral (avoid other websites to embed our cameras)
![Page 12: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/12.jpg)
MADRID · NOV 27-28 · 2015
STEP 1: WEB player (RTMP)
![Page 13: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/13.jpg)
MADRID · NOV 27-28 · 2015
ROUND 1
A javascript serving a Directx (wtf?) script.
![Page 14: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/14.jpg)
MADRID · NOV 27-28 · 2015
ROUND 1
Cons:
- The camera has a limited pool of connections available. Visitors exceeding this poll get rejected.
Works?
- In some browsers
![Page 15: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/15.jpg)
MADRID · NOV 27-28 · 2015
ROUND 2
A VLC cli process reading the RTSP, transcoding and publishing an RTMP stream in a socket
![Page 16: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/16.jpg)
MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES…
![Page 17: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/17.jpg)
MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES… for 1 camera.
![Page 18: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/18.jpg)
MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES… for 1 camera.
What if more cameras?
- 1 VLC process for each camera.
![Page 19: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/19.jpg)
MADRID · NOV 27-28 · 2015
ROUND 2
Round 2 version with multiple cameras
![Page 20: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/20.jpg)
MADRID · NOV 27-28 · 2015
ROUND 2
Cons:
- A lot of processes in the server eating memory as hungry animals.
- VLC is unable to recover from network cuts.
- Each VLC process opens a socket in a different port, which in my case is a mess as I’m hosting the VLC processes inside a container and then routing the outside in.
- VLC is permanently connected to the camera
![Page 21: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/21.jpg)
MADRID · NOV 27-28 · 2015
AND THEN ONE DAY…
Adobe updates flash player, breaks my streamings
![Page 22: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/22.jpg)
MADRID · NOV 27-28 · 2015
BROKEN STREAMING
Two possible reasons:
1. Something is wrong with my VLC commandvlc-wrapper-vvvh,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview"-sout'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'
2. Something is wrong with the new version of Flash
![Page 23: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/23.jpg)
MADRID · NOV 27-28 · 2015
BROKEN STREAMING
Two possible reasons:
1. Something is wrong with my VLC commandvlc-wrapper-vvvh,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview"-sout'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'
2. Something is wrong with the new version of Flash
Of course option 2. I contacted Adobe: - ME: your upgrade has a bug and it broke my streamings- ADOBE: …(silence)
![Page 24: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/24.jpg)
MADRID · NOV 27-28 · 2015
ROUND 3
A super cool media server capable to encapsulate H264 RSTP into RTMP for flash players: Erlyvideo.
![Page 25: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/25.jpg)
MADRID · NOV 27-28 · 2015
ROUND 3
Pros:
- Written in Erlang: amazingly light and fast- Connects to the source on demand
- One process, several cameras- Tolerant to network cuts (due to the previous point)
- Referrer filtering
Cons:
- Documented in russian (partially translated into english). Support forums also 80% russian.
- HLS support (for smartphone apps) only supported in its new commercial version: Flussonic.
![Page 26: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/26.jpg)
MADRID · NOV 27-28 · 2015
STEP 2: Smartphone app (HLS)
![Page 27: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/27.jpg)
MADRID · NOV 27-28 · 2015
Nginx RTMP module
An Nginx module capable to read streams in several formats as input, store them and then serve them encapsulated in other protocols (eg: HLS)
![Page 28: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/28.jpg)
MADRID · NOV 27-28 · 2015
Nginx RTMP module
Works?
- YES, perfectly
Cons:
- Not very well documented
- Not tolerant to network cuts
- Unable to regain lost connectivity
![Page 29: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/29.jpg)
MADRID · NOV 27-28 · 2015
Overall vision
![Page 30: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/30.jpg)
MADRID · NOV 27-28 · 2015
Service schema
A video source. A separate server for each streaming format. A website visitor and an smartphone user.
![Page 31: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/31.jpg)
MADRID · NOV 27-28 · 2015
An automated process
Using linux containers and some glue code everything can play together in an automated way
![Page 32: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/32.jpg)
MADRID · NOV 27-28 · 2015
CONSLUSIONS
![Page 33: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/33.jpg)
MADRID · NOV 27-28 · 2015
What I would do now if starting from scratch
1. Do it the opensource way: there are powerful tools
2. Look for a single solution for both web and mobile (probably libav, ffmpeg or Red5)
3. Spend time looking for a solution capable to serve also HTML5 formats (Mp4, Webm, Ogg)
![Page 34: Realtime video streaming the open source way](https://reader034.vdocuments.net/reader034/viewer/2022052606/58a4f3f51a28abd8548b6c81/html5/thumbnails/34.jpg)
MADRID · NOV 27-28 · 2015
THANK YOU!