video overdds
DESCRIPTION
TRANSCRIPT
Copyright © MilSOFT,Turkey UNCLASSIFIED 1
Video Over DDSDDS Information Day
Brussels, Belgium-June-26,2007
Copyright © MilSOFT,Turkey UNCLASSIFIED 2
• Introduction• Image Exploitation System for UAVs• Why Use DDS for Video Streaming?• MilSOFT DDS • DDS Topics• QoS choices• Future Work• Conclusion
Outline
Copyright © MilSOFT,Turkey UNCLASSIFIED 3
Introduction
• DDS middleware• Location transparency with pub/sub mechanism
• Redundancy and fault tolerance mechanisms
• Multiple readers/writers
• Transient and persistent services
• Asynchronous, time-independent data distribution
• Rich set of QoS, suitable for real-time systems
• Wide range of Application Areas• C4I
• Industrial automation
• Distributed control and simulation
• Telecom equipment control
• Sensor networks
• Use Case: Image Exploitation System for UAVs
Copyright © MilSOFT,Turkey UNCLASSIFIED 4
Data Receiver
Pre-processor
DB Server
Screening Exploitation 1 Exploitation 2
Sensor Data(EO/IR, SAR, GMTI)
Telemetry
High CapacityStorage Unit
Raw Data Recording
DDS based Data
Streaming
DDS based Data
Streaming
Pre-processed & Georecorded Data
Artifacts
All Data in DB
Raw Data
DDS on UAV Image Exploitation System
Copyright © MilSOFT,Turkey UNCLASSIFIED 5
DDS on UAV Image Exploitation System
• Data • Events
• User information
• Resource Management
• Radar data streaming• SAR streaming
• GMTI streaming
• Telemetry streaming
• Video streaming• Live data streaming
• Recorded data streaming
• Data Publisher: DataReceiver
• Data Subscribers: Consoles and Pre-Processor
• Content of data MPEG-2 TS
Copyright © MilSOFT,Turkey UNCLASSIFIED 6
MPEG-2 TS (Transport Stream)
• Elementary Stream• Video, audio, telemetry data, etc...
• From each elementary stream Packetized Elementary Stream (PES) is formed
• And each PES is broken into fixed-size Transport Packets
• TS packets are 188 bytes of which 4 bytes are for the header and 184 bytes are for the payload
• Transport Stream can multiplex multiple elementary streams• The TS packet header contains a Packet Identifier (PID) to identify
different elementary streams
• 8 Mbit data rate for UAV Image Exploitation System
Copyright © MilSOFT,Turkey UNCLASSIFIED 7
Why use DDS for video streaming?
• Publish-Subscribe simplifies the development process• The clients do not need to know where the server is, and the server
does not need to know the clients
• DDS specification implies mutlicast communication and the interoperability wire protocol RTPS supports and recommends multicasting• Dramatically decreased bandwith usage with DDS implementations
which support multicast
• Quality Of Service attributes like reliability, durability, transport priority directly can be used for video streaming channels
• Generally DDS implementations have very low latencies in data transmission compared to other middlewares, which makes it a good choice
Copyright © MilSOFT,Turkey UNCLASSIFIED 8
Why MilSOFT DDS
• MilSOFT DDS is compliant with current DDS specification, version 1.2
• Provides Minimum Profile QoS attributes, which are neccessary for video streaming
• Completely decentralized design
• No dynamic memory allocation after initialization
• Zero-copy access to data
• Implements multicast communications
Copyright © MilSOFT,Turkey UNCLASSIFIED 9
More information on http://dds.milsoft.com.tr
MilSOFT DDS Middleware
Copyright © MilSOFT,Turkey UNCLASSIFIED 10
Video Over DDS Architecture
Copyright © MilSOFT,Turkey UNCLASSIFIED 11
DDS Types and Topics Used For Video Streaming
• So how do we encapsulate MPEG2-TS packets into DDS types?
• There are two DDS types and two corresponding Topics.• VideoHeader VideoHeaderTopic
• VideoPacket VideoPacketType
• VideoHeader• Contains the header of the video, which is needed to initialize the
decoder. (FPS, height, width, etc... )
• VideoPacket• Contains MPEG2-TS packets
Copyright © MilSOFT,Turkey UNCLASSIFIED 12
VideoHeader Topic
struct VideoHeader
{long headerType; // key
sequence<char, HEADER_MAX_DATA> headerData;
};
• Published once for each Transport Stream
• Contains the necessary information • initialize the video decoder
• information about the other multiplexed elementary streams
• Key to distinguish the video header from other header data
• Late joiners also get this topic
Copyright © MilSOFT,Turkey UNCLASSIFIED 13
QOS for VideoHeader Topic
• VideoHeaderTopic is necessary to initialize decoding of the video
• Header information is critical for clients• Use Reliability QoS (RELIABLE)
• Both publishers and subscribers are Reliable
• Late joining viewers need this information• Use Durability QoS (TRANSIENT_LOCAL)
• There are different instances of the header according to it’s type. We need only the last update of each instance!• History QoS is default, kind = KEEP_LAST, depth = 1
Copyright © MilSOFT,Turkey UNCLASSIFIED 14
VideoPacket Topic
struct VideoPacketType
{
long frameNumber;
sequence<char, PACKET_MAX_DATA_SIZE> packetData;
};
• MPEG2-TS packets are too small in size (188 bytes).
• VideoPacket type sends multiple TS packets of a video frame at each VideoPacket sample• Decreases the overhead and increases the bandwidth utilization
• A frame can span multiple VideoPacket samples (fragmented)
• Each sample contains TS packets of only one frame
Copyright © MilSOFT,Turkey UNCLASSIFIED 15
QOS for VideoPacket Topic
• For VideoPacketTopic we may need both Reliable and Best Effort delivery, according to client needs• Use Offered >= Requested semantics
• Server publishes with a Reliable writer ( Offer Reliable communication )
• Clients subscribe with either Reliable or Best Effort readers
• Data is not needed by late joiners• Durability is Volatile
• VideoPacket Type has no key, but we can use History QoS for buffering• History QoS, kind = KEEP_LAST, depth value can be used as a buffer
(set it to more than 1)
Copyright © MilSOFT,Turkey UNCLASSIFIED 16
Other Concerns And Solutions
• We don’t want out of sequence frames• DDS already satisfies this, no out of order packets!
• We don’t want frames which are too late• Use Lifespan QoS
• Decoding takes time• Either read data from a DataReaderListener and immediately put into
the queue of another thread
• Use WaitSets
Copyright © MilSOFT,Turkey UNCLASSIFIED 17
Future Work
• Implement advanced services like rewind, replay, fast-forward etc...
• Network reservations• Use RSVP (IntServ) to reserve bandwith
• Analyze different methods to give DDS video streams a priority• Use Transport_Priority QoS (not completely standard)
• More standardization is needed in this area
• Standardize Flow Control mechanisms in DDS for better utilization of network bandwith
Copyright © MilSOFT,Turkey UNCLASSIFIED 18
Conclusion
• Video over DDS is possible!
• DDS provides a good set of QoS attributes for video streaming
• Performance requirements• Image Exploitation system for UAVs uses 8 Mbit MPEG-2 TS• 20 Mbit video streaming is possible with DDS
Copyright © MilSOFT,Turkey UNCLASSIFIED 19
Thank You!