Download - Dali: A Multimedia Software Library
![Page 1: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/1.jpg)
Dalí A Multimedia Software
Library
Wei Tsang OoiBrian Smith, Hsi Haye Chan, Matthew Chiu, Sugata Mukhopadhyay, Dan Rubinovitz, Jiesang Song, Steve Weiss.
![Page 2: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/2.jpg)
Motivations
![Page 3: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/3.jpg)
Current Software Solution
BlackBox
C Source
![Page 4: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/4.jpg)
MPEG to RealVideo
BlackBox
RealVideo
Encoder
RGB
MPEG Real
OpenFile(f)GetNextFrame()
![Page 5: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/5.jpg)
MPEG to RealVideo
RGB
Decoder
YUV to RGB
Encoder
RGB to YUV
RealMPEG
YUV
compressed data
OpenFile(f)GetNextFrame()
YUV
![Page 6: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/6.jpg)
MPEG to RealVideo
YUVDecoder
YUV to RGB
Encoder
RGB to YUV
RealMPEG
compressed data
![Page 7: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/7.jpg)
Decode I-Frame only
BlackBox
RGB
MPEG
?OpenFile(f)GetNextFrame()
![Page 8: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/8.jpg)
Decode I-Frame only
RGB
MPEG
?
Decoder
YUV to RGB
compressed data
YUV
OpenFile(f)GetNextFrame()
![Page 9: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/9.jpg)
Decode I-Frame only
RGB
MPEG
?Decoder
YUV to RGB
I Frame filter
compressed data
I-frames
![Page 10: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/10.jpg)
Others Motivations..
BlackBox
Read from network ? Tune I/O ? Run on handheld ?
![Page 11: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/11.jpg)
Current Software Solution
BlackBox
C Source
![Page 12: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/12.jpg)
Dalí is in between..
BlackBox
C Source
Dalí
![Page 13: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/13.jpg)
Example
![Page 14: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/14.jpg)
Example :
Process a subset of the gray scale version of each I-frame
![Page 15: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/15.jpg)
MPEG structures
GOP hdrsequence hdr picture picture
pic hdr pic data
![Page 16: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/16.jpg)
What we want to do
parse sequence header
![Page 17: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/17.jpg)
What we want to do
parse sequence headerwhile not eof do
- find & parse picture header
![Page 18: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/18.jpg)
What we want to do
parse sequence headerwhile not eof do
- find & parse picture header- if it is an I-frame then
- parse the picture into DCT domain
- IDCT into spatial domain
![Page 19: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/19.jpg)
What we want to do
parse sequence headerwhile not eof do
- find & parse picture header- if it is an I-frame then
- parse the picture into DCT domain- IDCT into spatial domain- take the gray scale version of the frame and process it
![Page 20: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/20.jpg)
Abstractions
![Page 21: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/21.jpg)
Abstractions
For input/output For representing elements in MPEG For storing images in DCT domain For storing images in spatial
domain
![Page 22: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/22.jpg)
BitStream & BitParser
BitStream
File
BitParser
BitParser
![Page 23: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/23.jpg)
Parse Encode
MPEG Headers
MpegGopHdrMpegSeqHdr MpegPicHdrPicture Data
Find Skip Dump
![Page 24: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/24.jpg)
Parse Encode
Find
MpegGopHdrMpegSeqHdr MpegPicHdrPicture Data
Find Skip Dump
![Page 25: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/25.jpg)
Parse Encode
Skip
MpegGopHdrMpegSeqHdr MpegPicHdrPicture Data
Find Skip Dump
![Page 26: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/26.jpg)
Parse Encode
Dump
MpegGopHdrMpegSeqHdr MpegPicHdrPicture Data
Find Skip Dump
MpegGopHdr
![Page 27: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/27.jpg)
Parse Encode
Parse
MpegGopHdrMpegSeqHdr MpegPicHdrPicture Data
Find Skip Dump
width: 320height: 240pic rate: 30 fps etc
![Page 28: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/28.jpg)
Parse Encode
Encode
MpegSeqHdr
Find Skip Dump
width: 320height: 240pic rate: 30 fps etc
![Page 29: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/29.jpg)
ScImage
ScBlock :- { short dc; char index[63]; int ac[63];}
represents 8x8 pixels
![Page 30: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/30.jpg)
ByteImage
unsigned char0..255
represents a pixel
![Page 31: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/31.jpg)
The Code
![Page 32: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/32.jpg)
Example
MpegSeqHdr *seqHdr = MpegSeqHdrNew ();MpegPicHdr *picHdr = MpegPicHdrNew ();BitStream *bs = BitStreamNew (65536);
BitStream
![Page 33: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/33.jpg)
Example
BitParser *bp = BitParserNew ();BitParserAttach (bp, bs);
BitStream
BitParser
![Page 34: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/34.jpg)
Example
BitStreamReadFromFile (bs, f, 0);
BitStream
BitParser File
MpegSeqHdr
MpegGopHdr
MpegPicHdr
![Page 35: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/35.jpg)
Example
MpegSeqHdrFind (bp);
BitParser
MpegSeqHdr
MpegGopHdr
MpegPicHdr
![Page 36: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/36.jpg)
Example
MpegSeqHdrParse (bp, seqHdr);w = (seqHdr->width + 7)/8;h = (seqHdr->height + 7)/8;
BitParser
MpegSeqHdr
MpegGopHdr
MpegPicHdr
![Page 37: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/37.jpg)
ExampleScImage *y = ScNew (w, h);ScImage *u = ScNew (w/2, h/2);ScImage *v = ScNew (w/2, h/2);
y u v
![Page 38: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/38.jpg)
Examplestatus = MpegPicHdrFind (bp);while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr);
status = MpegPicHdrFind (bp);}
![Page 39: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/39.jpg)
Examplestatus = MpegPicHdrFind (bp);while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) {
}
status = MpegPicHdrFind (bp);}
![Page 40: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/40.jpg)
Examplestatus = MpegPicHdrFind (bp);while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { MpegPicIParse ( bp,seqHdr,picHdr, scy,scu,scv); : }
status = MpegPicHdrFind (bp);}
![Page 41: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/41.jpg)
Examplestatus = MpegPicHdrFind (bp);while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { MpegPicIParse ( bp,seqHdr,picHdr, scy,scu,scv); : } BitStreamFill(bp, f);
status = MpegPicHdrFind (bp);}
![Page 42: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/42.jpg)
Refill the BitStream
BitParser
![Page 43: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/43.jpg)
Refill the BitStream
BitParser
![Page 44: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/44.jpg)
Refill the BitStream
BitParser File
![Page 45: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/45.jpg)
Memory Mapped BitStream
bs = BitStreamMMapReadNew(“tennis.mpg”);
File
![Page 46: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/46.jpg)
Operate on a subset
![Page 47: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/47.jpg)
width: 8height: 6x: 0y: 0parentWidth: 8isVirtual: 0firstByte*
Physical ByteImage
![Page 48: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/48.jpg)
width: 5height: 4x: 1y: 1parentWidth: 8isVirtual: 1firstByte*
Virtual ByteImage
![Page 49: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/49.jpg)
Example
ScImage *virt = ScClip (y,6,2,20,20);
6 20 blocks
![Page 50: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/50.jpg)
Example
ScImage *virt = ScClip (y,6,2,20,20);ByteImage *byte = ByteNew (160, 160);ScToByte (virt, byte);
6 20 blocks
![Page 51: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/51.jpg)
Implementation
![Page 52: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/52.jpg)
Implementation
C library Tcl bindings available
![Page 53: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/53.jpg)
Organization
Code are organized into packages Basic
– ByteImage, ScImage, etc. MPEG
– MPEG-1 Video/Audio/System Vision
– Useful vision routines
![Page 54: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/54.jpg)
Other Packages
AVI WAVE JPEG GIF
![Page 55: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/55.jpg)
Availability
http://www.cs.cornell.edu/dali open source binaries for linux/win32/sunos examples documentations
![Page 56: Dali: A Multimedia Software Library](https://reader034.vdocuments.net/reader034/viewer/2022051514/5489b18db479590f0d8b595d/html5/thumbnails/56.jpg)
The End