mackenzie, adrian 2006 - codecs

8/11/2019 Mackenzie, Adrian 2006 - Codecs

1/6

Codecs: Encoding/decoding images and sounds

Adrian Mackenzie

[email protected]

October 2006

Codecs (coder-decoder) perform encodin and decodin on !ideo" speec#" music and te$t.%#e& scale" reorder" decompose and reconstitute perceptible imaes and sounds so t#att#e& can et t#rou# information net'orks and electronic media. Codecs are intimatel&associated 'it# c#anes in t#e spectral densit&" t#e distribution of ener&" of sound and

imae in electronic media.

Codecs pose numerous anal&tical problems for soft'are sc#olars. %#e& are mat#ematicall&deep. Comin to rips 'it# t#em ma& entail lent#& immersion in tec#nical details.Alt#ou# t#e& are responsible for displa&in man& mo!in imaes on screens toda&"codecs t#emsel!es often #ide in #ard'are and lo'er-le!el code. %#e& come to li#toccasionall&" usuall& in t#e form of an error messae sa&in t#at somet#in is missin* t#eri#t codec #as not been installed. +espite or per#aps because of t#eir con!olutedobscurit&" t#e& catal&ze ne' relations bet'een people" t#ins" spaces and times in e!entsand forms.

Patent pools and codec floods

,ideo codecs suc# as M/-" M/-1" .26. .263" t#e important .261" t#eora" dirac"+i!4" 4!i+" M5/" M," 7eal,ideo" etc are stre'n across net'orked electronic media.7ou#l& a #undred different audio and !ideo codecs are currentl& a!ailable" some inmultiple implementations. 8ecause codecs often borro' tec#ni9ues and strateies ofprocessin sound and imae from eac# ot#er" t#e& #a!e tanled eneaoloies.

:ea!in aside t#e tanle of relations bet'een different codecs and !ideo tec#noloies"e!en one codec" t#e 'ell-establis#ed and uncontentious M/-2 codin standard" ise$traordinaril& comple$. M/-2 (a.k.a. .262) desinates a 'ell-establis#ed set ofencodin and decodin procedures for diital audio and !ideo formalised as a standard(;>) in t#e mid ??0s. %#e standards for M/-2 are 'idel& described. Man&diarams" definitions and e$planations of codin and decodin t#e bitstream are a!ailablein print and online (


2/6

2

b& entertainment" telecommunications" o!ernment" academic and militar& o'nersaccordin to ikipedia (ikipedia" 2006)). %#e lare patent pool attests to t#e economicsinificance of M/-2 codecs. As t#e basis of commercial +,+s" t#e transmission formatfor satellite and cable diital tele!ision (+,8 and A%


3/6

3

t#e earlier" lo'-bitrate standard" M/- (;


4/6

1

;n t#e source code for codecs suc# as t#eora" t#e densit& of matri$ or arra& manipulationstands out in t#e transform p#ase of t#e encodin. %#ousands of blocks in eac# pictureundero +C%. ;n contrast to filmJs use of linear se9uences of frames" or tele!ision and

!ideoJs interlacin of scan-lines to compose imaes" transforms suc# as +C% subectimaes to #i#l& intricate reorderins.


5/6

F

motion !ectors describin relati!e mo!ements of blocks. %#e IpictureJ after encodin isnot#in but a series of !ectors describin '#at #appens to blocks. At t#e decodin end" anM/ decoder turns t#e streams of !ectors back into arranements of blocks mo!inbet'een frames.

From complicated to composite

All t#is is a bit complicated. Motion prediction takes time. %#e ratio of intra-frame andinter-frame pictures in a i!en bitstream depends on '#ere t#e encodin is done and t#eband'idt# of t#e e$pected transmission c#annel (+,+" 3/ cellp#one" satellite diital %,"+%," t#e internet" etc). ;n an M/ data-stream" t#e precise mi$ture of different frame-t&pes (;" -for'ard and 8-back'ard) is defined at encodin time in t#e /roup of ictures

(/O) structure. ;t is t&picall& 2 or F pictures in a se9uence suc# as;8888888888. One intra-coded picture is follo'ed b& a dozen or soblock motion-compensation picture. %#eir order in t#e bitstream does not correlatedirectl& 'it# t#e order of displa&. %#e combination of for'ard-prediction and back'ard-prediction found in t#e /O means t#at t#e M/ bitstream effecti!el& treats imaes asmassi!e doubl&-linked list (Nnut#" ??D" 2>0). %#e ratio of different frame t&pes to eac#ot#er affects t#e encodin time because motion compensation is muc# slo'er to encodet#an t#e #i#l& optimised block transforms. Codecs must make direct tradeoffs bet'eencomputational time and space. %#e tradeoffs sometimes result in artifacts !isible on screenas suc# as blockin and mosaic effects. At times" motion prediction does not 'ork. A

c#ane in camera s#ot" t#e effect of an edit" mi#t mean t#at no blocks are s#ared bet'eenadacent frames. ;n t#at case" t#e codec falls back on intra-frame encodin.

Man& of t#e complications of t#e codecs arise because t#e& link !er& different scales oftec#noloical infrastructure" markets and embodied cultural practice. Codecs connectnet'ork band'idt# constraints (a commerciall& marketed ser!ice)" con!entions ofspectators#ip" embodied conition" and media-#istorical forms. Codecs respond to t#eeconomic-tec#nical cappin of band'idt# in telecommunications markets. %#eir teeminpatent pools reflect #i# estimates of t#eir !alue. %#e Beneric met#odB of encodin and

decodin imaes for transmission relates !er& closel& to t#e constraints and conditions oftelecommunications and media net'orks. As a con!ention" t#e M/-2 standard citese$plicitl& or implicitl& a reat number of p#&sical 9uantities ranin from screendimensions" resolution and colour models t#rou# to net'ork and transmissioninfrastructures to t#e clock rates and memor& sizes of semiconductor and data storaetec#noloies. et t#e codec must propaate li#t" colour and sound on screen 'it#incalibrated ps&c#o-perceptual limits. Anal&sin suc# intersections re9uires 'a&s ofarticulatin di!erse realities.


6/6

6

References

ffmpe (2006)" M/ Multimedia -" ;. ;. (??F). ;nformation tec#nolo& - /eneric codin of mo!in picturesand associated audio information* -2 (??F) ;nformation tec#nolo& - /eneric codin of mo!in pictures andassociated audio information* ,ideo

Nittler" . (??3) +raculas ,ermac#tnis %ec#nisc#e

mackenzie, adrian 2006 - codecs

Documents