compression standard for interactive imagingread.pudn.com/downloads99/ebook/406102/jpeg2000... ·...

JPEG2000JPEG2000

Compression standard forCompression standard forinteractive imaginginteractive imaging

David TaubmanDavid TaubmanThe University of New South WalesThe University of New South Wales

27 March, 200327 March, 2003 © David Taubman, UNSW© David Taubman, UNSW

Image Compression Image Compression PreliminariesPreliminaries

!! Array of sample intensities (e.g. 8 bits)Array of sample intensities (e.g. 8 bits)–– Colour: red, green and blue samplesColour: red, green and blue samples–– 64 x 64 (small thumbnail)64 x 64 (small thumbnail)–– 1024 x 1024 (small photographic)1024 x 1024 (small photographic)–– 64k x 64k (typical GIS/military image) 64k x 64k (typical GIS/military image)

!! Lossless compressionLossless compression–– Only ~2:1 compression (photographic)Only ~2:1 compression (photographic)

!! Lossy compressionLossy compression–– Allows up to ~100:1 compressionAllows up to ~100:1 compression–– Can’t recover original samples exactlyCan’t recover original samples exactly


Development of JPEG2000Development of JPEG2000!! Developed by “JPEG” Working GroupDeveloped by “JPEG” Working Group

–– ISO/IEC JTC1/SC29/WG1ISO/IEC JTC1/SC29/WG1–– Major international effort!Major international effort!

!! 1996: work item proposed by Ricoh1996: work item proposed by Ricoh!! Nov 1997: 24 submissions evaluatedNov 1997: 24 submissions evaluated

–– WTCQ selected after extensive testingWTCQ selected after extensive testing

!! Nov 1998: major algorithm changeNov 1998: major algorithm change–– EBCOT proposal adoptedEBCOT proposal adopted

!! Jan 2001: part 1 became IS 15444Jan 2001: part 1 became IS 15444--11!! Parts 1Parts 1--6 complete; 86 complete; 8--11 ongoing11 ongoing


Some Benefits of JPEG2000Some Benefits of JPEG2000!! Compress once Compress once –– decompress many waysdecompress many ways

–– one codestream, many applications (example)one codestream, many applications (example)

!! Flexible information progressionsFlexible information progressions–– by quality, by resolution (size), top down, …by quality, by resolution (size), top down, …

!! StateState--ofof--thethe--art compression efficiencyart compression efficiency–– smallest compressed size, for given qualitysmallest compressed size, for given quality

!! Compressed domain editingCompressed domain editing–– crop images without accumulation of errorscrop images without accumulation of errors

!! Content not limited by rendering devicesContent not limited by rendering devices–– Limited hardware still handles complex imagesLimited hardware still handles complex images


Principles:Principles:Statistical Redundancy (1)Statistical Redundancy (1)

!! Importance of prior expectationsImportance of prior expectations–– Consider 4 events (e.g., snow, hail, rain, fine)Consider 4 events (e.g., snow, hail, rain, fine)

!! If we know it is summer, snow & hail highly unlikelyIf we know it is summer, snow & hail highly unlikely!! If we know precipitation is rare, only “fine” is likelyIf we know precipitation is rare, only “fine” is likely

–– No need to spend 2 bits indicating the outcomeNo need to spend 2 bits indicating the outcome!! e.g., send 0 if fine; 10 if rain; 110 if snow; 111 if haile.g., send 0 if fine; 10 if rain; 110 if snow; 111 if hail!! expect to send only 1 bit (0=fine) most of the timeexpect to send only 1 bit (0=fine) most of the time

!! Lessons to learnLessons to learn–– Average bitAverage bit--raterate depends on prior likelihoodsdepends on prior likelihoods–– Likelihoods depend on Likelihoods depend on contextcontext

!! e.g., “it is summer” or “precipitation in region is rare”e.g., “it is summer” or “precipitation in region is rare”


Principles:Principles:Statistical Redundancy (2)Statistical Redundancy (2)

probabilityprobabilitymodelsmodels

eventeventencoderencoder

learning

learning

probabilityprobabilitymodelsmodels

eventeventdecoderdecoder

codedcodedbitbit--streamstream

contextcontextcontextcontextmodelingmodeling

contextcontextmodelingmodeling

expectedexpectedlikelihoodsmemorymemory memorymemorylikelihoods

!! Context modelingContext modeling–– exploit knowledge of events already encodedexploit knowledge of events already encoded

!! knowledge is also available at decoderknowledge is also available at decoder

!! Probability Probability modelingmodeling–– learn likelihoods of events in each contextlearn likelihoods of events in each context

!! use previously coded events in same contextuse previously coded events in same context

outputoutputevents

inputinputevents eventsevents


Need for QuantizationNeed for Quantization

!! Coded Coded eventsevents can be image sample valuescan be image sample values–– Code 8 bit/sample (bps) image with < 8 bpsCode 8 bit/sample (bps) image with < 8 bps–– Process is fully reversible (lossless)Process is fully reversible (lossless)–– ButBut compression ratio not usually large compression ratio not usually large –– ~2:1~2:1

!! Need to quantize sample values into binsNeed to quantize sample values into bins–– Reduce coded bitsReduce coded bits–– Cost is distortionCost is distortion–– Click for exampleClick for example

!! 3,1.5,0.8,0.5 bps3,1.5,0.8,0.5 bpsbin 0bin 0bin 1bin 1bin 2bin 2bin 3bin 3bin 4bin 4

sam

ple

val

ue

sam

ple

val

ue bin indexbin index

encoderencoder

bin 0bin 0bin 1bin 1bin 2bin 2bin 3bin 3bin 4bin 4

outp

ut

valu

eoutp

ut

valu

e

quantizerquantizer dequantizerdequantizer

bin indexbin indexdecoderdecoder

bitsbits


Principles:Principles:Wavelet Transform (1)Wavelet Transform (1)


!! Subband images fromSubband images from–– lowlow-- & high& high--pass filteringpass filtering–– plus subplus sub--samplingsampling

!! Num samples preservedNum samples preserved–– same as original imagesame as original image

!! Transform is invertibleTransform is invertible–– get original image back get original image back

from subband imagesfrom subband images

!! Quantize subbandsQuantize subbands–– Click for exampleClick for example

!! 1,.5,.25,.12,.06,.03 bps1,.5,.25,.12,.06,.03 bps



Principles:Principles:Resolution Scalability (1)Resolution Scalability (1)

synthesissynthesis

resolution 0resolution 0new detailsnew details

forforresolution 1resolution 1 Embedded codestreamEmbedded codestream



synthesissynthesis


forforresolution 1resolution 1

new detailsnew detailsforfor

resolution 2resolution 2



!! Can choose image sizeCan choose image size–– after compressionafter compression–– prior to decompressionprior to decompression–– during transmission


forforresolution 1resolution 1

new detailsnew detailsforfor

resolution 2resolution 2

Embedded codestreamEmbedded codestream

during transmission


Principles:Principles:Embedded Block Coding (1)Embedded Block Coding (1)

embeddedembeddedcodecode--blockblock

bitbit--streamsstreams

embeddedembeddedcodecode--blockblock

bitbit--streamsstreams

LLLL22

LHLH22 HHHH22

HLHL22

HLHL11

HHHH11LHLH11


Principles:Principles:Embedded Block Coding (2)Embedded Block Coding (2)

!! Each subband image divided into blocksEach subband image divided into blocks–– typically 32x32 or 64x64 samples per blocktypically 32x32 or 64x64 samples per block–– each block coded independentlyeach block coded independently

!! Embedded block coding algorithmEmbedded block coding algorithm–– sample accuracy progressively refinedsample accuracy progressively refined

!! quantizer quantizer bin size effectively shrinksbin size effectively shrinks as bitas bit--stream stream progressesprogresses

!! ultimate representation lossless (no quantization)ultimate representation lossless (no quantization)

–– highly efficient representationhighly efficient representation!! sophisticated context modeling within each blocksophisticated context modeling within each block!! adaptive probability modeling within each blockadaptive probability modeling within each block


Principles:Principles:Abstract Quality LayersAbstract Quality Layers

Block 1Block 1bitbit--streamstream






!! Layer 1Layer 1–– coarse quantizationcoarse quantization–– low quality imagerylow quality imagery

!! Layer 2Layer 2–– finer quantizationfiner quantization–– better imagerybetter imagery

!! Layer 3Layer 3–– finest quantizationfinest quantization–– best imagerybest imagery

!! Block contributions to each layer optimizedBlock contributions to each layer optimized–– Each block bitEach block bit--stream optimally truncatedstream optimally truncated–– Permits custom interpretation of image qualityPermits custom interpretation of image quality


Principles:Principles:Dimensions of ScalabilityDimensions of Scalability

subset havingsubset havinglow resolution,low resolution,

at very high qualityat very high quality

subset havingsubset havingmoderate resolution,moderate resolution,

with coarse quantizationwith coarse quantizationRes 0Res 0 DetailsDetails

for Res 1for Res 1

Laye

r 1La

yer 1

Laye

r 2La

yer 2

Laye

r 3La

yer 3

resolutionresolution

Resolution Scalable EmbeddingResolution Scalable Embedding

Qua

lity

Scal

able

Em

bedd

ing

Qua

lity

Scal

able

Em

bedd

ing

Resolution and DistortionResolution and DistortionScalable EmbeddingScalable Embedding

DetailsDetailsfor Res 2for Res 2

qual

ity la

yers

qual

ity la

yers


Principles:Principles:Spatial Access with CodeSpatial Access with Code--BlocksBlocks


Information Sequencing:Information Sequencing:Layer Dominant (LRCP)Layer Dominant (LRCP)

more bitsmore bits


Information Sequencing:Information Sequencing:Resolution Dominant (RLCP)Resolution Dominant (RLCP)

more bitsmore bits


Information Sequencing:Information Sequencing:Position Dominant (PCRL)Position Dominant (PCRL)

more bitsmore bits


Information Sequencing:Information Sequencing:Component Dominant (CPRL)Component Dominant (CPRL)

more bitsmore bits


Information SequencingInformation SequencingPrinciplesPrinciples

!! Fundamental mechanismFundamental mechanism–– sequencing of codesequencing of code--block layer contributionsblock layer contributions

!! Signalling mechanismsSignalling mechanisms–– codecode--blocks partitioned into blocks partitioned into precinctsprecincts

!! 1 spatial region of 1 resolution, in 1 component1 spatial region of 1 resolution, in 1 component

–– precincts divided into precincts divided into packetspackets!! 1 layer of 1 precinct1 layer of 1 precinct

–– packet progressionpacket progression follows selected schemefollows selected scheme

!! Can always rearrange packet sequenceCan always rearrange packet sequence–– choose sequence best suited to compressorchoose sequence best suited to compressor–– later later rearrange to suitrearrange to suit consumer applicationsconsumer applications


JPEG2000 vs. JPEG:JPEG2000 vs. JPEG:Visual PerformanceVisual Performance

1.25 bpp1.25 bpp

BikeBikeWomanWoman

Average of 6 imagesAverage of 6 images

1.0 bpp1.0 bpp

0.75 bpp0.75 bpp

0.5 bpp0.5 bpp

0.25 bpp0.25 bpp

0.25 bpp0.25 bpp

JPEG2000 bitJPEG2000 bit--rate (bits per pixel)rate (bits per pixel)

Equiv

alen

t JP

EG

bit

Equiv

alen

t JP

EG

bit-- r

ate

rate

Results based on 6 observersResults based on 6 observers

Source:Source: Chinen & Chien, “Visual evaluation of JPEG2000 color image compChinen & Chien, “Visual evaluation of JPEG2000 color image compression ression performance,” ISO/IEC JTC1/SC29/WG1 Technical Report N1583.performance,” ISO/IEC JTC1/SC29/WG1 Technical Report N1583.

1.25 bpp1.25 bpp1.0 bpp1.0 bpp0.75 bpp0.75 bpp0.5 bpp0.5 bpp


JPEG2000 vs. JPEG:JPEG2000 vs. JPEG:Blocking ArtefactsBlocking Artefacts

JPEG2000 @ 0.25 bits/pixelJPEG2000 @ 0.25 bits/pixel JPEG @ 0.25 bits/pixelJPEG @ 0.25 bits/pixel


JPEG2000 vs. JPEG:JPEG2000 vs. JPEG:FeaturesFeatures

!! Rich information progressionsRich information progressions–– JPEG: also offers progressive refinementJPEG: also offers progressive refinement

!! But, done with But, done with not internot inter--operableoperable modesmodes

!! Progressive lossy to lossless compressionProgressive lossy to lossless compression–– JPEG: no integrated lossy/lossless capabilityJPEG: no integrated lossy/lossless capability

!! Spatial random accessSpatial random access–– JPEG: need to tile imageJPEG: need to tile image

!! Lossless image croppingLossless image cropping–– JPEG: editing JPEG: editing ⇒⇒ quantization error buildquantization error build--upup

!! Exact rate control in one shotExact rate control in one shot


Complexity Issues:Complexity Issues:Kakadu Speed (2GHz)Kakadu Speed (2GHz)

!! Large Image (24 bpp colour)Large Image (24 bpp colour)–– Compress to 0.5 bits/pel Compress to 0.5 bits/pel ⇒⇒ 3.0 (4.0) Mpels/s3.0 (4.0) Mpels/s

!! decompress decompress ⇒⇒ 10 Mpels/s10 Mpels/s

–– Compress to 1.0 bits/pel Compress to 1.0 bits/pel ⇒⇒ 2.4 (3.2) Mpels/s2.4 (3.2) Mpels/s!! decompress decompress ⇒⇒ 7.5 Mpels/s7.5 Mpels/s

–– Compress to lossless Compress to lossless ⇒⇒ > 1 Mpels/s> 1 Mpels/s!! decompress decompress ⇒⇒ 1.3 Mpels/s1.3 Mpels/s

!! Video (352x240, 4:2:0 colour)Video (352x240, 4:2:0 colour)–– Compress to 0.5 bits/pel Compress to 0.5 bits/pel ⇒⇒ 6060 frames/sframes/s

!! decompress decompress ⇒⇒ 160160 frames/sframes/s

–– Compress to 1.0 bits/pel Compress to 1.0 bits/pel ⇒⇒ 45 frames/s45 frames/s!! decompress decompress ⇒⇒ 110 frames/s110 frames/s


Complexity Issues:Complexity Issues:MemoryMemory

!! Transform and codeTransform and code--block bufferingblock buffering–– Need Need 200 to 400 image lines200 to 400 image lines equivalentequivalent–– Does not depend on image heightDoes not depend on image height

!! CodeCode--block bitblock bit--stream bufferingstream buffering–– Most packet progression orders:Most packet progression orders:

!! About 20% more than final size of About 20% more than final size of entire codeentire code--streamstream

–– TopTop--down packet progression order:down packet progression order:!! Equivalent to about Equivalent to about 1000 lines of compressed data1000 lines of compressed data

!! Much more memory than JPEGMuch more memory than JPEG–– But, no need to buffer entire imageBut, no need to buffer entire image


JPEG2000 File FormatsJPEG2000 File Formats!! JPEG2000 algorithm produces codeJPEG2000 algorithm produces code--streamstream

–– no interpretation of compressed sample valuesno interpretation of compressed sample values

!! JP2 file format (simple wrapper)JP2 file format (simple wrapper)–– colour rendering info (1/3 colour ICC profiles)colour rendering info (1/3 colour ICC profiles)–– other rendering info (alpha, resolution/size)other rendering info (alpha, resolution/size)

!! JPX file format (extends JP2)JPX file format (extends JP2)–– rich colour descriptions (arbitrary ICC profiles)rich colour descriptions (arbitrary ICC profiles)–– multiple codemultiple code--streams, animation, IPR, XML, …streams, animation, IPR, XML, …

!! JPM file format (compound documents)JPM file format (compound documents)!! MJ2 file format (Motion JPEG2000)MJ2 file format (Motion JPEG2000)

–– one codeone code--stream per framestream per frame


Parts to the StandardParts to the Standard!! Part 1: core technology (+ JP2 file format)Part 1: core technology (+ JP2 file format)!! Part 2: extended coding tools (+ JPX)Part 2: extended coding tools (+ JPX)!! Part 3 (12 & 13): motion JPEG2000 (MJ2)Part 3 (12 & 13): motion JPEG2000 (MJ2)!! Part 4: conformance testing for Part 1Part 4: conformance testing for Part 1!! Part 5: reference software (JJ2000+Jasper)Part 5: reference software (JJ2000+Jasper)!! Part 6: compound documents (JPM)Part 6: compound documents (JPM)!! Part 8: JPSEC (encryption, watermarking)Part 8: JPSEC (encryption, watermarking)!! Part 9: JPIP (interactive protocols)Part 9: JPIP (interactive protocols)!! Part 10: JP3D (hyperspectral/volumetric)Part 10: JP3D (hyperspectral/volumetric)!! Part 11: JPWL (robust comms for wireless)Part 11: JPWL (robust comms for wireless)


Interactive Distribution with JPIPInteractive Distribution with JPIP(JPEG2000 Internet Protocol)(JPEG2000 Internet Protocol)

SourceSourceCodeCode--streamstream

GraphicalGraphicalUser InterfaceUser Interface

!! DemonstrationDemonstration–– jpip://dstjpip://dst--m/wharf3.jp2m/wharf3.jp2–– jpip://dstjpip://dst--m/phoenix.jp2m/phoenix.jp2

Client CacheClient CacheCache ModelCache Model

JPIPJPIPServerServer

JPIPJPIPClientClientmessagesmessages

(precinct byte ranges)(precinct byte ranges)

update windowupdate windowof interestof interest

periodically renderdisplayed region

load ondemand

windowof interest

RegionRegionDecompressorDecompressor

augment model augment cache

NetworkNetwork


Applications for JPEG2000Applications for JPEG2000!! InternetInternet!! MedicalMedical

–– adopted by DICOM: medical imaging standardadopted by DICOM: medical imaging standard

!! Military/Surveillance/GISMilitary/Surveillance/GIS–– 7 of 8 military image standards to require J2K7 of 8 military image standards to require J2K

!! Digital Digital CamerasCameras–– lossless image cropping; support for internetlossless image cropping; support for internet

!! Scanners/printersScanners/printers–– chips appearing for these markets alreadychips appearing for these markets already

!! Mobile Mobile DevicesDevices–– efficient interactive browsing of large imagesefficient interactive browsing of large images

!! Digital Cinema and other VideoDigital Cinema and other Video–– produce/archive/distribute; Yahoo Messanger; etc.produce/archive/distribute; Yahoo Messanger; etc.


Available Available Implementations:Implementations:SoftwareSoftware

!! JJ2000: Java (Canon/Ericsson/EPFL)JJ2000: Java (Canon/Ericsson/EPFL)–– free reference softwarefree reference software

!! Jasper: “C” (Adams/Imagepower/UBC)Jasper: “C” (Adams/Imagepower/UBC)–– free reference softwarefree reference software

!! Kakadu: “C++” (Taubman/UNSW)Kakadu: “C++” (Taubman/UNSW)–– probably most popular commercial systemprobably most popular commercial system

!! Luratech/AlgoVisionLuratech/AlgoVision!! Aware Inc.Aware Inc.!! Ricoh (TRUEW)Ricoh (TRUEW)!! ImagepowerImagepower


Available Implementations:Available Implementations:HardwareHardware

!! Analog DevicesAnalog Devices–– ADVADV--JP2000 was first to implement (partially)JP2000 was first to implement (partially)

!! AmphionAmphion–– IP Core claims 60 Msamples/secondIP Core claims 60 Msamples/second–– May do only the wavelet transform partMay do only the wavelet transform part

!! Picture Elements Inc.Picture Elements Inc.–– Nearing release of highNearing release of high--performance performance

compressor for scannerscompressor for scanners

!! Ricoh, Imagepower, …Ricoh, Imagepower, …–– Working on chips; status unknownWorking on chips; status unknown


Useful ReferencesUseful References

!! WebWeb--sitesite–– www.jpeg.orgwww.jpeg.org is an excellent starting pointis an excellent starting point–– www.crc.ricoh.com/~gormish/jpeg2000.htmlwww.crc.ricoh.com/~gormish/jpeg2000.html

!! BookBook–– “JPEG2000: image compression fundamentals, “JPEG2000: image compression fundamentals,

standards and practice,” Kluwer, by Taubman standards and practice,” Kluwer, by Taubman and Marcellinand Marcellin

!! General ArticlesGeneral Articles–– IEEE Signal Processing Magazine, Sept 2001IEEE Signal Processing Magazine, Sept 2001–– Taubman & Marcellin, Proc. IEEE, August 2002Taubman & Marcellin, Proc. IEEE, August 2002

compression standard for interactive imagingread.pudn.com/downloads99/ebook/406102/jpeg2000... ·...

Documents