CAC ANNUAL MEETING

DATA HIDING IN COMPRESSED MULTIMEDIA SIGNALS

Bijan Mobasseri, PIS. R. NelaturyDom CinalliDan CrossAaron EvansColin O’Connor Sathya Akunuru

ECE DepartmentVillanova UniversityVillanova, PA 19085

October 30, 2002

2/46

Background info

• Funding agency: The US Air Force Office of Scientific Research(AFOSR)

• Monitor: AFRL/IFEC, Information Directorate, Rome, NY

• Project: Smart Digital Video• PI: Bijan Mobasseri

3/46

Outline

• Data hiding/watermarking requirements• Established watermarking approaches• Project summaries:

– Compressed media watermarking– Video authentication through self-watermarking– Lossless watermarking using error-resilient coding– Time-frequency watermarking– Metadata embedding

4/46

Data hiding/watermarking requirements

• Data hiding must at least meet the following three conditions:– Transparency; no visible impact on cover signal– Robustness; filtering, compression, cropping– Security; must assume the algorithm is known

• Places to hide data are:– Spatial- pixel amplitudes, LSB, QIM– Transform domain- spread spectrum, Fourier/wavelet, LPM– Joint- time/frequency distribution

5/46

Applications of watermarking

• Here are few, and growing list– Copyright protection- prevent unauthorized duplication– Fingerprinting-to find out who gave it away– Copy protection- to keep a tab on the number of copies

made– Broadcast monitoring- automatic monitoring of commercials– Authentication- insuring data integrity and tamper

resistance/detection– Indexing- helping multimedia search capability– Metadata hiding- embedding patient’s records in their

medical images– Data hiding- covert communications in plain sight

6/46

Basic idea

7/46

Watermark Embedding and Extraction

• Cover image: f• Watermark: w• Embedding

function:E• Secret key:kStego image=S=E(f,w,k)

• AuthenticationT(S): tampered

signal

ˆ w =g(T (S))

ρ=<w, ˆ w >

ρ>Threshold ⇒

watermark present

8/46

Detector response to forgeries

• Let’s say someone attempts to forge a watermarked document using their own signature

• We then have

• None of the two terms register significant response

Iw =I +kW*

λ =<Iw,W x,y( )>=

< I +kW *( ),W >=

<I,W >+k <W,W * >

9/46

Block Diagram

• Embedding

• Watermark extraction

SOURCEWATER

MARKING

TAMPERINGCORRUPTED WM SOURCE

CORRUPTED WM SOURCE -

SOURCE

DISTORTEDWM X

ORIGINALWM

10/46

Quality of extracted watermark

11/46

Trade-offs

12/46

LSB watermarking

• Probably the earliest attempt at watermarking was to flip the least significant bit of each pixel

• LSB being at noise level, would have no impact on quality. However, the slightest change in pixel intensity would make the watermark unreadable

Pixel 1 Pixel 2

LSB

13/46

BITPLANE WATERMAKRING

14/46

Hiding information in 24 bit images

• A 1024x768 24-bit color image can potentially hide 2,359,296 bits

• How would you hide the letter A? “A” can be hidden in the LSB of 3 pixels such as

• The binary value of A is 10000011. Changed bits are shown

00100111 11101001 1100100000100111 11001000 1110100111001000 00100111 11101001

00100111 11101000 1100100000100111 11001000 1110100111001000 00100111 11101001

15/46

Transform domain watermarking

• Spatial watermarking is fast but brittle.• It is best to do watermarking in

transformed domains DFT– DCT– DWT

• The first successful implementation was done by Cox et al at NEC/Princeton under spread spectrum watermarking

16/46

Basic idea

• Instead of tweaking pixels, alter selected coefficients of image transform

• Then do inverse transform. This way, watermark spreads throughout the image affecting every pixel in some way

• It is not possible to find the watermark in the spatial domain

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

17/46

SPREAD SPECTRUM WATERMARKING(Cox, NEC)

555555 55555555 55555555 55

555555 55

DCT

550 10 -1 0

2 3 2 0

-1 2 0 0

0 0 0 0

16 1 0 0

0 -1 0 0

-1 2 0 0

0 0 0 0

Quan

16 1 -1 0

1 -1 0 0

-1 2 0 0

0 0 0 0

Original frame

18/46

Challenges

• Which DCT coefficients should you choose?

• We have to worry about two competing requirements– Robustness - means low

frequency terms should be modified

– Imperceptibility - means low frequency terms should be avoided

19/46

DFT watermarking

20/46

Watermarking DWT

VIDEO WATERMAKING

22/46

Tampering scenarios: cut and splice of surveillance video

• A block of frames removed and video spliced

• Video must be embedded with proper sequencing codes so as to reveal the breakage

23/46

Cut, insert and splice

• Incriminating/sensitive portion is removed and replaced

24/46

Cut, swap and splice

25/46

Collusion attack

26/46

MPEG bitstream syntax

27/46

Embedding watermark bits in VLCs

• Variable length codes are the lynchpin of MPEG

• There is a subset of MPEG VLC codes that represent identical runs but differ in level by

just one

From: Langelaar et al, IEEE SP Magazine

September 2000

28/46

Data hiding capacities

Total lc-VLCs Total VLCs24008 204493

655548 4321760

61298 461027

3614 61985

345315 3290373

24412 204362

run 0 0 0 0 0 0 0level 5 6 8 9 10 11 12406974ai1.mpeg 3734 2630 1835 1423 1122 827 1036% occurrence 15.6 11.0 7.6 5.9 4.7 3.4 4.3status.mpeg 136422 90706 60594 46844 36066 32513 25822% occurrence 20.8 13.8 9.2 7.1 5.5 5.0 3.9avalon.mpeg 13080 9506 5420 4353 3405 2865 2341% occurrence 21.3 15.5 8.8 7.1 5.6 4.7 3.8gtscrush.mpeg 1056 700 281 246 193 131 143% occurrence 29.2 19.4 7.8 6.8 5.3 3.6 4.0final-days.mpg 78900 54245 30487 24116 18751 15464 12791% occurrence 22.8 15.7 8.8 7.0 5.4 4.5 3.7simp.mpg 5997 4007 2438 1441 1627 1225 807% occurrence 24.6 16.4 10.0 5.9 6.7 5.0 3.3

average % occurrence 22.4 15.3 8.7 6.6 5.5 4.4 3.8

SELF-WATERMARKING*

*D. Cross, B. Mobasseri, “Watermarking for self-authentication of compressed video,” IEEE ICIP2002,

Rochester, NY, September 22-25, 2002,

30/46

Self-watermarking:the concept

• In self-watermarking, the watermark is extracted from the source itself

• Self-watermarking prevents watermark pirating

• Most work on self-watermarking has been done on images.

31/46

Self-watermarking of compressed video

1 0

VLC(0,5)

VLC(0,16)

VLC(1,15)

VLC(0,6)

VLC(1,10)

VLC(1,11)

VLC(0,12)

Lossless Watermarking of Compressed Media*

*B. Mobasseri, D. Cinalli “Watermarking of Compressed Multimedia using Error-Resilient VLCs,” MMSP02, December 9-11, 2002

33/46

The idea:watermark as intentional bit errors

• A close look reveals that watermarking of VLCs is essentially equivalent to channel errors.

• Bit errors and watermark bits have identical impact. They both cause bit errors in affected VLCs.

34/46

The solution:lossless watermarking

• Embed watermark bits in the VLCs as controlled bit errors

• MPEG-2 VLCs, however, have no inherent error protection. Any bit error will cause detection failure up to start code

• Bidirectionally decodable codewords are capable of isolating and reversing channel errors

• This approach leads to lossless watermarking

35/46

Bi-directional VLCs

• Each VLC is represented twice in the new bitstream. It is this property that allows error resiliency

• Burst error shall not be so long to simultaneously affect the same bit of identical VLC

36/46

Watermarking capacity

• If watermarking begins with the first bit of the VLC and L=l, every bit of the VLC may be watermarked, then

C=L bits/packet• We define packet as one

macroblock

37/46

Data

Non- Coded MBsVideo # Coded MBs

Intra & InterInter Skipped

Avg. # ofVLCs / MB

Avg. # ofBits / MB

Avg. % increase insize per MB with

thresholdCapacity

1 902 7,153 45 4.83 25.33 .87% 4,5102 57,326 66,162 40,240 8.02 42.07 13.6% 286,6303 309,740 99,981 42,391 15.38 76.76 18.89% 1,548,700

TIME-FREQUENCY WATERMARKING

B. Mobasseri, “Digital watermarking in joint time-frequency domain,”,IEEE ICIP, Rochester, NY, September, 22-25, 2002

39/46

The Idea

40/46

TF watermarking

0 10 20 30 40 50 60 70 80-2

-1.5

-1

-0.5

0

0.5

1

1.5

2x 10

-3

0

20

40

60

80

100

120

140

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

-505

x 105

original WVD

€

Wx nT , f( ) = 2T x n + m( )m∑ x* n −m( ) exp(− j4πfmT ), f ≤

1

4T

WD +

€

Y t, f( ) = X t, f( ) + w t, f( ); t , f ∈Ω{ }

D

WD-1

0 10 20 30 40 50 60 70 80-2

-1.5

-1

-0.5

0

0.5

1

1.5

2x 10

-3

WD

0

20

40

60

80

100

120

140

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

-505

x 105

original WVD

WM

JPEG

41/46

Results

42/46

Effect of compression

JPEG:Q=5

Metadata Embedding

44/46

Background

• Video images & metadata recorded and handled as two separate streams

– Storage overhead– Bookkeeping issues– Accuracy and human error– Cumbersome to display

• It would be nice to permanently attach metadata to video and make it available during playback

MetadataVideo

45/46

Sample Metadata and video footage

Surveillance VideoXML Coded Metadata

THE END